2024-04-25T18:22:18.781 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T18:22:18.781 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614 -- /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614/orig.config.yaml 2024-04-25T18:22:18.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T18:22:18.928 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7673614' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 7797745e6d8b3ac036504dc61491d368615024da owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4419 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 7797745e6d8b3ac036504dc61491d368615024da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBtLquDmnzWLI7/a0Pegb0caYyJ6TT7NFuELdJvksl2kYlm40L0T+HQg4QZ+1DuhB6/9gYPNADj6KhjNDoQFVhM= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPKokEoAPGlXidTm96brqEZqrR7QVzJqwiJ4hr6x7pZ47ri28vg5eNmjpGnZjoc2B93uxQYFlv/SekHfdTXcBLQ= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLEndUvYqYuYwgvvFy78a1Q6MNAKu2YM9O9qUmz9eFpDVEgxn5VzbENb1+R0iVsfPhQf5IAHvFcJe1lkn9AXEfs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_17:43:55 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T18:22:18.928 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa; will attempt to use it 2024-04-25T18:22:18.929 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa/tasks 2024-04-25T18:22:18.929 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T18:22:18.931 INFO:teuthology.task.internal:Checking packages... 2024-04-25T18:22:18.953 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T18:22:18.954 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T18:22:18.954 INFO:teuthology.packaging:ref: None 2024-04-25T18:22:18.954 INFO:teuthology.packaging:tag: None 2024-04-25T18:22:18.954 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T18:22:18.954 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:22:18.954 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T18:22:19.121 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T18:22:19.123 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T18:22:19.129 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T18:22:19.129 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T18:22:19.138 INFO:teuthology.task.internal:Saving configuration 2024-04-25T18:22:19.160 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T18:22:19.166 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T18:22:19.191 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:17:01.681102', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBtLquDmnzWLI7/a0Pegb0caYyJ6TT7NFuELdJvksl2kYlm40L0T+HQg4QZ+1DuhB6/9gYPNADj6KhjNDoQFVhM='} 2024-04-25T18:22:19.211 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:17:01.682846', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPKokEoAPGlXidTm96brqEZqrR7QVzJqwiJ4hr6x7pZ47ri28vg5eNmjpGnZjoc2B93uxQYFlv/SekHfdTXcBLQ='} 2024-04-25T18:22:19.235 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:17:01.684937', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLEndUvYqYuYwgvvFy78a1Q6MNAKu2YM9O9qUmz9eFpDVEgxn5VzbENb1+R0iVsfPhQf5IAHvFcJe1lkn9AXEfs='} 2024-04-25T18:22:19.235 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T18:22:19.242 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T18:22:19.242 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T18:22:19.242 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T18:22:19.242 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T18:22:19.363 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc725f420d0>, signals=[15]) 2024-04-25T18:22:19.364 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T18:22:19.370 INFO:teuthology.task.internal:Opening connections... 2024-04-25T18:22:19.370 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T18:22:19.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:22:19.462 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-04-25T18:22:19.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:22:19.535 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T18:22:19.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:22:19.608 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T18:22:19.616 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-04-25T18:22:19.635 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-04-25T18:22:19.636 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-04-25T18:22:19.693 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-04-25T18:22:19.693 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-04-25T18:22:19.693 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-04-25T18:22:19.693 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-04-25T18:22:19.693 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-04-25T18:22:19.693 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-04-25T18:22:19.693 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T18:22:19.693 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-04-25T18:22:19.693 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-04-25T18:22:19.693 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T18:22:19.694 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-04-25T18:22:19.694 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T18:22:19.694 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T18:22:19.694 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T18:22:19.694 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-04-25T18:22:19.721 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-04-25T18:22:19.739 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-04-25T18:22:19.739 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-04-25T18:22:19.796 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-04-25T18:22:19.796 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-04-25T18:22:19.796 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-04-25T18:22:19.797 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-04-25T18:22:19.797 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-04-25T18:22:19.797 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-04-25T18:22:19.797 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T18:22:19.797 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-04-25T18:22:19.797 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-04-25T18:22:19.797 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T18:22:19.797 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-04-25T18:22:19.797 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T18:22:19.798 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T18:22:19.798 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T18:22:19.798 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-04-25T18:22:19.829 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-04-25T18:22:19.847 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-04-25T18:22:19.847 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-04-25T18:22:19.904 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-04-25T18:22:19.904 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-04-25T18:22:19.904 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-04-25T18:22:19.904 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-04-25T18:22:19.904 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-04-25T18:22:19.904 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-04-25T18:22:19.905 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T18:22:19.905 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-04-25T18:22:19.905 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-04-25T18:22:19.905 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T18:22:19.905 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-04-25T18:22:19.905 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T18:22:19.905 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T18:22:19.905 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T18:22:19.905 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-04-25T18:22:19.929 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T18:22:19.940 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T18:22:19.946 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T18:22:19.946 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-04-25T18:22:19.948 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-04-25T18:22:19.951 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-04-25T18:22:19.966 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T18:22:19.972 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T18:22:19.972 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-04-25T18:22:20.005 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-04-25T18:22:20.008 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-04-25T18:22:20.115 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T18:22:20.195 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T18:22:20.211 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T18:22:20.211 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-25T18:22:20.211 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-25T18:22:20.211 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T18:22:20.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:22:20.228 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T18:22:20.412 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:22:20.412 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T18:22:20.412 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-04-25T18:22:20.430 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-437.el9.x86_64 2024-04-25T18:22:20.430 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:22:20.430 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T18:22:20.431 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum upgrade -y linux-firmware 2024-04-25T18:22:23.315 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.0 MB 00:01 2024-04-25T18:22:25.658 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-25T18:22:29.842 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T18:22:31.307 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-25T18:22:33.644 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 21 MB 00:02 2024-04-25T18:22:38.700 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T18:22:40.685 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T18:22:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-04-25T18:22:40.686 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T18:22:40.803 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T18:22:40.803 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:22:41.030 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T18:22:41.031 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T18:22:41.033 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi112.front.sepia.ceph.com to complete... 2024-04-25T18:22:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25 18:22:53 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T18:22:53.168 INFO:teuthology.orchestra.run.smithi112.stderr:FINISHED --2024-04-25 18:22:53-- 2024-04-25T18:22:53.168 INFO:teuthology.orchestra.run.smithi112.stderr:Total wall clock time: 12s 2024-04-25T18:22:53.168 INFO:teuthology.orchestra.run.smithi112.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-25T18:22:53.170 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T18:22:53.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T18:22:55.314 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying... ######################################## 2024-04-25T18:22:55.533 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing... ######################################## 2024-04-25T18:22:55.645 INFO:teuthology.orchestra.run.smithi112.stdout:Updating / installing... 2024-04-25T18:23:03.543 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T18:23:42.591 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T18:23:42.592 DEBUG:teuthology.orchestra.run.smithi112:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T18:23:42.673 INFO:teuthology.orchestra.run.smithi112.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T18:23:42.673 INFO:teuthology.orchestra.run.smithi112.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T18:23:42.673 INFO:teuthology.orchestra.run.smithi112.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T18:23:42.673 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T18:23:42.674 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T18:23:42.674 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T18:23:42.674 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T18:23:42.674 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T18:23:42.674 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T18:23:42.674 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T18:23:42.674 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T18:23:42.674 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T18:23:42.674 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T18:23:42.674 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T18:23:42.675 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T18:23:42.676 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T18:23:42.677 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T18:23:42.678 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T18:23:42.679 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T18:23:42.680 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T18:23:42.680 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T18:23:42.680 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T18:23:42.680 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T18:23:42.680 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T18:23:42.680 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T18:23:42.680 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T18:23:42.680 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T18:23:42.680 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T18:23:42.680 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T18:23:42.681 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T18:23:42.681 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T18:23:42.681 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T18:23:42.681 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T18:23:42.681 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T18:23:42.681 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T18:23:42.681 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T18:23:42.681 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T18:23:42.682 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T18:23:42.682 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T18:23:42.682 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T18:23:42.682 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T18:23:42.682 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T18:23:42.682 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T18:23:42.682 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T18:23:42.682 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T18:23:42.682 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T18:23:42.683 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T18:23:42.683 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T18:23:42.683 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T18:23:42.683 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T18:23:42.683 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T18:23:42.683 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T18:23:42.683 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T18:23:42.683 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T18:23:42.684 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T18:23:42.684 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T18:23:42.684 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T18:23:42.684 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T18:23:42.684 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T18:23:42.684 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T18:23:42.684 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T18:23:42.684 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T18:23:42.685 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T18:23:42.685 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T18:23:42.685 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T18:23:42.685 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T18:23:42.685 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T18:23:42.685 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T18:23:42.685 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T18:23:42.685 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T18:23:42.685 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T18:23:42.686 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T18:23:42.686 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T18:23:42.686 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T18:23:42.686 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T18:23:42.686 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T18:23:42.686 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T18:23:42.686 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T18:23:42.687 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T18:23:42.687 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T18:23:42.687 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T18:23:42.687 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T18:23:42.687 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T18:23:42.687 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T18:23:42.687 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T18:23:42.687 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T18:23:42.687 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T18:23:42.688 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T18:23:42.688 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T18:23:42.688 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T18:23:42.688 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T18:23:42.688 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T18:23:42.688 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T18:23:42.688 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T18:23:42.688 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T18:23:42.689 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T18:23:42.689 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T18:23:42.689 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T18:23:42.689 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T18:23:42.689 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T18:23:42.689 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T18:23:42.689 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T18:23:42.689 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T18:23:42.689 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T18:23:42.690 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T18:23:42.690 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T18:23:42.690 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T18:23:42.690 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T18:23:42.690 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T18:23:42.690 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T18:23:42.690 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T18:23:42.690 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T18:23:42.691 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T18:23:42.691 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T18:23:42.691 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T18:23:42.691 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T18:23:42.691 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T18:23:42.691 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T18:23:42.691 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T18:23:42.691 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T18:23:42.692 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T18:23:42.692 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T18:23:42.692 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T18:23:42.692 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T18:23:42.692 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T18:23:42.692 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T18:23:42.692 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T18:23:42.692 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T18:23:42.692 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T18:23:42.693 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T18:23:42.693 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T18:23:42.693 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T18:23:42.693 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T18:23:42.693 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T18:23:42.693 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T18:23:42.694 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T18:23:42.695 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T18:23:42.696 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T18:23:42.697 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T18:23:42.698 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T18:23:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T18:23:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T18:23:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T18:23:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T18:23:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T18:23:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T18:23:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T18:23:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T18:23:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T18:23:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T18:23:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T18:23:42.700 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T18:23:42.701 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T18:23:42.702 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T18:23:42.702 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T18:23:42.702 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T18:23:42.702 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T18:23:42.702 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T18:23:42.702 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T18:23:42.702 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T18:23:42.702 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T18:23:42.702 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T18:23:42.703 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T18:23:42.703 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T18:23:42.703 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T18:23:42.703 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T18:23:42.703 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T18:23:42.703 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T18:23:42.703 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T18:23:42.704 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T18:23:42.704 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T18:23:42.704 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T18:23:42.704 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T18:23:42.704 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T18:23:42.704 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T18:23:42.704 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T18:23:42.705 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T18:23:42.706 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T18:23:42.707 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T18:23:42.707 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T18:23:42.707 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T18:23:42.707 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T18:23:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T18:23:42.709 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T18:23:42.710 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T18:23:42.711 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T18:23:42.712 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T18:23:42.713 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T18:23:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T18:23:42.715 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T18:23:42.715 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T18:23:42.715 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T18:23:42.715 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T18:23:42.715 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T18:23:42.715 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T18:23:42.715 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T18:23:42.715 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T18:23:42.715 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T18:23:42.715 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T18:23:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T18:23:42.717 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T18:23:42.718 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T18:23:42.719 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T18:23:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T18:23:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T18:23:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T18:23:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T18:23:42.724 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T18:23:42.725 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T18:23:42.725 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T18:23:42.725 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T18:23:42.725 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T18:23:42.725 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T18:23:42.725 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T18:23:42.725 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T18:23:42.725 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T18:23:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T18:23:42.727 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T18:23:42.728 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T18:23:42.729 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T18:23:42.730 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T18:23:42.731 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T18:23:42.732 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T18:23:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T18:23:42.734 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T18:23:42.735 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T18:23:42.736 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T18:23:42.737 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T18:23:42.738 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T18:23:42.739 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T18:23:42.740 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T18:23:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T18:23:42.742 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T18:23:42.743 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T18:23:42.744 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T18:23:42.745 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T18:23:42.746 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T18:23:42.747 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T18:23:42.748 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T18:23:42.749 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T18:23:42.750 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T18:23:42.751 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T18:23:42.752 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T18:23:42.753 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T18:23:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T18:23:42.755 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T18:23:42.756 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T18:23:42.757 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T18:23:42.758 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T18:23:42.759 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T18:23:42.760 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T18:23:42.761 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T18:23:42.762 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T18:23:42.763 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T18:23:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T18:23:42.765 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T18:23:42.766 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T18:23:42.767 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T18:23:42.768 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T18:23:42.769 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T18:23:42.770 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T18:23:42.771 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T18:23:42.772 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T18:23:42.773 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T18:23:42.774 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T18:23:42.775 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T18:23:42.776 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T18:23:42.777 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T18:23:42.778 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T18:23:42.779 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T18:23:42.780 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T18:23:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T18:23:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T18:23:42.783 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T18:23:42.784 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T18:23:42.785 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T18:23:42.786 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T18:23:42.787 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T18:23:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T18:23:42.789 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T18:23:42.790 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T18:23:42.791 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T18:23:42.792 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T18:23:42.793 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T18:23:42.794 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T18:23:42.795 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T18:23:42.796 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T18:23:42.797 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T18:23:42.798 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T18:23:42.799 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T18:23:42.800 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T18:23:42.801 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T18:23:42.802 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T18:23:42.803 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T18:23:42.804 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T18:23:42.805 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T18:23:42.806 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T18:23:42.807 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T18:23:42.808 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T18:23:42.809 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T18:23:42.810 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T18:23:42.811 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T18:23:42.812 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T18:23:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T18:23:42.814 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T18:23:42.815 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T18:23:42.816 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T18:23:42.817 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T18:23:42.818 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T18:23:42.819 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T18:23:42.820 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T18:23:42.821 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T18:23:42.822 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T18:23:42.822 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T18:23:42.822 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T18:23:42.822 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T18:23:42.822 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T18:23:42.822 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T18:23:42.822 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T18:23:42.822 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T18:23:42.822 INFO:teuthology.orchestra.run.smithi112.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T18:23:42.826 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T18:23:42.826 DEBUG:teuthology.orchestra.run.smithi112:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T18:23:42.848 INFO:teuthology.orchestra.run.smithi112.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T18:23:42.848 INFO:teuthology.orchestra.run.smithi112.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T18:23:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T18:23:42.848 INFO:teuthology.orchestra.run.smithi112.stdout:fi 2024-04-25T18:23:42.848 INFO:teuthology.orchestra.run.smithi112.stdout:for file in vmlinuz System.map config; do 2024-04-25T18:23:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T18:23:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:23:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: fi 2024-04-25T18:23:42.848 INFO:teuthology.orchestra.run.smithi112.stdout:done 2024-04-25T18:23:42.849 INFO:teuthology.orchestra.run.smithi112.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T18:23:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T18:23:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:23:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:23:42.849 INFO:teuthology.orchestra.run.smithi112.stdout:fi 2024-04-25T18:23:42.849 INFO:teuthology.orchestra.run.smithi112.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T18:23:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T18:23:42.849 INFO:teuthology.orchestra.run.smithi112.stdout:fi 2024-04-25T18:23:42.849 INFO:teuthology.orchestra.run.smithi112.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T18:23:42.849 INFO:teuthology.orchestra.run.smithi112.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T18:23:42.850 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T18:23:42.850 INFO:teuthology.orchestra.run.smithi112.stdout:fi 2024-04-25T18:23:42.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -qi grub2-tools 2024-04-25T18:23:42.921 INFO:teuthology.orchestra.run.smithi112.stdout:Name : grub2-tools 2024-04-25T18:23:42.921 INFO:teuthology.orchestra.run.smithi112.stdout:Epoch : 1 2024-04-25T18:23:42.921 INFO:teuthology.orchestra.run.smithi112.stdout:Version : 2.06 2024-04-25T18:23:42.921 INFO:teuthology.orchestra.run.smithi112.stdout:Release : 68.el9 2024-04-25T18:23:42.921 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: x86_64 2024-04-25T18:23:42.921 INFO:teuthology.orchestra.run.smithi112.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T18:23:42.921 INFO:teuthology.orchestra.run.smithi112.stdout:Group : Unspecified 2024-04-25T18:23:42.922 INFO:teuthology.orchestra.run.smithi112.stdout:Size : 8264254 2024-04-25T18:23:42.922 INFO:teuthology.orchestra.run.smithi112.stdout:License : GPLv3+ 2024-04-25T18:23:42.922 INFO:teuthology.orchestra.run.smithi112.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T18:23:42.922 INFO:teuthology.orchestra.run.smithi112.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T18:23:42.922 INFO:teuthology.orchestra.run.smithi112.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T18:23:42.922 INFO:teuthology.orchestra.run.smithi112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T18:23:42.922 INFO:teuthology.orchestra.run.smithi112.stdout:Packager : builder@centos.org 2024-04-25T18:23:42.922 INFO:teuthology.orchestra.run.smithi112.stdout:Vendor : CentOS 2024-04-25T18:23:42.922 INFO:teuthology.orchestra.run.smithi112.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T18:23:42.923 INFO:teuthology.orchestra.run.smithi112.stdout:Summary : Support tools for GRUB. 2024-04-25T18:23:42.923 INFO:teuthology.orchestra.run.smithi112.stdout:Description : 2024-04-25T18:23:42.923 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:23:42.923 INFO:teuthology.orchestra.run.smithi112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T18:23:42.923 INFO:teuthology.orchestra.run.smithi112.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T18:23:42.923 INFO:teuthology.orchestra.run.smithi112.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T18:23:42.923 INFO:teuthology.orchestra.run.smithi112.stdout:hardware devices. 2024-04-25T18:23:42.923 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:23:42.923 INFO:teuthology.orchestra.run.smithi112.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T18:23:42.924 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T18:23:42.924 INFO:teuthology.task.kernel:Updating grub on smithi112 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T18:23:42.924 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T18:23:43.496 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2024-04-25T18:23:44.916 INFO:teuthology.orchestra.run.smithi112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T18:23:44.937 INFO:teuthology.orchestra.run.smithi112.stderr:done 2024-04-25T18:23:44.940 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-04-25T18:23:44.957 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.DKTDXeHSBu 2024-04-25T18:23:44.958 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.DKTDXeHSBu 2024-04-25T18:23:45.024 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.DKTDXeHSBu 2024-04-25T18:23:45.156 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.DKTDXeHSBu is 6KB 2024-04-25T18:23:45.205 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.DKTDXeHSBu 2024-04-25T18:23:45.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T18:23:45.280 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T18:23:45.280 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T18:23:45.280 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T18:23:45.280 INFO:teuthology.orchestra.run.smithi112.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T18:23:45.282 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T18:23:45.425 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2024-04-25T18:23:45.498 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-04-25T18:24:15.427 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T18:24:15.428 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T18:24:15.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:24:33.826 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-04-25T18:24:42.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T18:24:42.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:24:45.890 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-04-25T18:24:57.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T18:24:57.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:25:16.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-04-25T18:25:31.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T18:25:31.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:25:32.303 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-25T18:25:32.994 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T18:25:32.994 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T18:25:32.995 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-04-25T18:25:33.034 INFO:teuthology.orchestra.run.smithi112.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T18:25:33.034 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T18:25:33.034 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T18:25:33.034 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T18:25:34.036 DEBUG:teuthology.parallel:result is None 2024-04-25T18:25:34.036 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T18:25:34.069 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T18:25:34.069 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T18:25:34.075 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T18:25:34.078 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T18:25:34.116 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T18:25:34.130 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T18:25:34.143 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T18:25:34.143 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T18:25:34.162 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T18:25:34.164 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T18:25:34.298 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T18:25:34.306 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T18:25:34.306 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T18:25:34.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:25:34.323 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T18:25:34.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:25:34.339 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T18:25:34.347 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T18:25:34.365 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T18:25:34.398 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:25:34.418 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:25:34.419 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:25:34.435 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:25:34.502 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:25:34.574 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:25:34.577 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T18:25:34.584 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T18:25:34.584 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T18:25:34.589 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T18:25:34.591 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T18:25:34.648 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T18:25:34.657 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T18:25:34.658 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T18:25:34.662 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T18:25:34.664 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T18:25:34.694 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:25:34.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:25:34.882 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:25:34.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:25:35.023 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:25:35.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T18:25:35.110 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:25:35.148 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:25:35.220 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:25:35.256 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:25:35.304 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-25T18:25:35.304 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T18:25:35.369 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:25:35.407 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:25:35.469 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:25:35.544 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:25:35.597 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T18:25:35.597 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T18:25:35.664 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-04-25T18:25:35.668 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-04-25T18:25:35.670 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-04-25T18:25:35.702 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T18:25:35.740 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T18:25:35.755 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T18:25:36.219 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T18:25:36.227 INFO:teuthology.task.internal:Starting timer... 2024-04-25T18:25:36.227 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T18:25:36.248 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T18:25:36.258 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T18:25:36.258 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-04-25T18:25:36.381 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-04-25T18:25:36.387 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-04-25T18:25:36.465 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-04-25T18:25:36.467 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-04-25T18:25:36.543 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-04-25T18:25:36.545 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T18:25:36.546 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-04-25T18:25:36.598 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-04-25T18:25:36.598 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-04-25T18:25:36.622 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-04-25T18:25:36.623 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-04-25T18:25:36.689 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-04-25T18:25:36.690 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-04-25T18:25:36.690 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T18:25:36.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:25:36.761 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T18:25:36.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:25:36.788 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T18:25:36.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:25:36.816 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T18:25:36.816 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-04-25T18:25:36.925 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-04-25T18:25:36.949 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-04-25T18:25:36.983 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T18:25:36.993 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T18:25:37.015 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T18:25:37.016 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi161.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T18:30:35.870 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-04-25T18:30:35.871 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T18:30:35.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:30:35.981 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-25T18:30:36.125 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T18:30:36.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-04-25T18:30:36.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:30:36.196 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-04-25T18:30:36.272 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-04-25T18:30:36.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-25T18:30:36.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:30:36.344 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-04-25T18:30:36.426 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-25T18:30:36.427 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T18:30:36.437 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T18:30:36.437 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T18:30:36.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T18:30:36.445 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T18:30:36.445 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T18:30:36.447 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T18:30:36.447 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T18:30:36.472 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T18:30:36.485 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T18:30:36.497 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T18:30:36.508 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-04-25T18:30:36.511 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T18:30:36.517 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T18:30:36.518 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-04-25T18:30:36.532 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T18:30:36.535 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-04-25T18:30:36.545 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T18:30:36.546 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-04-25T18:30:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T18:30:36.560 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T18:30:36.573 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T18:30:36.597 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-04-25T18:30:36.601 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T18:30:36.601 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-04-25T18:30:36.601 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.601 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.601 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.601 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-04-25T18:30:36.628 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-25T18:30:36.633 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T18:30:36.633 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-25T18:30:36.633 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.633 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.633 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.633 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.647 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-04-25T18:30:36.684 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T18:30:36.721 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T18:30:36.850 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T18:30:36.857 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T18:30:36.858 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T18:30:36.858 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.858 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.858 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.858 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:30:36.859 INFO:teuthology.run_tasks:Running task install... 2024-04-25T18:30:36.868 DEBUG:teuthology.task.install:project ceph 2024-04-25T18:30:36.868 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T18:30:36.868 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T18:30:36.868 INFO:teuthology.task.install:Using flavor: default 2024-04-25T18:30:36.874 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T18:30:36.874 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T18:30:36.874 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T18:30:36.874 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:30:36.876 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T18:30:36.877 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:30:36.877 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T18:30:36.878 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:30:37.028 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T18:30:37.028 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T18:30:37.036 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T18:30:37.036 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T18:30:37.048 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T18:30:37.048 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T18:30:37.139 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T18:30:37.139 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:30:37.139 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T18:30:37.149 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T18:30:37.150 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T18:30:37.150 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T18:30:37.177 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T18:30:37.178 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T18:30:37.215 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T18:30:37.215 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-25T18:30:37.215 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T18:30:37.232 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T18:30:37.232 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T18:30:37.241 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T18:30:37.241 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T18:30:37.244 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T18:30:37.307 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T18:30:37.320 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T18:30:37.359 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T18:30:37.381 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T18:30:37.385 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-04-25T18:30:37.387 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-04-25T18:30:37.444 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-04-25T18:30:37.446 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-04-25T18:30:37.470 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T18:30:37.544 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-04-25T18:30:37.548 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-04-25T18:30:37.613 INFO:teuthology.orchestra.run.smithi171.stdout:48 files removed 2024-04-25T18:30:37.640 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T18:30:37.672 INFO:teuthology.orchestra.run.smithi161.stdout:48 files removed 2024-04-25T18:30:37.700 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T18:30:37.867 INFO:teuthology.orchestra.run.smithi112.stdout:48 files removed 2024-04-25T18:30:37.905 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T18:30:38.254 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 237 kB/s | 83 kB 00:00 2024-04-25T18:30:38.375 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-04-25T18:30:38.646 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-25T18:30:38.742 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-25T18:30:38.840 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 168 kB/s | 83 kB 00:00 2024-04-25T18:30:38.946 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 7.2 kB/s | 1.9 kB 00:00 2024-04-25T18:30:39.109 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-25T18:30:39.357 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-25T18:30:39.699 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T18:30:39.853 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T18:30:39.906 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-04-25T18:30:40.936 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.0 MB 00:00 2024-04-25T18:30:41.542 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-25T18:30:41.846 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T18:30:43.146 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-25T18:30:45.992 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.2 MB 00:00 2024-04-25T18:30:46.305 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.2 MB 00:00 2024-04-25T18:30:47.847 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.2 MB 00:00 2024-04-25T18:30:48.341 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-25T18:30:48.604 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-25T18:30:49.625 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-25T18:30:49.811 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T18:30:50.041 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T18:30:51.200 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T18:30:54.934 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T18:30:55.213 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-25T18:30:57.102 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T18:30:57.103 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:30:57.103 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:30:57.114 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T18:30:57.114 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T18:30:57.115 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T18:30:57.116 INFO:teuthology.orchestra.run.smithi171.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T18:30:57.118 INFO:teuthology.orchestra.run.smithi171.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T18:30:57.120 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T18:30:57.121 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T18:30:57.181 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-04-25T18:30:57.210 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-25T18:30:57.210 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-04-25T18:30:57.211 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-25T18:30:57.211 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-04-25T18:30:57.211 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T18:30:57.211 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T18:30:57.211 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:30:57.211 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T18:30:57.211 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T18:30:57.211 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T18:30:57.211 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T18:30:57.211 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T18:30:57.211 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T18:30:57.212 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T18:30:57.212 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T18:30:57.212 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T18:30:57.212 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T18:30:57.212 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T18:30:57.212 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T18:30:57.212 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T18:30:57.212 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T18:30:57.212 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T18:30:57.212 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T18:30:57.212 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T18:30:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T18:30:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T18:30:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T18:30:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T18:30:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T18:30:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T18:30:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T18:30:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T18:30:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T18:30:57.213 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T18:30:57.214 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T18:30:57.214 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T18:30:57.214 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T18:30:57.214 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T18:30:57.214 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:30:57.214 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T18:30:57.214 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T18:30:57.214 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-04-25T18:30:57.215 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T18:30:57.215 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:30:57.215 INFO:teuthology.orchestra.run.smithi171.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T18:30:57.215 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T18:30:57.215 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-04-25T18:30:57.215 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T18:30:57.215 INFO:teuthology.orchestra.run.smithi171.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T18:30:57.215 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T18:30:57.215 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T18:30:57.215 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T18:30:57.216 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:30:57.217 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T18:30:57.218 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T18:30:57.218 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T18:30:57.218 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T18:30:57.218 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T18:30:57.218 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T18:30:57.218 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T18:30:57.218 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T18:30:57.218 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T18:30:57.218 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T18:30:57.218 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T18:30:57.218 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T18:30:57.219 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T18:30:57.219 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T18:30:57.219 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T18:30:57.219 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T18:30:57.219 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T18:30:57.219 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T18:30:57.219 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T18:30:57.219 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T18:30:57.219 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T18:30:57.219 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T18:30:57.219 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T18:30:57.220 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T18:30:57.221 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T18:30:57.222 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T18:30:57.222 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T18:30:57.222 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T18:30:57.222 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T18:30:57.222 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T18:30:57.222 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T18:30:57.222 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T18:30:57.222 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T18:30:57.222 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T18:30:57.222 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T18:30:57.222 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T18:30:57.223 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T18:30:57.224 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-04-25T18:30:57.225 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T18:30:57.225 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:30:57.225 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-04-25T18:30:57.225 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-25T18:30:57.225 INFO:teuthology.orchestra.run.smithi171.stdout:Install 148 Packages 2024-04-25T18:30:57.225 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 4 Packages 2024-04-25T18:30:57.225 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:30:57.226 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 222 M 2024-04-25T18:30:57.226 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-04-25T18:30:57.356 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:30:57.357 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:30:57.368 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T18:30:57.369 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T18:30:57.370 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T18:30:57.371 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T18:30:57.372 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T18:30:57.375 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T18:30:57.375 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T18:30:57.436 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-04-25T18:30:57.446 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-04-25T18:30:57.446 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-04-25T18:30:57.446 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-04-25T18:30:57.446 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-04-25T18:30:57.446 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T18:30:57.446 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T18:30:57.446 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:30:57.447 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T18:30:57.447 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T18:30:57.447 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T18:30:57.447 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T18:30:57.447 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T18:30:57.447 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T18:30:57.447 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T18:30:57.447 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T18:30:57.447 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T18:30:57.447 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T18:30:57.447 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T18:30:57.448 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-04-25T18:30:57.449 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T18:30:57.450 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T18:30:57.451 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T18:30:57.451 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T18:30:57.451 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T18:30:57.451 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T18:30:57.451 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T18:30:57.451 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T18:30:57.451 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T18:30:57.451 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T18:30:57.451 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T18:30:57.451 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T18:30:57.451 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T18:30:57.452 INFO:teuthology.orchestra.run.smithi161.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T18:30:57.453 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T18:30:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T18:30:57.455 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T18:30:57.456 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T18:30:57.457 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T18:30:57.457 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T18:30:57.457 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T18:30:57.457 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T18:30:57.457 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T18:30:57.457 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T18:30:57.457 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T18:30:57.457 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T18:30:57.457 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T18:30:57.457 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T18:30:57.457 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T18:30:57.458 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-04-25T18:30:57.459 INFO:teuthology.orchestra.run.smithi161.stdout:Install 148 Packages 2024-04-25T18:30:57.460 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 4 Packages 2024-04-25T18:30:57.460 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:30:57.460 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 222 M 2024-04-25T18:30:57.460 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-04-25T18:30:57.844 INFO:teuthology.orchestra.run.smithi171.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-25T18:30:58.177 INFO:teuthology.orchestra.run.smithi171.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.6 MB/s | 890 kB 00:00 2024-04-25T18:30:58.278 INFO:teuthology.orchestra.run.smithi171.stdout:(3/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 7.9 MB/s | 5.4 MB 00:00 2024-04-25T18:30:58.303 INFO:teuthology.orchestra.run.smithi171.stdout:(4/152): ceph-immutable-object-cache-19.0.0-315 1.1 MB/s | 145 kB 00:00 2024-04-25T18:30:58.457 INFO:teuthology.orchestra.run.smithi161.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 28 kB/s | 6.6 kB 00:00 2024-04-25T18:30:58.486 INFO:teuthology.orchestra.run.smithi171.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 11 MB/s | 2.4 MB 00:00 2024-04-25T18:30:58.695 INFO:teuthology.orchestra.run.smithi171.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.8 MB/s | 1.5 MB 00:00 2024-04-25T18:30:58.945 INFO:teuthology.orchestra.run.smithi171.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 10 MB/s | 4.7 MB 00:00 2024-04-25T18:30:59.323 INFO:teuthology.orchestra.run.smithi161.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 890 kB 00:00 2024-04-25T18:30:59.540 INFO:teuthology.orchestra.run.smithi161.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 669 kB/s | 145 kB 00:00 2024-04-25T18:30:59.553 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:30:59.555 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:30:59.577 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T18:30:59.578 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T18:30:59.579 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T18:30:59.581 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T18:30:59.583 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T18:30:59.587 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T18:30:59.588 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T18:30:59.660 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T18:30:59.682 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T18:30:59.682 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-04-25T18:30:59.682 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T18:30:59.682 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-04-25T18:30:59.682 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T18:30:59.682 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T18:30:59.682 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:30:59.682 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T18:30:59.683 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T18:30:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T18:30:59.685 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T18:30:59.686 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T18:30:59.687 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T18:30:59.688 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T18:30:59.689 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T18:30:59.690 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T18:30:59.690 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T18:30:59.690 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T18:30:59.690 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T18:30:59.690 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T18:30:59.690 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T18:30:59.690 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T18:30:59.690 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T18:30:59.690 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T18:30:59.690 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T18:30:59.690 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T18:30:59.691 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T18:30:59.692 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout:Install 148 Packages 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 4 Packages 2024-04-25T18:30:59.693 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:30:59.694 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 222 M 2024-04-25T18:30:59.694 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-25T18:31:00.437 INFO:teuthology.orchestra.run.smithi171.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 7.1 MB/s | 11 MB 00:01 2024-04-25T18:31:00.512 INFO:teuthology.orchestra.run.smithi171.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 335 kB/s | 25 kB 00:00 2024-04-25T18:31:00.854 INFO:teuthology.orchestra.run.smithi171.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 6.5 MB/s | 21 MB 00:03 2024-04-25T18:31:00.903 INFO:teuthology.orchestra.run.smithi112.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-25T18:31:00.907 INFO:teuthology.orchestra.run.smithi161.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 2.4 MB 00:01 2024-04-25T18:31:00.946 INFO:teuthology.orchestra.run.smithi171.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.4 MB/s | 225 kB 00:00 2024-04-25T18:31:01.029 INFO:teuthology.orchestra.run.smithi171.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 325 kB/s | 27 kB 00:00 2024-04-25T18:31:01.121 INFO:teuthology.orchestra.run.smithi171.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 345 kB/s | 32 kB 00:00 2024-04-25T18:31:01.504 INFO:teuthology.orchestra.run.smithi112.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.5 MB/s | 890 kB 00:00 2024-04-25T18:31:01.538 INFO:teuthology.orchestra.run.smithi171.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.8 MB/s | 747 kB 00:00 2024-04-25T18:31:01.629 INFO:teuthology.orchestra.run.smithi112.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.1 MB/s | 145 kB 00:00 2024-04-25T18:31:01.713 INFO:teuthology.orchestra.run.smithi171.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 933 kB/s | 163 kB 00:00 2024-04-25T18:31:01.757 INFO:teuthology.orchestra.run.smithi161.stdout:(5/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 6.0 MB/s | 21 MB 00:03 2024-04-25T18:31:01.832 INFO:teuthology.orchestra.run.smithi161.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.5 MB/s | 5.4 MB 00:03 2024-04-25T18:31:01.855 INFO:teuthology.orchestra.run.smithi171.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 895 kB/s | 126 kB 00:00 2024-04-25T18:31:01.866 INFO:teuthology.orchestra.run.smithi161.stdout:(7/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 1.6 MB/s | 1.5 MB 00:00 2024-04-25T18:31:02.231 INFO:teuthology.orchestra.run.smithi171.stdout:(17/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 4.8 MB/s | 17 MB 00:03 2024-04-25T18:31:02.257 INFO:teuthology.orchestra.run.smithi171.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 1.2 MB/s | 500 kB 00:00 2024-04-25T18:31:02.279 INFO:teuthology.orchestra.run.smithi112.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.6 MB/s | 2.4 MB 00:00 2024-04-25T18:31:02.349 INFO:teuthology.orchestra.run.smithi171.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 492 kB/s | 45 kB 00:00 2024-04-25T18:31:02.449 INFO:teuthology.orchestra.run.smithi171.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.3 MB/s | 135 kB 00:00 2024-04-25T18:31:02.541 INFO:teuthology.orchestra.run.smithi161.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 6.0 MB/s | 4.7 MB 00:00 2024-04-25T18:31:02.624 INFO:teuthology.orchestra.run.smithi171.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 921 kB/s | 161 kB 00:00 2024-04-25T18:31:02.625 INFO:teuthology.orchestra.run.smithi161.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 303 kB/s | 25 kB 00:00 2024-04-25T18:31:02.696 INFO:teuthology.orchestra.run.smithi112.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.6 MB/s | 1.5 MB 00:00 2024-04-25T18:31:02.900 INFO:teuthology.orchestra.run.smithi171.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.1 MB/s | 320 kB 00:00 2024-04-25T18:31:03.127 INFO:teuthology.orchestra.run.smithi171.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 298 kB 00:00 2024-04-25T18:31:03.233 INFO:teuthology.orchestra.run.smithi171.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 934 kB/s | 99 kB 00:00 2024-04-25T18:31:03.342 INFO:teuthology.orchestra.run.smithi171.stdout:(25/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 4.8 MB/s | 5.4 MB 00:01 2024-04-25T18:31:03.367 INFO:teuthology.orchestra.run.smithi171.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 638 kB/s | 85 kB 00:00 2024-04-25T18:31:03.492 INFO:teuthology.orchestra.run.smithi171.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.3 MB/s | 170 kB 00:00 2024-04-25T18:31:03.576 INFO:teuthology.orchestra.run.smithi171.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 410 kB/s | 34 kB 00:00 2024-04-25T18:31:03.595 INFO:teuthology.orchestra.run.smithi112.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.2 MB/s | 4.7 MB 00:00 2024-04-25T18:31:03.676 INFO:teuthology.orchestra.run.smithi171.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.4 MB/s | 142 kB 00:00 2024-04-25T18:31:03.922 INFO:teuthology.orchestra.run.smithi112.stdout:(7/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 5.4 MB 00:03 2024-04-25T18:31:03.993 INFO:teuthology.orchestra.run.smithi171.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 4.7 MB/s | 3.1 MB 00:00 2024-04-25T18:31:06.102 INFO:teuthology.orchestra.run.smithi171.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 3.5 MB/s | 7.4 MB 00:02 2024-04-25T18:31:06.260 INFO:teuthology.orchestra.run.smithi171.stdout:(32/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 1.4 MB/s | 3.5 MB 00:02 2024-04-25T18:31:06.330 INFO:teuthology.orchestra.run.smithi112.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 3.7 MB/s | 21 MB 00:05 2024-04-25T18:31:06.360 INFO:teuthology.orchestra.run.smithi171.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3156.gd6 962 kB/s | 249 kB 00:00 2024-04-25T18:31:06.386 INFO:teuthology.orchestra.run.smithi171.stdout:(34/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 393 kB/s | 49 kB 00:00 2024-04-25T18:31:06.469 INFO:teuthology.orchestra.run.smithi171.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 151 kB/s | 16 kB 00:00 2024-04-25T18:31:06.497 INFO:teuthology.orchestra.run.smithi112.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 151 kB/s | 25 kB 00:00 2024-04-25T18:31:06.578 INFO:teuthology.orchestra.run.smithi171.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.4 MB/s | 273 kB 00:00 2024-04-25T18:31:06.670 INFO:teuthology.orchestra.run.smithi171.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 200 kB/s | 18 kB 00:00 2024-04-25T18:31:06.795 INFO:teuthology.orchestra.run.smithi171.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 642 kB/s | 80 kB 00:00 2024-04-25T18:31:06.845 INFO:teuthology.orchestra.run.smithi171.stdout:(39/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.0 MB/s | 762 kB 00:00 2024-04-25T18:31:06.904 INFO:teuthology.orchestra.run.smithi171.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-04-25T18:31:06.929 INFO:teuthology.orchestra.run.smithi171.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-25T18:31:06.963 INFO:teuthology.orchestra.run.smithi171.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-25T18:31:06.997 INFO:teuthology.orchestra.run.smithi171.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T18:31:07.039 INFO:teuthology.orchestra.run.smithi171.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-25T18:31:07.064 INFO:teuthology.orchestra.run.smithi171.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-25T18:31:07.089 INFO:teuthology.orchestra.run.smithi171.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T18:31:07.115 INFO:teuthology.orchestra.run.smithi171.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T18:31:07.148 INFO:teuthology.orchestra.run.smithi171.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-04-25T18:31:07.155 INFO:teuthology.orchestra.run.smithi112.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 4.7 MB/s | 17 MB 00:03 2024-04-25T18:31:07.190 INFO:teuthology.orchestra.run.smithi171.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-04-25T18:31:07.215 INFO:teuthology.orchestra.run.smithi171.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T18:31:07.241 INFO:teuthology.orchestra.run.smithi171.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-25T18:31:07.266 INFO:teuthology.orchestra.run.smithi171.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T18:31:07.291 INFO:teuthology.orchestra.run.smithi171.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 660 kB/s | 33 kB 00:00 2024-04-25T18:31:07.330 INFO:teuthology.orchestra.run.smithi112.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.3 MB/s | 225 kB 00:00 2024-04-25T18:31:07.333 INFO:teuthology.orchestra.run.smithi171.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 444 kB/s | 18 kB 00:00 2024-04-25T18:31:07.409 INFO:teuthology.orchestra.run.smithi171.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-04-25T18:31:07.434 INFO:teuthology.orchestra.run.smithi171.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 113 kB/s | 11 kB 00:00 2024-04-25T18:31:07.459 INFO:teuthology.orchestra.run.smithi171.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T18:31:07.464 INFO:teuthology.orchestra.run.smithi112.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 204 kB/s | 27 kB 00:00 2024-04-25T18:31:07.484 INFO:teuthology.orchestra.run.smithi171.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-04-25T18:31:07.509 INFO:teuthology.orchestra.run.smithi171.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T18:31:07.551 INFO:teuthology.orchestra.run.smithi171.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 582 kB 00:00 2024-04-25T18:31:07.577 INFO:teuthology.orchestra.run.smithi171.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 383 kB/s | 25 kB 00:00 2024-04-25T18:31:07.589 INFO:teuthology.orchestra.run.smithi112.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 256 kB/s | 32 kB 00:00 2024-04-25T18:31:07.602 INFO:teuthology.orchestra.run.smithi171.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-04-25T18:31:07.627 INFO:teuthology.orchestra.run.smithi171.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T18:31:07.653 INFO:teuthology.orchestra.run.smithi171.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T18:31:07.670 INFO:teuthology.orchestra.run.smithi171.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 848 kB/s | 35 kB 00:00 2024-04-25T18:31:07.837 INFO:teuthology.orchestra.run.smithi171.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-04-25T18:31:07.865 INFO:teuthology.orchestra.run.smithi112.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.7 MB/s | 747 kB 00:00 2024-04-25T18:31:07.912 INFO:teuthology.orchestra.run.smithi171.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-25T18:31:07.954 INFO:teuthology.orchestra.run.smithi171.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-25T18:31:07.973 INFO:teuthology.orchestra.run.smithi112.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 163 kB 00:00 2024-04-25T18:31:07.996 INFO:teuthology.orchestra.run.smithi171.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T18:31:08.021 INFO:teuthology.orchestra.run.smithi171.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T18:31:08.082 INFO:teuthology.orchestra.run.smithi112.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.2 MB/s | 126 kB 00:00 2024-04-25T18:31:08.121 INFO:teuthology.orchestra.run.smithi171.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-25T18:31:08.188 INFO:teuthology.orchestra.run.smithi171.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-04-25T18:31:08.214 INFO:teuthology.orchestra.run.smithi171.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 972 kB/s | 87 kB 00:00 2024-04-25T18:31:08.264 INFO:teuthology.orchestra.run.smithi171.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-25T18:31:08.265 INFO:teuthology.orchestra.run.smithi112.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 2.7 MB/s | 500 kB 00:00 2024-04-25T18:31:08.323 INFO:teuthology.orchestra.run.smithi171.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.4 MB/s | 248 kB 00:00 2024-04-25T18:31:08.398 INFO:teuthology.orchestra.run.smithi171.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 6.0 MB 00:00 2024-04-25T18:31:08.423 INFO:teuthology.orchestra.run.smithi171.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 259 kB/s | 26 kB 00:00 2024-04-25T18:31:08.449 INFO:teuthology.orchestra.run.smithi171.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-25T18:31:08.474 INFO:teuthology.orchestra.run.smithi171.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T18:31:08.499 INFO:teuthology.orchestra.run.smithi171.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-04-25T18:31:08.591 INFO:teuthology.orchestra.run.smithi171.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-25T18:31:08.717 INFO:teuthology.orchestra.run.smithi171.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-04-25T18:31:08.742 INFO:teuthology.orchestra.run.smithi171.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-04-25T18:31:08.767 INFO:teuthology.orchestra.run.smithi171.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T18:31:08.792 INFO:teuthology.orchestra.run.smithi171.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 847 kB/s | 42 kB 00:00 2024-04-25T18:31:08.818 INFO:teuthology.orchestra.run.smithi171.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T18:31:08.843 INFO:teuthology.orchestra.run.smithi171.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T18:31:08.868 INFO:teuthology.orchestra.run.smithi171.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T18:31:08.894 INFO:teuthology.orchestra.run.smithi171.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T18:31:08.927 INFO:teuthology.orchestra.run.smithi171.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 919 kB/s | 54 kB 00:00 2024-04-25T18:31:08.977 INFO:teuthology.orchestra.run.smithi171.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 871 kB/s | 42 kB 00:00 2024-04-25T18:31:09.036 INFO:teuthology.orchestra.run.smithi171.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 793 kB/s | 44 kB 00:00 2024-04-25T18:31:09.178 INFO:teuthology.orchestra.run.smithi171.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-04-25T18:31:09.378 INFO:teuthology.orchestra.run.smithi171.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-04-25T18:31:09.403 INFO:teuthology.orchestra.run.smithi171.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 161 kB/s | 36 kB 00:00 2024-04-25T18:31:09.428 INFO:teuthology.orchestra.run.smithi171.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-25T18:31:09.454 INFO:teuthology.orchestra.run.smithi171.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-25T18:31:09.479 INFO:teuthology.orchestra.run.smithi171.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T18:31:09.496 INFO:teuthology.orchestra.run.smithi171.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-25T18:31:09.513 INFO:teuthology.orchestra.run.smithi171.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-04-25T18:31:09.524 INFO:teuthology.orchestra.run.smithi112.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 4.3 MB/s | 5.4 MB 00:01 2024-04-25T18:31:09.539 INFO:teuthology.orchestra.run.smithi171.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-04-25T18:31:09.556 INFO:teuthology.orchestra.run.smithi171.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 529 kB/s | 22 kB 00:00 2024-04-25T18:31:09.606 INFO:teuthology.orchestra.run.smithi171.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-25T18:31:09.650 INFO:teuthology.orchestra.run.smithi112.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 364 kB/s | 45 kB 00:00 2024-04-25T18:31:09.750 INFO:teuthology.orchestra.run.smithi112.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.3 MB/s | 135 kB 00:00 2024-04-25T18:31:09.806 INFO:teuthology.orchestra.run.smithi171.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-04-25T18:31:09.848 INFO:teuthology.orchestra.run.smithi171.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 469 kB/s | 19 kB 00:00 2024-04-25T18:31:09.858 INFO:teuthology.orchestra.run.smithi112.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.5 MB/s | 161 kB 00:00 2024-04-25T18:31:09.873 INFO:teuthology.orchestra.run.smithi171.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-25T18:31:09.983 INFO:teuthology.orchestra.run.smithi112.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.5 MB/s | 320 kB 00:00 2024-04-25T18:31:10.074 INFO:teuthology.orchestra.run.smithi171.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-25T18:31:10.092 INFO:teuthology.orchestra.run.smithi112.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.7 MB/s | 298 kB 00:00 2024-04-25T18:31:10.099 INFO:teuthology.orchestra.run.smithi171.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 115 kB/s | 26 kB 00:00 2024-04-25T18:31:10.124 INFO:teuthology.orchestra.run.smithi171.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-04-25T18:31:10.149 INFO:teuthology.orchestra.run.smithi171.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-25T18:31:10.175 INFO:teuthology.orchestra.run.smithi171.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-25T18:31:10.201 INFO:teuthology.orchestra.run.smithi112.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 924 kB/s | 99 kB 00:00 2024-04-25T18:31:10.233 INFO:teuthology.orchestra.run.smithi171.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-25T18:31:10.259 INFO:teuthology.orchestra.run.smithi171.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 6.5 MB/s | 548 kB 00:00 2024-04-25T18:31:10.284 INFO:teuthology.orchestra.run.smithi171.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 879 kB/s | 43 kB 00:00 2024-04-25T18:31:10.309 INFO:teuthology.orchestra.run.smithi171.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-04-25T18:31:10.318 INFO:teuthology.orchestra.run.smithi112.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 735 kB/s | 85 kB 00:00 2024-04-25T18:31:10.334 INFO:teuthology.orchestra.run.smithi171.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-25T18:31:10.360 INFO:teuthology.orchestra.run.smithi171.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-25T18:31:10.385 INFO:teuthology.orchestra.run.smithi171.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-25T18:31:10.410 INFO:teuthology.orchestra.run.smithi171.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-04-25T18:31:10.435 INFO:teuthology.orchestra.run.smithi171.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 709 kB/s | 35 kB 00:00 2024-04-25T18:31:10.460 INFO:teuthology.orchestra.run.smithi171.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-25T18:31:10.561 INFO:teuthology.orchestra.run.smithi171.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-25T18:31:10.586 INFO:teuthology.orchestra.run.smithi171.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-25T18:31:10.620 INFO:teuthology.orchestra.run.smithi171.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-25T18:31:10.653 INFO:teuthology.orchestra.run.smithi171.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-25T18:31:10.679 INFO:teuthology.orchestra.run.smithi171.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-04-25T18:31:10.704 INFO:teuthology.orchestra.run.smithi171.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-04-25T18:31:10.838 INFO:teuthology.orchestra.run.smithi171.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 124 kB/s | 20 kB 00:00 2024-04-25T18:31:10.888 INFO:teuthology.orchestra.run.smithi171.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-04-25T18:31:10.913 INFO:teuthology.orchestra.run.smithi171.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 631 kB/s | 46 kB 00:00 2024-04-25T18:31:10.939 INFO:teuthology.orchestra.run.smithi171.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-25T18:31:10.964 INFO:teuthology.orchestra.run.smithi171.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T18:31:10.990 INFO:teuthology.orchestra.run.smithi171.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-25T18:31:11.015 INFO:teuthology.orchestra.run.smithi171.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 331 kB/s | 16 kB 00:00 2024-04-25T18:31:11.040 INFO:teuthology.orchestra.run.smithi171.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-25T18:31:11.065 INFO:teuthology.orchestra.run.smithi171.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-04-25T18:31:11.090 INFO:teuthology.orchestra.run.smithi171.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-04-25T18:31:11.093 INFO:teuthology.orchestra.run.smithi112.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 4.0 MB/s | 3.1 MB 00:00 2024-04-25T18:31:11.116 INFO:teuthology.orchestra.run.smithi171.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-25T18:31:11.141 INFO:teuthology.orchestra.run.smithi171.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-25T18:31:11.166 INFO:teuthology.orchestra.run.smithi171.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-04-25T18:31:11.192 INFO:teuthology.orchestra.run.smithi171.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T18:31:11.210 INFO:teuthology.orchestra.run.smithi112.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.4 MB/s | 170 kB 00:00 2024-04-25T18:31:11.217 INFO:teuthology.orchestra.run.smithi171.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-25T18:31:11.242 INFO:teuthology.orchestra.run.smithi171.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-25T18:31:11.276 INFO:teuthology.orchestra.run.smithi171.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-04-25T18:31:11.301 INFO:teuthology.orchestra.run.smithi171.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 345 kB/s | 20 kB 00:00 2024-04-25T18:31:11.326 INFO:teuthology.orchestra.run.smithi171.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T18:31:11.393 INFO:teuthology.orchestra.run.smithi171.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-25T18:31:11.410 INFO:teuthology.orchestra.run.smithi112.stdout:(28/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 1.4 MB/s | 11 MB 00:07 2024-04-25T18:31:11.435 INFO:teuthology.orchestra.run.smithi112.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 152 kB/s | 34 kB 00:00 2024-04-25T18:31:11.507 INFO:teuthology.orchestra.run.smithi161.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 1.1 MB/s | 11 MB 00:09 2024-04-25T18:31:11.544 INFO:teuthology.orchestra.run.smithi112.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.0 MB/s | 142 kB 00:00 2024-04-25T18:31:11.732 INFO:teuthology.orchestra.run.smithi161.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.0 MB/s | 225 kB 00:00 2024-04-25T18:31:11.849 INFO:teuthology.orchestra.run.smithi161.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 232 kB/s | 27 kB 00:00 2024-04-25T18:31:11.975 INFO:teuthology.orchestra.run.smithi161.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 254 kB/s | 32 kB 00:00 2024-04-25T18:31:12.303 INFO:teuthology.orchestra.run.smithi112.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 4.1 MB/s | 3.5 MB 00:00 2024-04-25T18:31:12.450 INFO:teuthology.orchestra.run.smithi112.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.7 MB/s | 249 kB 00:00 2024-04-25T18:31:12.491 INFO:teuthology.orchestra.run.smithi161.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.4 MB/s | 747 kB 00:00 2024-04-25T18:31:12.537 INFO:teuthology.orchestra.run.smithi112.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 577 kB/s | 49 kB 00:00 2024-04-25T18:31:12.612 INFO:teuthology.orchestra.run.smithi112.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 222 kB/s | 16 kB 00:00 2024-04-25T18:31:12.642 INFO:teuthology.orchestra.run.smithi161.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.1 MB/s | 163 kB 00:00 2024-04-25T18:31:12.721 INFO:teuthology.orchestra.run.smithi112.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.5 MB/s | 273 kB 00:00 2024-04-25T18:31:12.818 INFO:teuthology.orchestra.run.smithi161.stdout:(16/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 1.5 MB/s | 17 MB 00:10 2024-04-25T18:31:12.843 INFO:teuthology.orchestra.run.smithi161.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 632 kB/s | 126 kB 00:00 2024-04-25T18:31:12.846 INFO:teuthology.orchestra.run.smithi112.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 6.0 MB/s | 762 kB 00:00 2024-04-25T18:31:12.877 INFO:teuthology.orchestra.run.smithi171.stdout:(148/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 4.0 MB/s | 49 MB 00:12 2024-04-25T18:31:12.930 INFO:teuthology.orchestra.run.smithi112.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 222 kB/s | 18 kB 00:00 2024-04-25T18:31:12.944 INFO:teuthology.orchestra.run.smithi171.stdout:(149/152): librados2-19.0.0-3156.gd69f13d8.el7. 2.1 MB/s | 3.4 MB 00:01 2024-04-25T18:31:12.977 INFO:teuthology.orchestra.run.smithi171.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 14 MB/s | 1.3 MB 00:00 2024-04-25T18:31:13.001 INFO:teuthology.orchestra.run.smithi161.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 2.7 MB/s | 500 kB 00:00 2024-04-25T18:31:13.087 INFO:teuthology.orchestra.run.smithi161.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 529 kB/s | 45 kB 00:00 2024-04-25T18:31:13.105 INFO:teuthology.orchestra.run.smithi112.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 462 kB/s | 80 kB 00:00 2024-04-25T18:31:13.119 INFO:teuthology.orchestra.run.smithi171.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-04-25T18:31:13.162 INFO:teuthology.orchestra.run.smithi161.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.8 MB/s | 135 kB 00:00 2024-04-25T18:31:13.237 INFO:teuthology.orchestra.run.smithi161.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.1 MB/s | 161 kB 00:00 2024-04-25T18:31:13.281 INFO:teuthology.orchestra.run.smithi112.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-25T18:31:13.363 INFO:teuthology.orchestra.run.smithi161.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.5 MB/s | 320 kB 00:00 2024-04-25T18:31:13.386 INFO:teuthology.orchestra.run.smithi171.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 3.1 MB 00:01 2024-04-25T18:31:13.392 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:31:13.392 INFO:teuthology.orchestra.run.smithi171.stdout:Total 14 MB/s | 222 MB 00:16 2024-04-25T18:31:13.480 INFO:teuthology.orchestra.run.smithi161.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.5 MB/s | 298 kB 00:00 2024-04-25T18:31:13.555 INFO:teuthology.orchestra.run.smithi161.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 99 kB 00:00 2024-04-25T18:31:13.622 INFO:teuthology.orchestra.run.smithi161.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.2 MB/s | 85 kB 00:00 2024-04-25T18:31:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:(40/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 6.7 MB/s | 49 MB 00:07 2024-04-25T18:31:13.865 INFO:teuthology.orchestra.run.smithi112.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 335 kB/s | 195 kB 00:00 2024-04-25T18:31:13.898 INFO:teuthology.orchestra.run.smithi112.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.9 MB/s | 253 kB 00:00 2024-04-25T18:31:13.966 INFO:teuthology.orchestra.run.smithi112.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-25T18:31:13.991 INFO:teuthology.orchestra.run.smithi112.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-25T18:31:14.016 INFO:teuthology.orchestra.run.smithi112.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-25T18:31:14.041 INFO:teuthology.orchestra.run.smithi112.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-25T18:31:14.075 INFO:teuthology.orchestra.run.smithi112.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 215 kB 00:00 2024-04-25T18:31:14.142 INFO:teuthology.orchestra.run.smithi112.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 6.8 MB/s | 689 kB 00:00 2024-04-25T18:31:14.147 INFO:teuthology.orchestra.run.smithi161.stdout:(26/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 4.3 MB/s | 49 MB 00:11 2024-04-25T18:31:14.184 INFO:teuthology.orchestra.run.smithi112.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 9.1 MB/s | 943 kB 00:00 2024-04-25T18:31:14.209 INFO:teuthology.orchestra.run.smithi112.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T18:31:14.214 INFO:teuthology.orchestra.run.smithi161.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 170 kB 00:00 2024-04-25T18:31:14.234 INFO:teuthology.orchestra.run.smithi112.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 442 kB/s | 21 kB 00:00 2024-04-25T18:31:14.256 INFO:teuthology.orchestra.run.smithi161.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 822 kB/s | 34 kB 00:00 2024-04-25T18:31:14.277 INFO:teuthology.orchestra.run.smithi112.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 798 kB/s | 33 kB 00:00 2024-04-25T18:31:14.307 INFO:teuthology.orchestra.run.smithi161.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.8 MB/s | 142 kB 00:00 2024-04-25T18:31:14.310 INFO:teuthology.orchestra.run.smithi112.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-04-25T18:31:14.369 INFO:teuthology.orchestra.run.smithi112.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 407 kB/s | 18 kB 00:00 2024-04-25T18:31:14.415 INFO:teuthology.orchestra.run.smithi161.stdout:(30/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.4 MB/s | 5.4 MB 00:01 2024-04-25T18:31:14.461 INFO:teuthology.orchestra.run.smithi112.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-04-25T18:31:14.494 INFO:teuthology.orchestra.run.smithi112.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 93 kB/s | 11 kB 00:00 2024-04-25T18:31:14.507 INFO:teuthology.orchestra.run.smithi161.stdout:(31/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.5 MB/s | 3.1 MB 00:00 2024-04-25T18:31:14.512 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-04-25T18:31:14.520 INFO:teuthology.orchestra.run.smithi112.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T18:31:14.545 INFO:teuthology.orchestra.run.smithi112.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 941 kB/s | 45 kB 00:00 2024-04-25T18:31:14.587 INFO:teuthology.orchestra.run.smithi112.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.9 MB/s | 662 kB 00:00 2024-04-25T18:31:14.607 INFO:teuthology.orchestra.run.smithi161.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.4 MB/s | 249 kB 00:00 2024-04-25T18:31:14.620 INFO:teuthology.orchestra.run.smithi112.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.3 MB/s | 582 kB 00:00 2024-04-25T18:31:14.646 INFO:teuthology.orchestra.run.smithi112.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 476 kB/s | 25 kB 00:00 2024-04-25T18:31:14.655 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-04-25T18:31:14.655 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-04-25T18:31:14.671 INFO:teuthology.orchestra.run.smithi112.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-04-25T18:31:14.699 INFO:teuthology.orchestra.run.smithi161.stdout:(33/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 9.0 MB/s | 3.5 MB 00:00 2024-04-25T18:31:14.705 INFO:teuthology.orchestra.run.smithi112.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.0 MB/s | 292 kB 00:00 2024-04-25T18:31:14.724 INFO:teuthology.orchestra.run.smithi161.stdout:(34/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 422 kB/s | 49 kB 00:00 2024-04-25T18:31:14.750 INFO:teuthology.orchestra.run.smithi161.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 328 kB/s | 16 kB 00:00 2024-04-25T18:31:14.764 INFO:teuthology.orchestra.run.smithi112.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.1 MB/s | 188 kB 00:00 2024-04-25T18:31:14.789 INFO:teuthology.orchestra.run.smithi112.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 435 kB/s | 35 kB 00:00 2024-04-25T18:31:14.825 INFO:teuthology.orchestra.run.smithi161.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.7 MB/s | 273 kB 00:00 2024-04-25T18:31:14.859 INFO:teuthology.orchestra.run.smithi161.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 6.9 MB/s | 762 kB 00:00 2024-04-25T18:31:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 218 kB/s | 18 kB 00:00 2024-04-25T18:31:14.947 INFO:teuthology.orchestra.run.smithi112.stdout:(66/152): ceph-mgr-diskprediction-local-19.0.0- 2.2 MB/s | 7.4 MB 00:03 2024-04-25T18:31:15.034 INFO:teuthology.orchestra.run.smithi161.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 459 kB/s | 80 kB 00:00 2024-04-25T18:31:15.064 INFO:teuthology.orchestra.run.smithi112.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T18:31:15.093 INFO:teuthology.orchestra.run.smithi161.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.0 MB/s | 803 kB 00:00 2024-04-25T18:31:15.118 INFO:teuthology.orchestra.run.smithi161.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-25T18:31:15.143 INFO:teuthology.orchestra.run.smithi161.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T18:31:15.181 INFO:teuthology.orchestra.run.smithi112.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-04-25T18:31:15.193 INFO:teuthology.orchestra.run.smithi161.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-25T18:31:15.219 INFO:teuthology.orchestra.run.smithi161.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-25T18:31:15.240 INFO:teuthology.orchestra.run.smithi112.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 5.8 MB/s | 1.6 MB 00:00 2024-04-25T18:31:15.244 INFO:teuthology.orchestra.run.smithi161.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T18:31:15.269 INFO:teuthology.orchestra.run.smithi161.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T18:31:15.282 INFO:teuthology.orchestra.run.smithi112.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 4.4 MB/s | 911 kB 00:00 2024-04-25T18:31:15.294 INFO:teuthology.orchestra.run.smithi161.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T18:31:15.307 INFO:teuthology.orchestra.run.smithi112.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-25T18:31:15.320 INFO:teuthology.orchestra.run.smithi161.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T18:31:15.362 INFO:teuthology.orchestra.run.smithi161.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T18:31:15.387 INFO:teuthology.orchestra.run.smithi161.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T18:31:15.391 INFO:teuthology.orchestra.run.smithi112.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.5 MB/s | 1.0 MB 00:00 2024-04-25T18:31:15.413 INFO:teuthology.orchestra.run.smithi161.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-04-25T18:31:15.424 INFO:teuthology.orchestra.run.smithi112.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 753 kB/s | 87 kB 00:00 2024-04-25T18:31:15.438 INFO:teuthology.orchestra.run.smithi161.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T18:31:15.463 INFO:teuthology.orchestra.run.smithi161.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 661 kB/s | 33 kB 00:00 2024-04-25T18:31:15.505 INFO:teuthology.orchestra.run.smithi161.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 449 kB/s | 18 kB 00:00 2024-04-25T18:31:15.547 INFO:teuthology.orchestra.run.smithi161.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 274 kB/s | 11 kB 00:00 2024-04-25T18:31:15.605 INFO:teuthology.orchestra.run.smithi161.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-04-25T18:31:15.631 INFO:teuthology.orchestra.run.smithi161.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-25T18:31:15.641 INFO:teuthology.orchestra.run.smithi112.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-25T18:31:15.656 INFO:teuthology.orchestra.run.smithi161.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-25T18:31:15.707 INFO:teuthology.orchestra.run.smithi161.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-04-25T18:31:15.708 INFO:teuthology.orchestra.run.smithi112.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 909 kB/s | 244 kB 00:00 2024-04-25T18:31:15.732 INFO:teuthology.orchestra.run.smithi161.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-25T18:31:15.765 INFO:teuthology.orchestra.run.smithi161.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 444 kB/s | 25 kB 00:00 2024-04-25T18:31:15.790 INFO:teuthology.orchestra.run.smithi161.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-04-25T18:31:15.816 INFO:teuthology.orchestra.run.smithi161.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T18:31:15.833 INFO:teuthology.orchestra.run.smithi112.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-04-25T18:31:15.841 INFO:teuthology.orchestra.run.smithi161.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T18:31:15.858 INFO:teuthology.orchestra.run.smithi112.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 248 kB 00:00 2024-04-25T18:31:15.867 INFO:teuthology.orchestra.run.smithi161.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-04-25T18:31:15.884 INFO:teuthology.orchestra.run.smithi112.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 160 kB/s | 26 kB 00:00 2024-04-25T18:31:15.942 INFO:teuthology.orchestra.run.smithi112.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 336 kB/s | 19 kB 00:00 2024-04-25T18:31:15.961 INFO:teuthology.orchestra.run.smithi161.stdout:(66/152): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-04-25T18:31:16.009 INFO:teuthology.orchestra.run.smithi112.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 282 kB/s | 35 kB 00:00 2024-04-25T18:31:16.043 INFO:teuthology.orchestra.run.smithi112.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-04-25T18:31:16.075 INFO:teuthology.orchestra.run.smithi161.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-25T18:31:16.101 INFO:teuthology.orchestra.run.smithi112.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.8 MB/s | 250 kB 00:00 2024-04-25T18:31:16.227 INFO:teuthology.orchestra.run.smithi112.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-04-25T18:31:16.369 INFO:teuthology.orchestra.run.smithi112.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-04-25T18:31:16.393 INFO:teuthology.orchestra.run.smithi161.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.6 MB/s | 4.7 MB 00:00 2024-04-25T18:31:16.394 INFO:teuthology.orchestra.run.smithi112.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 274 kB/s | 77 kB 00:00 2024-04-25T18:31:16.419 INFO:teuthology.orchestra.run.smithi112.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 239 kB/s | 42 kB 00:00 2024-04-25T18:31:16.430 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-04-25T18:31:16.431 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-04-25T18:31:16.434 INFO:teuthology.orchestra.run.smithi161.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-04-25T18:31:16.460 INFO:teuthology.orchestra.run.smithi161.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 2.3 MB/s | 911 kB 00:00 2024-04-25T18:31:16.469 INFO:teuthology.orchestra.run.smithi112.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 159 kB 00:00 2024-04-25T18:31:16.476 INFO:teuthology.orchestra.run.smithi161.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-25T18:31:16.503 INFO:teuthology.orchestra.run.smithi112.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-04-25T18:31:16.528 INFO:teuthology.orchestra.run.smithi112.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-25T18:31:16.561 INFO:teuthology.orchestra.run.smithi161.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T18:31:16.561 INFO:teuthology.orchestra.run.smithi112.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 648 kB/s | 51 kB 00:00 2024-04-25T18:31:16.569 INFO:teuthology.orchestra.run.smithi161.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-04-25T18:31:16.587 INFO:teuthology.orchestra.run.smithi112.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 945 kB/s | 54 kB 00:00 2024-04-25T18:31:16.679 INFO:teuthology.orchestra.run.smithi112.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 334 kB/s | 42 kB 00:00 2024-04-25T18:31:16.819 INFO:teuthology.orchestra.run.smithi161.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-25T18:31:16.844 INFO:teuthology.orchestra.run.smithi161.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 893 kB/s | 244 kB 00:00 2024-04-25T18:31:16.854 INFO:teuthology.orchestra.run.smithi112.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 171 kB/s | 44 kB 00:00 2024-04-25T18:31:16.936 INFO:teuthology.orchestra.run.smithi161.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-25T18:31:16.961 INFO:teuthology.orchestra.run.smithi161.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-04-25T18:31:16.986 INFO:teuthology.orchestra.run.smithi161.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 184 kB/s | 26 kB 00:00 2024-04-25T18:31:16.996 INFO:teuthology.orchestra.run.smithi112.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 728 kB/s | 221 kB 00:00 2024-04-25T18:31:17.020 INFO:teuthology.orchestra.run.smithi161.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 230 kB/s | 19 kB 00:00 2024-04-25T18:31:17.062 INFO:teuthology.orchestra.run.smithi161.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-04-25T18:31:17.080 INFO:teuthology.orchestra.run.smithi112.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 171 kB/s | 36 kB 00:00 2024-04-25T18:31:17.096 INFO:teuthology.orchestra.run.smithi161.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-04-25T18:31:17.154 INFO:teuthology.orchestra.run.smithi161.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-25T18:31:17.255 INFO:teuthology.orchestra.run.smithi161.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 5.1 MB 00:00 2024-04-25T18:31:17.280 INFO:teuthology.orchestra.run.smithi161.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-04-25T18:31:17.305 INFO:teuthology.orchestra.run.smithi161.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 520 kB/s | 77 kB 00:00 2024-04-25T18:31:17.313 INFO:teuthology.orchestra.run.smithi112.stdout:(96/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-25T18:31:17.330 INFO:teuthology.orchestra.run.smithi161.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 566 kB/s | 42 kB 00:00 2024-04-25T18:31:17.338 INFO:teuthology.orchestra.run.smithi112.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 145 kB/s | 47 kB 00:00 2024-04-25T18:31:17.355 INFO:teuthology.orchestra.run.smithi161.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-25T18:31:17.372 INFO:teuthology.orchestra.run.smithi112.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 305 kB 00:00 2024-04-25T18:31:17.381 INFO:teuthology.orchestra.run.smithi161.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T18:31:17.397 INFO:teuthology.orchestra.run.smithi112.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-25T18:31:17.406 INFO:teuthology.orchestra.run.smithi161.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-25T18:31:17.422 INFO:teuthology.orchestra.run.smithi112.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T18:31:17.423 INFO:teuthology.orchestra.run.smithi161.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 767 kB/s | 51 kB 00:00 2024-04-25T18:31:17.440 INFO:teuthology.orchestra.run.smithi161.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 921 kB/s | 54 kB 00:00 2024-04-25T18:31:17.448 INFO:teuthology.orchestra.run.smithi112.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 850 kB/s | 64 kB 00:00 2024-04-25T18:31:17.457 INFO:teuthology.orchestra.run.smithi161.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-04-25T18:31:17.473 INFO:teuthology.orchestra.run.smithi112.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 923 kB/s | 45 kB 00:00 2024-04-25T18:31:17.482 INFO:teuthology.orchestra.run.smithi161.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-04-25T18:31:17.498 INFO:teuthology.orchestra.run.smithi112.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 303 kB/s | 22 kB 00:00 2024-04-25T18:31:17.532 INFO:teuthology.orchestra.run.smithi161.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-25T18:31:17.548 INFO:teuthology.orchestra.run.smithi112.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 8.5 MB/s | 863 kB 00:00 2024-04-25T18:31:17.566 INFO:teuthology.orchestra.run.smithi161.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 436 kB/s | 36 kB 00:00 2024-04-25T18:31:17.675 INFO:teuthology.orchestra.run.smithi161.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 339 kB/s | 47 kB 00:00 2024-04-25T18:31:17.742 INFO:teuthology.orchestra.run.smithi161.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 305 kB 00:00 2024-04-25T18:31:17.765 INFO:teuthology.orchestra.run.smithi112.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 90 kB/s | 19 kB 00:00 2024-04-25T18:31:17.974 INFO:teuthology.orchestra.run.smithi112.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 662 kB/s | 309 kB 00:00 2024-04-25T18:31:18.009 INFO:teuthology.orchestra.run.smithi161.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-25T18:31:18.033 INFO:teuthology.orchestra.run.smithi112.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-25T18:31:18.034 INFO:teuthology.orchestra.run.smithi161.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 162 kB/s | 58 kB 00:00 2024-04-25T18:31:18.059 INFO:teuthology.orchestra.run.smithi161.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 297 kB/s | 94 kB 00:00 2024-04-25T18:31:18.075 INFO:teuthology.orchestra.run.smithi112.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-04-25T18:31:18.084 INFO:teuthology.orchestra.run.smithi161.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 853 kB/s | 64 kB 00:00 2024-04-25T18:31:18.109 INFO:teuthology.orchestra.run.smithi161.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-04-25T18:31:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 489 kB/s | 49 kB 00:00 2024-04-25T18:31:18.134 INFO:teuthology.orchestra.run.smithi161.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-25T18:31:18.158 INFO:teuthology.orchestra.run.smithi112.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 813 kB/s | 67 kB 00:00 2024-04-25T18:31:18.176 INFO:teuthology.orchestra.run.smithi161.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-04-25T18:31:18.245 INFO:teuthology.orchestra.run.smithi112.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T18:31:18.501 INFO:teuthology.orchestra.run.smithi112.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-04-25T18:31:18.510 INFO:teuthology.orchestra.run.smithi161.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 58 kB/s | 19 kB 00:00 2024-04-25T18:31:18.573 INFO:teuthology.orchestra.run.smithi112.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.0 MB/s | 838 kB 00:00 2024-04-25T18:31:18.585 INFO:teuthology.orchestra.run.smithi161.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 688 kB/s | 309 kB 00:00 2024-04-25T18:31:18.618 INFO:teuthology.orchestra.run.smithi112.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 375 kB/s | 43 kB 00:00 2024-04-25T18:31:18.652 INFO:teuthology.orchestra.run.smithi161.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-04-25T18:31:18.660 INFO:teuthology.orchestra.run.smithi112.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 1.3 MB/s | 548 kB 00:00 2024-04-25T18:31:18.702 INFO:teuthology.orchestra.run.smithi112.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 250 kB/s | 32 kB 00:00 2024-04-25T18:31:18.744 INFO:teuthology.orchestra.run.smithi112.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 119 kB/s | 15 kB 00:00 2024-04-25T18:31:18.778 INFO:teuthology.orchestra.run.smithi112.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-25T18:31:18.786 INFO:teuthology.orchestra.run.smithi161.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 844 kB/s | 548 kB 00:00 2024-04-25T18:31:18.811 INFO:teuthology.orchestra.run.smithi161.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 306 kB/s | 49 kB 00:00 2024-04-25T18:31:18.812 INFO:teuthology.orchestra.run.smithi112.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-25T18:31:18.845 INFO:teuthology.orchestra.run.smithi112.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-25T18:31:18.870 INFO:teuthology.orchestra.run.smithi161.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 803 kB/s | 67 kB 00:00 2024-04-25T18:31:18.912 INFO:teuthology.orchestra.run.smithi112.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 265 kB/s | 35 kB 00:00 2024-04-25T18:31:18.946 INFO:teuthology.orchestra.run.smithi112.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-04-25T18:31:19.012 INFO:teuthology.orchestra.run.smithi161.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 754 kB/s | 151 kB 00:00 2024-04-25T18:31:19.038 INFO:teuthology.orchestra.run.smithi112.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-25T18:31:19.080 INFO:teuthology.orchestra.run.smithi112.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-04-25T18:31:19.105 INFO:teuthology.orchestra.run.smithi112.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 754 kB/s | 144 kB 00:00 2024-04-25T18:31:19.129 INFO:teuthology.orchestra.run.smithi161.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-04-25T18:31:19.147 INFO:teuthology.orchestra.run.smithi112.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-25T18:31:19.189 INFO:teuthology.orchestra.run.smithi112.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-04-25T18:31:19.254 INFO:teuthology.orchestra.run.smithi161.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-04-25T18:31:19.256 INFO:teuthology.orchestra.run.smithi112.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 130 kB/s | 19 kB 00:00 2024-04-25T18:31:19.280 INFO:teuthology.orchestra.run.smithi161.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 289 kB/s | 43 kB 00:00 2024-04-25T18:31:19.298 INFO:teuthology.orchestra.run.smithi112.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 131 kB/s | 20 kB 00:00 2024-04-25T18:31:19.329 INFO:teuthology.orchestra.run.smithi161.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-04-25T18:31:19.348 INFO:teuthology.orchestra.run.smithi112.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-04-25T18:31:19.355 INFO:teuthology.orchestra.run.smithi161.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-04-25T18:31:19.380 INFO:teuthology.orchestra.run.smithi161.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-25T18:31:19.398 INFO:teuthology.orchestra.run.smithi112.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 327 kB/s | 46 kB 00:00 2024-04-25T18:31:19.432 INFO:teuthology.orchestra.run.smithi112.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 591 kB/s | 79 kB 00:00 2024-04-25T18:31:19.481 INFO:teuthology.orchestra.run.smithi161.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-25T18:31:19.482 INFO:teuthology.orchestra.run.smithi112.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 432 kB/s | 58 kB 00:00 2024-04-25T18:31:19.506 INFO:teuthology.orchestra.run.smithi161.stdout:(119/152): python3-cheroot-8.6.0-4.el9.noarch.r 977 kB/s | 172 kB 00:00 2024-04-25T18:31:19.539 INFO:teuthology.orchestra.run.smithi161.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-04-25T18:31:19.540 INFO:teuthology.orchestra.run.smithi112.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-25T18:31:19.565 INFO:teuthology.orchestra.run.smithi161.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-04-25T18:31:19.598 INFO:teuthology.orchestra.run.smithi161.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-25T18:31:19.600 INFO:teuthology.orchestra.run.smithi112.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 98 kB/s | 16 kB 00:00 2024-04-25T18:31:19.650 INFO:teuthology.orchestra.run.smithi112.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 258 kB/s | 28 kB 00:00 2024-04-25T18:31:19.675 INFO:teuthology.orchestra.run.smithi112.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 469 kB/s | 90 kB 00:00 2024-04-25T18:31:19.690 INFO:teuthology.orchestra.run.smithi161.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-25T18:31:19.726 INFO:teuthology.orchestra.run.smithi112.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2024-04-25T18:31:19.740 INFO:teuthology.orchestra.run.smithi161.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-04-25T18:31:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T18:31:19.766 INFO:teuthology.orchestra.run.smithi161.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 64 kB/s | 11 kB 00:00 2024-04-25T18:31:19.791 INFO:teuthology.orchestra.run.smithi161.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-25T18:31:19.818 INFO:teuthology.orchestra.run.smithi112.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 417 kB/s | 59 kB 00:00 2024-04-25T18:31:19.875 INFO:teuthology.orchestra.run.smithi161.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-04-25T18:31:19.893 INFO:teuthology.orchestra.run.smithi112.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 214 kB/s | 36 kB 00:00 2024-04-25T18:31:19.900 INFO:teuthology.orchestra.run.smithi161.stdout:(128/152): python3-jaraco-collections-3.0.0-8.e 145 kB/s | 23 kB 00:00 2024-04-25T18:31:19.925 INFO:teuthology.orchestra.run.smithi161.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 147 kB/s | 20 kB 00:00 2024-04-25T18:31:19.943 INFO:teuthology.orchestra.run.smithi112.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 278 kB/s | 51 kB 00:00 2024-04-25T18:31:19.977 INFO:teuthology.orchestra.run.smithi112.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-04-25T18:31:20.002 INFO:teuthology.orchestra.run.smithi112.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 830 kB/s | 90 kB 00:00 2024-04-25T18:31:20.017 INFO:teuthology.orchestra.run.smithi161.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-04-25T18:31:20.042 INFO:teuthology.orchestra.run.smithi161.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-04-25T18:31:20.052 INFO:teuthology.orchestra.run.smithi112.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-25T18:31:20.067 INFO:teuthology.orchestra.run.smithi161.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 555 kB/s | 79 kB 00:00 2024-04-25T18:31:20.103 INFO:teuthology.orchestra.run.smithi112.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-04-25T18:31:20.109 INFO:teuthology.orchestra.run.smithi161.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2024-04-25T18:31:20.137 INFO:teuthology.orchestra.run.smithi112.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-25T18:31:20.160 INFO:teuthology.orchestra.run.smithi161.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-04-25T18:31:20.185 INFO:teuthology.orchestra.run.smithi161.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-25T18:31:20.219 INFO:teuthology.orchestra.run.smithi161.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-04-25T18:31:20.237 INFO:teuthology.orchestra.run.smithi112.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-25T18:31:20.244 INFO:teuthology.orchestra.run.smithi161.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-04-25T18:31:20.277 INFO:teuthology.orchestra.run.smithi161.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-04-25T18:31:20.319 INFO:teuthology.orchestra.run.smithi161.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T18:31:20.329 INFO:teuthology.orchestra.run.smithi112.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-04-25T18:31:20.353 INFO:teuthology.orchestra.run.smithi161.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-04-25T18:31:20.378 INFO:teuthology.orchestra.run.smithi161.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-25T18:31:20.445 INFO:teuthology.orchestra.run.smithi161.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 405 kB/s | 51 kB 00:00 2024-04-25T18:31:20.454 INFO:teuthology.orchestra.run.smithi112.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-25T18:31:20.471 INFO:teuthology.orchestra.run.smithi161.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 973 kB/s | 90 kB 00:00 2024-04-25T18:31:20.504 INFO:teuthology.orchestra.run.smithi161.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-04-25T18:31:20.563 INFO:teuthology.orchestra.run.smithi112.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 7.4 MB/s | 3.4 MB 00:00 2024-04-25T18:31:20.663 INFO:teuthology.orchestra.run.smithi161.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-04-25T18:31:20.705 INFO:teuthology.orchestra.run.smithi161.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 85 kB/s | 20 kB 00:00 2024-04-25T18:31:20.730 INFO:teuthology.orchestra.run.smithi161.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 850 kB/s | 192 kB 00:00 2024-04-25T18:31:20.872 INFO:teuthology.orchestra.run.smithi161.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-25T18:31:20.956 INFO:teuthology.orchestra.run.smithi161.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-04-25T18:31:21.004 INFO:teuthology.orchestra.run.smithi112.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 3.6 MB/s | 3.1 MB 00:00 2024-04-25T18:31:21.012 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:31:21.013 INFO:teuthology.orchestra.run.smithi112.stdout:Total 10 MB/s | 222 MB 00:21 2024-04-25T18:31:21.064 INFO:teuthology.orchestra.run.smithi161.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-25T18:31:21.331 INFO:teuthology.orchestra.run.smithi161.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 5.2 MB/s | 3.1 MB 00:00 2024-04-25T18:31:21.506 INFO:teuthology.orchestra.run.smithi161.stdout:(152/152): librados2-19.0.0-3156.gd69f13d8.el7. 4.2 MB/s | 3.4 MB 00:00 2024-04-25T18:31:21.515 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:31:21.515 INFO:teuthology.orchestra.run.smithi161.stdout:Total 9.2 MB/s | 222 MB 00:24 2024-04-25T18:31:22.360 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-04-25T18:31:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T18:31:22.583 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-04-25T18:31:22.654 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T18:31:22.724 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T18:31:22.728 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-04-25T18:31:22.728 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-04-25T18:31:22.757 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T18:31:22.783 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T18:31:22.825 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T18:31:23.152 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T18:31:23.267 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:31:23.389 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:31:23.426 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:31:23.516 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:31:23.547 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:31:23.695 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T18:31:23.744 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T18:31:23.772 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:31:23.812 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-25T18:31:23.876 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:31:23.898 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:31:23.961 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:31:23.982 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:31:24.005 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:31:24.052 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-25T18:31:24.052 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-25T18:31:24.158 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:31:24.190 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:31:24.296 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T18:31:24.385 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T18:31:24.431 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T18:31:24.467 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T18:31:24.540 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T18:31:24.574 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T18:31:24.593 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-04-25T18:31:24.593 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-04-25T18:31:24.618 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T18:31:24.650 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T18:31:24.678 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T18:31:24.748 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T18:31:24.801 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T18:31:24.831 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T18:31:24.884 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T18:31:24.960 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T18:31:24.994 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T18:31:25.038 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T18:31:25.066 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T18:31:25.136 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T18:31:25.181 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T18:31:25.256 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T18:31:25.399 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T18:31:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T18:31:25.971 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T18:31:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T18:31:26.099 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T18:31:26.142 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T18:31:26.181 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T18:31:26.237 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T18:31:26.423 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T18:31:26.492 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T18:31:26.546 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T18:31:26.627 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T18:31:26.659 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T18:31:26.696 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T18:31:26.738 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-25T18:31:26.739 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-25T18:31:26.742 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T18:31:26.778 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T18:31:26.821 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T18:31:26.864 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T18:31:26.999 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T18:31:27.044 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T18:31:27.074 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T18:31:27.115 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T18:31:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T18:31:27.241 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T18:31:27.281 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T18:31:27.307 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T18:31:27.331 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T18:31:27.407 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T18:31:27.533 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T18:31:27.564 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T18:31:27.591 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T18:31:27.651 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T18:31:27.886 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T18:31:28.076 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T18:31:28.170 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T18:31:28.204 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T18:31:28.238 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T18:31:28.280 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T18:31:28.313 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T18:31:28.405 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T18:31:28.461 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T18:31:28.491 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T18:31:28.538 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T18:31:28.638 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T18:31:28.692 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T18:31:29.412 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T18:31:29.456 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T18:31:29.490 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T18:31:29.588 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T18:31:29.639 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T18:31:29.727 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T18:31:29.767 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T18:31:29.934 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T18:31:29.973 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T18:31:30.051 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T18:31:30.692 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T18:31:30.789 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-04-25T18:31:30.804 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T18:31:30.893 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T18:31:30.931 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T18:31:30.985 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T18:31:31.018 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T18:31:31.053 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T18:31:31.086 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T18:31:31.415 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T18:31:31.512 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:31:31.634 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:31:31.663 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:31:31.754 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:31:31.793 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:31:31.941 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T18:31:31.989 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T18:31:32.017 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:31:32.026 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T18:31:32.110 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T18:31:32.120 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:31:32.138 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T18:31:32.143 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:31:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T18:31:32.197 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:31:32.219 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:31:32.240 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:31:32.436 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:31:32.468 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:31:32.483 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T18:31:32.526 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:31:32.575 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T18:31:32.617 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:31:32.647 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T18:31:32.662 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T18:31:32.683 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T18:31:32.701 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T18:31:32.737 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T18:31:32.808 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T18:31:32.852 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T18:31:32.895 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T18:31:32.928 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T18:31:32.956 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T18:31:33.025 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T18:31:33.079 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T18:31:33.109 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T18:31:33.153 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T18:31:33.162 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T18:31:33.187 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:31:33.230 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T18:31:33.258 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:31:33.264 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T18:31:33.280 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T18:31:33.308 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T18:31:33.336 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T18:31:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T18:31:33.451 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T18:31:33.525 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T18:31:33.637 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-25T18:31:33.661 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T18:31:33.720 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T18:31:33.746 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T18:31:33.852 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T18:31:33.961 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T18:31:34.014 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T18:31:34.046 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T18:31:34.187 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T18:31:34.244 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T18:31:34.323 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T18:31:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T18:31:34.395 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T18:31:34.425 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T18:31:34.466 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T18:31:34.609 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T18:31:34.627 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T18:31:34.703 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T18:31:34.711 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:31:34.757 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T18:31:34.821 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T18:31:34.853 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T18:31:34.898 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:31:34.898 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T18:31:34.928 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:31:34.945 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T18:31:34.981 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T18:31:35.024 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T18:31:35.065 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T18:31:35.096 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:31:35.156 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:31:35.201 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T18:31:35.255 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T18:31:35.310 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T18:31:35.352 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T18:31:35.446 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T18:31:35.486 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T18:31:35.506 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T18:31:35.520 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:31:35.526 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T18:31:35.551 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T18:31:35.553 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:31:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T18:31:35.576 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T18:31:35.593 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:31:35.650 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T18:31:35.740 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:31:35.777 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T18:31:35.778 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:31:35.817 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T18:31:35.844 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T18:31:35.864 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:31:35.879 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T18:31:35.895 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:31:35.918 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T18:31:35.949 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T18:31:35.969 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:31:35.979 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T18:31:36.022 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T18:31:36.056 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T18:31:36.095 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:31:36.097 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T18:31:36.132 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T18:31:36.132 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:31:36.222 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T18:31:36.228 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:31:36.228 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T18:31:36.228 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T18:31:36.228 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:31:36.250 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:31:36.270 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T18:31:36.291 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:31:36.302 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T18:31:36.348 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T18:31:36.371 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:31:36.446 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T18:31:36.495 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T18:31:36.537 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T18:31:36.663 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T18:31:36.713 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T18:31:36.761 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T18:31:36.863 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T18:31:36.924 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T18:31:37.005 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T18:31:37.056 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T18:31:37.095 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T18:31:37.208 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T18:31:37.213 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T18:31:37.259 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T18:31:37.292 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T18:31:37.314 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T18:31:37.354 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T18:31:37.393 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T18:31:37.413 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T18:31:37.441 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T18:31:37.519 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T18:31:37.520 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T18:31:37.549 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T18:31:37.569 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T18:31:37.596 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T18:31:37.631 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T18:31:37.744 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T18:31:37.747 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T18:31:37.776 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T18:31:37.817 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T18:31:37.853 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T18:31:37.914 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T18:31:38.092 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T18:31:38.162 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T18:31:38.490 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T18:31:38.615 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T18:31:38.818 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T18:31:38.934 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T18:31:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T18:31:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T18:31:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T18:31:39.126 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T18:31:39.411 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T18:31:39.499 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T18:31:39.581 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T18:31:39.681 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T18:31:39.715 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T18:31:39.755 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T18:31:39.831 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T18:31:39.855 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T18:31:39.897 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T18:31:39.905 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T18:31:39.924 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T18:31:39.958 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T18:31:39.971 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T18:31:40.041 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T18:31:40.239 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T18:31:40.268 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T18:31:40.302 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T18:31:40.329 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:31:40.341 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T18:31:40.399 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T18:31:40.418 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:31:40.450 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T18:31:40.486 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T18:31:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T18:31:40.614 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T18:31:40.667 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T18:31:40.705 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T18:31:40.738 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T18:31:40.955 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T18:31:40.965 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T18:31:40.990 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:31:41.061 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:31:41.084 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T18:31:41.141 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T18:31:41.194 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T18:31:41.243 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T18:31:41.289 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T18:31:41.349 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T18:31:41.423 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T18:31:41.473 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T18:31:41.520 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T18:31:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T18:31:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T18:31:41.693 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T18:31:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T18:31:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T18:31:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T18:31:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T18:31:42.200 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T18:31:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T18:31:43.162 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T18:31:43.220 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T18:31:43.253 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T18:31:43.302 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:31:43.347 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:31:43.396 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T18:31:43.464 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T18:31:43.562 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T18:31:43.633 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T18:31:43.876 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T18:31:43.889 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:31:43.918 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T18:31:43.950 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:31:44.042 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:31:44.042 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T18:31:44.043 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T18:31:44.043 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:31:44.068 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:31:44.076 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T18:31:44.914 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T18:31:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T18:31:46.652 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T18:31:46.748 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T18:31:46.819 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T18:31:46.864 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T18:31:47.245 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T18:31:47.287 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:31:47.407 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:31:47.444 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T18:31:47.500 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T18:31:47.991 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:31:47.991 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-04-25T18:31:47.991 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-04-25T18:31:47.991 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-04-25T18:31:47.991 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-04-25T18:31:47.991 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-04-25T18:31:47.992 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-04-25T18:31:47.992 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-04-25T18:31:47.992 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:31:48.044 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T18:31:48.082 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:31:48.162 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:31:48.173 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:31:48.178 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:31:48.179 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:48.179 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T18:31:48.179 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:31:48.200 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T18:31:49.665 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:31:49.691 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:31:49.691 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:49.691 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T18:31:49.691 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:31:49.691 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:31:49.692 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:31:49.916 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:31:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:31:49.939 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:49.939 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T18:31:49.939 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:31:49.939 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:31:49.939 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:31:50.383 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:31:50.406 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:31:50.406 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:50.406 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T18:31:50.406 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:31:50.406 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:31:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:31:50.513 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T18:31:50.562 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T18:31:50.617 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T18:31:50.803 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T18:31:50.851 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T18:31:50.928 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T18:31:51.011 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:31:51.017 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:31:51.086 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:31:51.156 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:31:51.201 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T18:31:51.233 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T18:31:51.260 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T18:31:51.721 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:31:51.733 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:31:51.751 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:31:51.783 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T18:31:51.786 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:31:51.844 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T18:31:51.874 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:31:51.961 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:31:51.961 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T18:31:51.961 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T18:31:51.961 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:31:52.003 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:31:52.913 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:31:52.951 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:31:53.918 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:31:53.943 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:31:54.045 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:31:54.207 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T18:31:54.236 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:31:54.259 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:31:54.259 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:54.259 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T18:31:54.259 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:31:54.259 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:31:54.260 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:31:54.304 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:31:54.315 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:31:54.415 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T18:31:54.469 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T18:31:55.392 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T18:31:55.423 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:31:55.446 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:31:55.446 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:55.446 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T18:31:55.446 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:31:55.446 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:31:55.446 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:31:55.490 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:31:55.512 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:31:55.513 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:55.513 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T18:31:55.513 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:31:55.558 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:31:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:31:55.572 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-04-25T18:31:55.572 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-04-25T18:31:55.572 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-04-25T18:31:55.572 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-04-25T18:31:55.572 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-04-25T18:31:55.572 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /root 2024-04-25T18:31:55.572 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-04-25T18:31:55.572 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:31:55.580 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:31:55.580 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:55.580 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T18:31:55.581 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:31:55.581 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:31:55.581 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:31:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:31:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:31:55.754 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:55.754 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T18:31:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:31:55.883 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:31:55.906 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:31:55.906 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:55.906 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T18:31:55.906 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:31:55.906 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:31:55.906 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:31:57.242 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:31:57.267 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:31:57.267 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:57.267 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T18:31:57.267 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:31:57.268 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:31:57.268 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:31:57.501 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:31:57.533 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:31:57.534 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:57.534 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T18:31:57.534 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:31:57.534 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:31:57.534 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:31:57.960 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:31:57.983 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:31:57.983 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:31:57.983 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T18:31:57.983 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:31:57.983 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:31:57.983 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:31:58.081 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T18:31:58.121 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T18:31:58.176 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T18:31:58.364 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T18:31:58.410 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T18:31:58.470 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T18:31:58.559 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:31:58.706 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:31:58.744 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T18:31:58.776 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T18:31:58.812 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T18:31:59.248 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:31:59.277 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:31:59.309 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T18:31:59.370 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T18:31:59.409 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:32:00.439 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:32:00.468 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:32:00.512 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T18:32:00.553 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T18:32:00.594 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T18:32:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T18:32:00.653 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T18:32:00.837 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T18:32:00.882 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T18:32:00.912 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T18:32:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:32:01.452 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:32:01.543 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:32:01.553 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:32:01.585 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:32:01.606 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:32:01.716 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T18:32:01.763 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:32:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:32:01.787 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:01.787 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T18:32:01.787 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:32:01.787 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:32:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:32:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:32:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:32:01.939 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T18:32:01.995 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T18:32:02.435 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T18:32:02.525 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T18:32:02.575 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T18:32:02.655 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T18:32:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T18:32:02.688 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:32:02.763 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:32:02.796 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:32:02.796 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:32:02.873 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:32:02.920 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T18:32:02.944 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:32:02.949 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:32:02.971 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:32:02.972 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:02.972 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T18:32:02.972 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:32:02.972 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:32:02.972 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:32:03.016 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:32:03.038 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:32:03.038 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:03.039 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T18:32:03.039 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:32:03.084 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:32:03.106 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:32:03.106 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:03.106 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T18:32:03.107 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:32:03.107 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:32:03.107 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:32:03.418 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:32:03.440 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:32:03.440 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:03.440 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T18:32:03.440 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:32:03.440 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:32:03.440 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:32:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:32:05.035 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-04-25T18:32:05.035 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-04-25T18:32:05.035 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-04-25T18:32:05.035 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-04-25T18:32:05.035 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-04-25T18:32:05.035 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-04-25T18:32:05.035 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-04-25T18:32:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:05.299 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:32:05.362 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:32:05.362 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:05.362 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T18:32:05.362 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:06.991 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:32:07.094 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:32:07.095 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:07.095 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T18:32:07.095 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:32:07.095 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:32:07.095 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:07.350 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:32:07.442 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:32:07.443 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:07.443 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T18:32:07.443 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:32:07.443 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:32:07.443 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:07.942 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:32:08.010 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:32:08.010 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:08.011 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T18:32:08.011 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:32:08.011 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:32:08.011 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T18:32:08.063 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T18:32:08.103 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T18:32:08.135 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T18:32:08.142 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T18:32:08.163 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T18:32:08.201 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T18:32:08.270 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T18:32:08.351 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T18:32:08.401 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T18:32:08.431 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T18:32:08.500 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T18:32:08.555 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T18:32:08.639 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T18:32:08.761 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:32:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:32:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T18:32:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T18:32:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T18:32:08.857 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T18:32:08.858 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T18:32:08.859 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T18:32:08.860 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T18:32:08.861 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T18:32:08.862 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T18:32:08.863 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T18:32:08.864 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T18:32:08.865 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T18:32:08.866 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:32:08.867 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T18:32:08.938 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:32:09.030 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T18:32:09.054 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:32:09.072 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T18:32:09.087 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:32:09.103 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T18:32:09.106 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:32:09.660 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:32:09.707 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:32:09.764 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T18:32:09.872 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T18:32:09.900 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:32:09.945 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T18:32:10.041 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T18:32:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T18:32:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T18:32:10.195 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T18:32:10.195 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:32:10.304 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:32:10.338 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:32:10.338 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:32:10.426 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:32:10.502 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:32:10.987 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:32:10.987 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.988 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.989 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T18:32:10.990 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.991 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T18:32:10.992 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T18:32:10.993 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T18:32:10.994 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T18:32:10.995 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T18:32:10.996 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.997 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:10.998 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T18:32:10.998 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T18:32:10.998 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T18:32:10.998 INFO:teuthology.orchestra.run.smithi171.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T18:32:10.998 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T18:32:10.998 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T18:32:10.998 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T18:32:10.998 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:32:10.998 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-04-25T18:32:11.211 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:32:11.261 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:32:11.404 DEBUG:teuthology.parallel:result is None 2024-04-25T18:32:12.355 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:32:12.385 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:32:12.530 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:32:12.718 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T18:32:12.792 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:32:12.859 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:32:12.859 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:12.859 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T18:32:12.859 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:32:12.859 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:32:12.859 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:12.922 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:32:12.961 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:32:13.108 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T18:32:13.172 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T18:32:14.171 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T18:32:14.208 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:32:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:32:14.275 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:14.275 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T18:32:14.275 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:32:14.275 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:32:14.275 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:14.354 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:32:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:32:14.422 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:14.422 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T18:32:14.422 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:14.493 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:32:14.568 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:32:14.568 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:14.569 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T18:32:14.569 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:32:14.569 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:32:14.569 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:14.952 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:32:15.019 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:32:15.019 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:32:15.019 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T18:32:15.019 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:32:15.019 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:32:15.019 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:16.147 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:32:16.147 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T18:32:16.147 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T18:32:16.148 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T18:32:16.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T18:32:16.150 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T18:32:16.150 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T18:32:16.150 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T18:32:16.150 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T18:32:16.150 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T18:32:16.150 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T18:32:16.150 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T18:32:16.150 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T18:32:16.150 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T18:32:16.150 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T18:32:16.150 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T18:32:16.151 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T18:32:16.152 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T18:32:16.153 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T18:32:16.154 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T18:32:16.155 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T18:32:16.156 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T18:32:16.157 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T18:32:16.158 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:32:16.159 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T18:32:18.293 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:32:18.293 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:32:18.293 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-04-25T18:32:18.293 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.293 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.293 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T18:32:18.294 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T18:32:18.294 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-04-25T18:32:18.294 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T18:32:18.294 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T18:32:18.294 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T18:32:18.294 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T18:32:18.294 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.294 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.294 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.294 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.294 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:18.295 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.296 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T18:32:18.297 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T18:32:18.298 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T18:32:18.299 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T18:32:18.300 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T18:32:18.301 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T18:32:18.302 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:18.303 INFO:teuthology.orchestra.run.smithi161.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T18:32:18.304 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T18:32:18.304 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T18:32:18.304 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T18:32:18.304 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T18:32:18.304 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T18:32:18.304 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T18:32:18.304 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-04-25T18:32:18.304 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-04-25T18:32:18.670 DEBUG:teuthology.parallel:result is None 2024-04-25T18:32:20.146 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T18:32:20.215 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T18:32:20.428 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T18:32:20.694 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T18:32:20.771 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T18:32:21.104 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T18:32:21.163 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T18:32:21.191 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T18:32:22.080 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:32:22.139 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:32:22.190 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:32:23.489 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T18:32:23.626 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T18:32:23.716 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T18:32:23.926 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T18:32:23.990 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T18:32:23.990 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:32:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:32:24.171 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:32:24.171 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:32:24.319 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:32:24.449 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:32:24.775 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:32:24.775 INFO:teuthology.orchestra.run.smithi112.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T18:32:24.775 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:30.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T18:32:30.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T18:32:30.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T18:32:30.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T18:32:30.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T18:32:30.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T18:32:30.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T18:32:30.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:32:30.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:32:30.673 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T18:32:30.674 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T18:32:30.675 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T18:32:30.675 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T18:32:30.675 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T18:32:30.675 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T18:32:30.675 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T18:32:30.675 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T18:32:30.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T18:32:30.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T18:32:30.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T18:32:30.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T18:32:30.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T18:32:30.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T18:32:30.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T18:32:30.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T18:32:30.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T18:32:30.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T18:32:30.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T18:32:30.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T18:32:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T18:32:30.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T18:32:30.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T18:32:30.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T18:32:30.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T18:32:30.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T18:32:30.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T18:32:30.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T18:32:30.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T18:32:30.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T18:32:30.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T18:32:30.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T18:32:30.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T18:32:30.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T18:32:30.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T18:32:30.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T18:32:30.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T18:32:30.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T18:32:30.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T18:32:30.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T18:32:30.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T18:32:30.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T18:32:30.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T18:32:30.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T18:32:30.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T18:32:30.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T18:32:30.698 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T18:32:30.698 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T18:32:30.698 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T18:32:30.698 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T18:32:30.698 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T18:32:30.698 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:32:30.698 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:32:30.698 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T18:32:30.698 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T18:32:30.698 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T18:32:30.698 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T18:32:30.699 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T18:32:30.700 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T18:32:30.700 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T18:32:30.700 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T18:32:30.700 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T18:32:30.700 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T18:32:30.700 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T18:32:30.700 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T18:32:30.700 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T18:32:30.700 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T18:32:30.700 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T18:32:30.700 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T18:32:30.701 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T18:32:30.701 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T18:32:30.701 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T18:32:30.701 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T18:32:30.701 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T18:32:30.701 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T18:32:30.701 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T18:32:30.701 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:32:30.701 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T18:32:33.313 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:32:33.313 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:33.313 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-04-25T18:32:33.313 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.313 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.313 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T18:32:33.313 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.314 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.315 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T18:32:33.316 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T18:32:33.317 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T18:32:33.318 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T18:32:33.319 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T18:32:33.320 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T18:32:33.321 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T18:32:33.322 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T18:32:33.323 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T18:32:33.324 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T18:32:33.324 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T18:32:33.324 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T18:32:33.324 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:32:33.324 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T18:32:33.886 DEBUG:teuthology.parallel:result is None 2024-04-25T18:32:33.887 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:32:34.034 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T18:32:34.089 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T18:32:34.089 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T18:32:34.089 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T18:32:34.091 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:32:34.311 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T18:32:34.337 INFO:teuthology.orchestra.run.smithi161.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T18:32:34.337 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T18:32:34.337 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T18:32:34.339 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:32:34.490 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T18:32:34.516 INFO:teuthology.orchestra.run.smithi171.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T18:32:34.516 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T18:32:34.516 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T18:32:34.518 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T18:32:34.518 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:32:34.518 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T18:32:34.588 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-25T18:32:34.589 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T18:32:34.620 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T18:32:34.620 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T18:32:34.651 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T18:32:34.651 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:32:34.651 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T18:32:34.720 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T18:32:34.824 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-25T18:32:34.824 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T18:32:34.853 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T18:32:34.920 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T18:32:34.920 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T18:32:34.949 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T18:32:35.015 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T18:32:35.015 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:32:35.015 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T18:32:35.088 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T18:32:35.193 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-25T18:32:35.194 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T18:32:35.218 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T18:32:35.281 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T18:32:35.281 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T18:32:35.306 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T18:32:35.384 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T18:32:35.385 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:32:35.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T18:32:35.451 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T18:32:35.557 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-25T18:32:35.557 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T18:32:35.582 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T18:32:35.645 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T18:32:35.645 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T18:32:35.669 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T18:32:35.731 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T18:32:35.832 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T18:32:35.832 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:32:35.833 INFO:tasks.cephadm:Cluster fsid is 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:32:35.833 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T18:32:35.833 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.112', 'mon.b': '172.21.15.161', 'mon.c': '172.21.15.171'} 2024-04-25T18:32:35.833 INFO:tasks.cephadm:First mon is mon.a on smithi112 2024-04-25T18:32:35.833 INFO:tasks.cephadm:First mgr is x 2024-04-25T18:32:35.834 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T18:32:35.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-04-25T18:32:35.912 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-04-25T18:32:35.936 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname $(hostname -s) 2024-04-25T18:32:35.960 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T18:32:35.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:32:36.104 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T18:32:36.316 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:32:36.317 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T18:32:36.317 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T18:32:37.013 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 18:32 /home/ubuntu/cephtest/cephadm 2024-04-25T18:32:37.014 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T18:32:37.669 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 18:32 /home/ubuntu/cephtest/cephadm 2024-04-25T18:32:37.669 DEBUG:teuthology.orchestra.run.smithi171:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T18:32:38.159 INFO:teuthology.orchestra.run.smithi171.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 18:32 /home/ubuntu/cephtest/cephadm 2024-04-25T18:32:38.159 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T18:32:38.204 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T18:32:38.223 DEBUG:teuthology.orchestra.run.smithi171:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T18:32:38.256 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T18:32:38.257 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T18:32:38.261 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T18:32:38.265 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T18:32:39.822 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:32:39.844 INFO:teuthology.orchestra.run.smithi171.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:32:40.565 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:33:10.799 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-25T18:33:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T18:33:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T18:33:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "repo_digests": [ 2024-04-25T18:33:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T18:33:10.800 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-04-25T18:33:10.800 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-25T18:33:10.895 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-04-25T18:33:10.896 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T18:33:10.896 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T18:33:10.896 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-04-25T18:33:10.896 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T18:33:10.896 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-04-25T18:33:10.896 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-04-25T18:33:24.813 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:33:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T18:33:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T18:33:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-04-25T18:33:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T18:33:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:33:24.814 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:33:24.852 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-04-25T18:33:24.935 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-04-25T18:33:24.967 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph 2024-04-25T18:33:24.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-04-25T18:33:25.086 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-04-25T18:33:25.114 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /etc/ceph 2024-04-25T18:33:25.142 INFO:tasks.cephadm:Writing seed config... 2024-04-25T18:33:25.143 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T18:33:25.143 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T18:33:25.143 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T18:33:25.143 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T18:33:25.143 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T18:33:25.143 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T18:33:25.143 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T18:33:25.143 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T18:33:25.144 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T18:33:25.145 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T18:33:25.146 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T18:33:25.147 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:33:25.147 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T18:33:25.201 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2fe18a52-0332-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T18:33:25.201 DEBUG:teuthology.orchestra.run.smithi112:mon.a> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T18:33:25.246 DEBUG:teuthology.orchestra.run.smithi112:mgr.x> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T18:33:25.293 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T18:33:25.293 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.112 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:33:25.558 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:33:25.558 INFO:teuthology.orchestra.run.smithi112.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '2fe18a52-0332-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.112', '--skip-admin-label'] 2024-04-25T18:33:25.559 INFO:teuthology.orchestra.run.smithi112.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T18:33:25.559 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying podman|docker is present... 2024-04-25T18:33:25.613 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T18:33:25.614 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying lvm2 is present... 2024-04-25T18:33:25.614 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying time synchronization is in place... 2024-04-25T18:33:25.629 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T18:33:25.629 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T18:33:25.646 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T18:33:25.646 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T18:33:25.665 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-04-25T18:33:25.683 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-04-25T18:33:25.683 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-04-25T18:33:25.683 INFO:teuthology.orchestra.run.smithi112.stdout:Repeating the final host check... 2024-04-25T18:33:25.736 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T18:33:25.736 INFO:teuthology.orchestra.run.smithi112.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T18:33:25.736 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl is present 2024-04-25T18:33:25.736 INFO:teuthology.orchestra.run.smithi112.stdout:lvcreate is present 2024-04-25T18:33:25.752 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T18:33:25.753 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T18:33:25.770 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T18:33:25.770 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T18:33:25.790 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-04-25T18:33:25.810 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-04-25T18:33:25.810 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-04-25T18:33:25.810 INFO:teuthology.orchestra.run.smithi112.stdout:Host looks OK 2024-04-25T18:33:25.810 INFO:teuthology.orchestra.run.smithi112.stdout:Cluster fsid: 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:33:25.811 INFO:teuthology.orchestra.run.smithi112.stdout:Acquiring lock 140191126416544 on /run/cephadm/2fe18a52-0332-11ef-bc93-c7b262605968.lock 2024-04-25T18:33:25.811 INFO:teuthology.orchestra.run.smithi112.stdout:Lock 140191126416544 acquired on /run/cephadm/2fe18a52-0332-11ef-bc93-c7b262605968.lock 2024-04-25T18:33:25.812 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 3300 ... 2024-04-25T18:33:25.813 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 6789 ... 2024-04-25T18:33:25.814 INFO:teuthology.orchestra.run.smithi112.stdout:Base mon IP(s) is [172.21.15.112:3300, 172.21.15.112:6789], mon addrv is [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-04-25T18:33:25.828 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.112 metric 100 2024-04-25T18:33:25.828 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.112 metric 100 2024-04-25T18:33:25.842 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T18:33:25.843 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-25T18:33:25.843 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2024-04-25T18:33:25.851 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T18:33:25.851 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T18:33:25.851 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T18:33:25.851 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T18:33:25.851 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2024-04-25T18:33:25.851 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T18:33:25.853 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-04-25T18:33:25.853 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-04-25T18:33:25.853 INFO:teuthology.orchestra.run.smithi112.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T18:33:25.855 INFO:teuthology.orchestra.run.smithi112.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T18:33:25.858 INFO:teuthology.orchestra.run.smithi112.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:33:26.574 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T18:33:26.575 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:33:26.575 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T18:33:26.575 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T18:33:26.575 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T18:33:26.575 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T18:33:26.575 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T18:33:27.955 INFO:teuthology.orchestra.run.smithi112.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T18:33:27.955 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T18:33:27.955 INFO:teuthology.orchestra.run.smithi112.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T18:33:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:stat: stdout 167 167 2024-04-25T18:33:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial keys... 2024-04-25T18:33:30.309 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQD5oSpmBx1KKBAACqIXkd/kXiEcWa6UfgA26Q== 2024-04-25T18:33:31.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQD6oSpmEIwXMBAAJJJerKNrbXjerKuB+Rny/g== 2024-04-25T18:33:32.609 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQD7oSpmPpRYOBAAGW769TJ7bsSmSObz1U8hiw== 2024-04-25T18:33:32.611 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial monmap... 2024-04-25T18:33:33.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T18:33:33.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T18:33:33.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:33:33.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T18:33:33.831 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool for a [v2:172.21.15.112:3300,v1:172.21.15.112:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T18:33:33.831 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = quincy 2024-04-25T18:33:33.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: set fsid to 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:33:33.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T18:33:33.831 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:33:33.831 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mon... 2024-04-25T18:33:35.089 INFO:teuthology.orchestra.run.smithi112.stdout:create mon.a on 2024-04-25T18:33:35.461 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T18:33:35.697 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T18:33:35.973 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2fe18a52-0332-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-2fe18a52-0332-11ef-bc93-c7b262605968.target. 2024-04-25T18:33:35.974 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2fe18a52-0332-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-2fe18a52-0332-11ef-bc93-c7b262605968.target. 2024-04-25T18:33:36.344 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.a 2024-04-25T18:33:36.344 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T18:33:36.626 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2fe18a52-0332-11ef-bc93-c7b262605968.target.wants/ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-2fe18a52-0332-11ef-bc93-c7b262605968@.service. 2024-04-25T18:33:37.294 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:37 smithi112 podman[27477]: 2024-04-25 18:33:37.292501381 +0000 UTC m=+0.405932071 container init 2947da3e3b884ceda002c565971db213eb149fd58a50da668244e0c30ac31d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T18:33:37.404 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:33:37.404 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-25T18:33:37.419 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:33:37.419 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T18:33:37.419 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-25T18:33:37.419 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T18:33:37.419 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon to start... 2024-04-25T18:33:37.420 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon... 2024-04-25T18:33:37.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:37 smithi112 podman[27477]: 2024-04-25 18:33:37.302900773 +0000 UTC m=+0.416331423 container start 2947da3e3b884ceda002c565971db213eb149fd58a50da668244e0c30ac31d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:33:37.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:37 smithi112 bash[27477]: 2947da3e3b884ceda002c565971db213eb149fd58a50da668244e0c30ac31d70 2024-04-25T18:33:37.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:37 smithi112 systemd[1]: Started Ceph mon.a for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:33:37.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:37 smithi112 ceph-mon[27495]: mkfs 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:33:37.567 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:37 smithi112 ceph-mon[27495]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T18:33:38.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T18:33:38.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: monmap epoch 1 2024-04-25T18:33:38.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:33:38.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: last_changed 2024-04-25T18:33:33.098667+0000 2024-04-25T18:33:38.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: created 2024-04-25T18:33:33.098667+0000 2024-04-25T18:33:38.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: min_mon_release 19 (squid) 2024-04-25T18:33:38.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: election_strategy: 1 2024-04-25T18:33:38.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T18:33:38.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: fsmap 2024-04-25T18:33:38.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T18:33:38.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: mgrmap e1: no daemons active 2024-04-25T18:33:38.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:38 smithi112 ceph-mon[27495]: from='client.? 172.21.15.112:0/21637482' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T18:33:38.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T18:33:38.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout id: 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:33:38.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T18:33:38.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:33:38.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout services: 2024-04-25T18:33:38.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.753457s) 2024-04-25T18:33:38.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T18:33:38.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T18:33:38.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:33:38.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout data: 2024-04-25T18:33:38.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T18:33:38.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T18:33:38.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T18:33:38.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T18:33:38.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:33:38.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.039+0000 7f4a58400700 1 Processor -- start 2024-04-25T18:33:38.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.040+0000 7f4a58400700 1 -- start start 2024-04-25T18:33:38.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.040+0000 7f4a58400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a50082b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:33:38.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.040+0000 7f4a58400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a50083110 con 0x7f4a50082720 2024-04-25T18:33:38.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.041+0000 7f4a55a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a50082b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:33:38.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.041+0000 7f4a55a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a50082b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48968/0 (socket says 172.21.15.112:48968) 2024-04-25T18:33:38.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.041+0000 7f4a55a00700 1 -- 172.21.15.112:0/4105097701 learned_addr learned my addr 172.21.15.112:0/4105097701 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:33:38.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.042+0000 7f4a55a00700 1 -- 172.21.15.112:0/4105097701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5010fef0 con 0x7f4a50082720 2024-04-25T18:33:38.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.042+0000 7f4a55a00700 1 --2- 172.21.15.112:0/4105097701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a50082b40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4a440098a0 tx=0x7f4a44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52fff2a6f7548d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:33:38.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.042+0000 7f4a4fe00700 1 -- 172.21.15.112:0/4105097701 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a44004160 con 0x7f4a50082720 2024-04-25T18:33:38.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.043+0000 7f4a4fe00700 1 -- 172.21.15.112:0/4105097701 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a44004300 con 0x7f4a50082720 2024-04-25T18:33:38.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.043+0000 7f4a4fe00700 1 -- 172.21.15.112:0/4105097701 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a440036a0 con 0x7f4a50082720 2024-04-25T18:33:38.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.043+0000 7f4a58400700 1 -- 172.21.15.112:0/4105097701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 msgr2=0x7f4a50082b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:33:38.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.043+0000 7f4a58400700 1 --2- 172.21.15.112:0/4105097701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a50082b40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4a440098a0 tx=0x7f4a44009bb0 comp rx=0 tx=0).stop 2024-04-25T18:33:38.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.043+0000 7f4a58400700 1 -- 172.21.15.112:0/4105097701 shutdown_connections 2024-04-25T18:33:38.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.043+0000 7f4a58400700 1 --2- 172.21.15.112:0/4105097701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a50082b40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.043+0000 7f4a58400700 1 -- 172.21.15.112:0/4105097701 >> 172.21.15.112:0/4105097701 conn(0x7f4a5007dce0 msgr2=0x7f4a50080100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.043+0000 7f4a58400700 1 -- 172.21.15.112:0/4105097701 shutdown_connections 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.043+0000 7f4a58400700 1 -- 172.21.15.112:0/4105097701 wait complete. 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.045+0000 7f4a58400700 1 Processor -- start 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.045+0000 7f4a58400700 1 -- start start 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.045+0000 7f4a58400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a501a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.045+0000 7f4a58400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a50110420 con 0x7f4a50082720 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.045+0000 7f4a55a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a501a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.045+0000 7f4a55a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a501a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48972/0 (socket says 172.21.15.112:48972) 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.045+0000 7f4a55a00700 1 -- 172.21.15.112:0/21637482 learned_addr learned my addr 172.21.15.112:0/21637482 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.046+0000 7f4a55a00700 1 -- 172.21.15.112:0/21637482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a501a2840 con 0x7f4a50082720 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.046+0000 7f4a55a00700 1 --2- 172.21.15.112:0/21637482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a501a2300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000c00 tx=0x7f4a44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:33:38.844 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.046+0000 7f4a4e000700 1 -- 172.21.15.112:0/21637482 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a44003db0 con 0x7f4a50082720 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.046+0000 7f4a4e000700 1 -- 172.21.15.112:0/21637482 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a44011630 con 0x7f4a50082720 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.046+0000 7f4a58400700 1 -- 172.21.15.112:0/21637482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a501a2ad0 con 0x7f4a50082720 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.046+0000 7f4a4e000700 1 -- 172.21.15.112:0/21637482 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a44011950 con 0x7f4a50082720 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.047+0000 7f4a58400700 1 -- 172.21.15.112:0/21637482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a501a57c0 con 0x7f4a50082720 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.047+0000 7f4a4e000700 1 -- 172.21.15.112:0/21637482 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4a4401a460 con 0x7f4a50082720 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.047+0000 7f4a4e000700 1 -- 172.21.15.112:0/21637482 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4a4401b780 con 0x7f4a50082720 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.048+0000 7f4a58400700 1 -- 172.21.15.112:0/21637482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a500640a0 con 0x7f4a50082720 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.051+0000 7f4a4e000700 1 -- 172.21.15.112:0/21637482 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4a4401b980 con 0x7f4a50082720 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.130+0000 7f4a58400700 1 -- 172.21.15.112:0/21637482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4a5002e280 con 0x7f4a50082720 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.131+0000 7f4a4e000700 1 -- 172.21.15.112:0/21637482 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f4a4401bb60 con 0x7f4a50082720 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.132+0000 7f4a58400700 1 -- 172.21.15.112:0/21637482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 msgr2=0x7f4a501a2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:33:38.845 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.132+0000 7f4a58400700 1 --2- 172.21.15.112:0/21637482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a501a2300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000c00 tx=0x7f4a44004750 comp rx=0 tx=0).stop 2024-04-25T18:33:38.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.132+0000 7f4a58400700 1 -- 172.21.15.112:0/21637482 shutdown_connections 2024-04-25T18:33:38.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.132+0000 7f4a58400700 1 --2- 172.21.15.112:0/21637482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a50082720 0x7f4a501a2300 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:33:38.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.132+0000 7f4a58400700 1 -- 172.21.15.112:0/21637482 >> 172.21.15.112:0/21637482 conn(0x7f4a5007dce0 msgr2=0x7f4a50199000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:33:38.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.132+0000 7f4a58400700 1 -- 172.21.15.112:0/21637482 shutdown_connections 2024-04-25T18:33:38.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:38.132+0000 7f4a58400700 1 -- 172.21.15.112:0/21637482 wait complete. 2024-04-25T18:33:38.846 INFO:teuthology.orchestra.run.smithi112.stdout:mon is available 2024-04-25T18:33:38.846 INFO:teuthology.orchestra.run.smithi112.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T18:33:39.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:39 smithi112 ceph-mon[27495]: from='client.? 172.21.15.112:0/3904999530' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T18:33:39.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:39 smithi112 ceph-mon[27495]: from='client.? 172.21.15.112:0/3904999530' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T18:33:40.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:33:40.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T18:33:40.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:33:40.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T18:33:40.135 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-04-25T18:33:40.135 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T18:33:40.136 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.387+0000 7f5ee9800700 1 Processor -- start 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.387+0000 7f5ee9800700 1 -- start start 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.387+0000 7f5ee9800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee4108b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.387+0000 7f5ee9800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee4109100 con 0x7f5ee4108710 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.388+0000 7f5ee2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee4108b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.388+0000 7f5ee2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee4108b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48986/0 (socket says 172.21.15.112:48986) 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.388+0000 7f5ee2a00700 1 -- 172.21.15.112:0/1233906385 learned_addr learned my addr 172.21.15.112:0/1233906385 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.388+0000 7f5ee2a00700 1 -- 172.21.15.112:0/1233906385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee4109910 con 0x7f5ee4108710 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.388+0000 7f5ee2a00700 1 --2- 172.21.15.112:0/1233906385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee4108b30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40098a0 tx=0x7f5ed4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f051b973498f22d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.389+0000 7f5ee1600700 1 -- 172.21.15.112:0/1233906385 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed4004160 con 0x7f5ee4108710 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.389+0000 7f5ee1600700 1 -- 172.21.15.112:0/1233906385 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed4004300 con 0x7f5ee4108710 2024-04-25T18:33:40.137 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.389+0000 7f5ee1600700 1 -- 172.21.15.112:0/1233906385 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed40036a0 con 0x7f5ee4108710 2024-04-25T18:33:40.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.389+0000 7f5ee9800700 1 -- 172.21.15.112:0/1233906385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 msgr2=0x7f5ee4108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:33:40.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.389+0000 7f5ee9800700 1 --2- 172.21.15.112:0/1233906385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee4108b30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40098a0 tx=0x7f5ed4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:33:40.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.389+0000 7f5ee9800700 1 -- 172.21.15.112:0/1233906385 shutdown_connections 2024-04-25T18:33:40.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.389+0000 7f5ee9800700 1 --2- 172.21.15.112:0/1233906385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee4108b30 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:33:40.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.390+0000 7f5ee9800700 1 -- 172.21.15.112:0/1233906385 >> 172.21.15.112:0/1233906385 conn(0x7f5ee4103cf0 msgr2=0x7f5ee4106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:33:40.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.390+0000 7f5ee9800700 1 -- 172.21.15.112:0/1233906385 shutdown_connections 2024-04-25T18:33:40.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.390+0000 7f5ee9800700 1 -- 172.21.15.112:0/1233906385 wait complete. 2024-04-25T18:33:40.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.391+0000 7f5ee9800700 1 Processor -- start 2024-04-25T18:33:40.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.391+0000 7f5ee9800700 1 -- start start 2024-04-25T18:33:40.141 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.391+0000 7f5ee9800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee419dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.391+0000 7f5ee9800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee4109e40 con 0x7f5ee4108710 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.391+0000 7f5ee2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee419dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.391+0000 7f5ee2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee419dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48988/0 (socket says 172.21.15.112:48988) 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.391+0000 7f5ee2a00700 1 -- 172.21.15.112:0/3904999530 learned_addr learned my addr 172.21.15.112:0/3904999530 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.392+0000 7f5ee2a00700 1 -- 172.21.15.112:0/3904999530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee419e400 con 0x7f5ee4108710 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.392+0000 7f5ee2a00700 1 --2- 172.21.15.112:0/3904999530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee419dec0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4000c00 tx=0x7f5ed4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.392+0000 7f5edb400700 1 -- 172.21.15.112:0/3904999530 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed4003db0 con 0x7f5ee4108710 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.392+0000 7f5edb400700 1 -- 172.21.15.112:0/3904999530 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed4011630 con 0x7f5ee4108710 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.392+0000 7f5ee9800700 1 -- 172.21.15.112:0/3904999530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee419e690 con 0x7f5ee4108710 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.392+0000 7f5edb400700 1 -- 172.21.15.112:0/3904999530 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed4011950 con 0x7f5ee4108710 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.392+0000 7f5ee9800700 1 -- 172.21.15.112:0/3904999530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee41a1380 con 0x7f5ee4108710 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.393+0000 7f5edb400700 1 -- 172.21.15.112:0/3904999530 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5ed4011af0 con 0x7f5ee4108710 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.393+0000 7f5edb400700 1 -- 172.21.15.112:0/3904999530 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ed401c790 con 0x7f5ee4108710 2024-04-25T18:33:40.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.394+0000 7f5ed8c00700 1 -- 172.21.15.112:0/3904999530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee40640a0 con 0x7f5ee4108710 2024-04-25T18:33:40.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.397+0000 7f5edb400700 1 -- 172.21.15.112:0/3904999530 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5ed4032020 con 0x7f5ee4108710 2024-04-25T18:33:40.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.473+0000 7f5ed8c00700 1 -- 172.21.15.112:0/3904999530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5ee419eb70 con 0x7f5ee4108710 2024-04-25T18:33:40.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.486+0000 7f5edb400700 1 -- 172.21.15.112:0/3904999530 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+494 (secure 0 0 0) 0x7f5ee419eb70 con 0x7f5ee4108710 2024-04-25T18:33:40.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.487+0000 7f5ed8c00700 1 -- 172.21.15.112:0/3904999530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 msgr2=0x7f5ee419dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:33:40.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.487+0000 7f5ed8c00700 1 --2- 172.21.15.112:0/3904999530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee419dec0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4000c00 tx=0x7f5ed4004750 comp rx=0 tx=0).stop 2024-04-25T18:33:40.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.487+0000 7f5ed8c00700 1 -- 172.21.15.112:0/3904999530 shutdown_connections 2024-04-25T18:33:40.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.487+0000 7f5ed8c00700 1 --2- 172.21.15.112:0/3904999530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ee4108710 0x7f5ee419dec0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:33:40.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.487+0000 7f5ed8c00700 1 -- 172.21.15.112:0/3904999530 >> 172.21.15.112:0/3904999530 conn(0x7f5ee4103cf0 msgr2=0x7f5ee407d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:33:40.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.488+0000 7f5ed8c00700 1 -- 172.21.15.112:0/3904999530 shutdown_connections 2024-04-25T18:33:40.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:39.488+0000 7f5ed8c00700 1 -- 172.21.15.112:0/3904999530 wait complete. 2024-04-25T18:33:40.143 INFO:teuthology.orchestra.run.smithi112.stdout:Generating new minimal ceph.conf... 2024-04-25T18:33:41.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:40 smithi112 ceph-mon[27495]: from='client.? 172.21.15.112:0/1702953486' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:33:41.582 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.752+0000 7fa96d600700 1 Processor -- start 2024-04-25T18:33:41.582 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.752+0000 7fa96d600700 1 -- start start 2024-04-25T18:33:41.582 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.753+0000 7fa96d600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa968108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:33:41.582 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.753+0000 7fa96d600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa968108f10 con 0x7fa968108520 2024-04-25T18:33:41.582 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.753+0000 7fa967e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa968108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:33:41.582 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.753+0000 7fa967e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa968108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44866/0 (socket says 172.21.15.112:44866) 2024-04-25T18:33:41.582 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.753+0000 7fa967e00700 1 -- 172.21.15.112:0/4151144801 learned_addr learned my addr 172.21.15.112:0/4151144801 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:33:41.584 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.753+0000 7fa967e00700 1 -- 172.21.15.112:0/4151144801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa968109720 con 0x7fa968108520 2024-04-25T18:33:41.584 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.754+0000 7fa967e00700 1 --2- 172.21.15.112:0/4151144801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa968108940 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa9580098a0 tx=0x7fa958009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a2a69c8849fa5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:33:41.584 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.754+0000 7fa966a00700 1 -- 172.21.15.112:0/4151144801 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa958004160 con 0x7fa968108520 2024-04-25T18:33:41.584 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.754+0000 7fa966a00700 1 -- 172.21.15.112:0/4151144801 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa958004300 con 0x7fa968108520 2024-04-25T18:33:41.584 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.754+0000 7fa966a00700 1 -- 172.21.15.112:0/4151144801 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa958011af0 con 0x7fa968108520 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.755+0000 7fa96d600700 1 -- 172.21.15.112:0/4151144801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 msgr2=0x7fa968108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.755+0000 7fa96d600700 1 --2- 172.21.15.112:0/4151144801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa968108940 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa9580098a0 tx=0x7fa958009bb0 comp rx=0 tx=0).stop 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.755+0000 7fa96d600700 1 -- 172.21.15.112:0/4151144801 shutdown_connections 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.755+0000 7fa96d600700 1 --2- 172.21.15.112:0/4151144801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa968108940 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.755+0000 7fa96d600700 1 -- 172.21.15.112:0/4151144801 >> 172.21.15.112:0/4151144801 conn(0x7fa968103b40 msgr2=0x7fa968105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.755+0000 7fa96d600700 1 -- 172.21.15.112:0/4151144801 shutdown_connections 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.755+0000 7fa96d600700 1 -- 172.21.15.112:0/4151144801 wait complete. 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.756+0000 7fa96d600700 1 Processor -- start 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.756+0000 7fa96d600700 1 -- start start 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.757+0000 7fa96d600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa96819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.757+0000 7fa96d600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa96810c950 con 0x7fa968108520 2024-04-25T18:33:41.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.757+0000 7fa967e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa96819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.757+0000 7fa967e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa96819c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44878/0 (socket says 172.21.15.112:44878) 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.757+0000 7fa967e00700 1 -- 172.21.15.112:0/1702953486 learned_addr learned my addr 172.21.15.112:0/1702953486 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.757+0000 7fa967e00700 1 -- 172.21.15.112:0/1702953486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96819c930 con 0x7fa968108520 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.757+0000 7fa967e00700 1 --2- 172.21.15.112:0/1702953486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa96819c3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa9580117b0 tx=0x7fa958011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.758+0000 7fa964c00700 1 -- 172.21.15.112:0/1702953486 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa958003af0 con 0x7fa968108520 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.758+0000 7fa964c00700 1 -- 172.21.15.112:0/1702953486 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa958020070 con 0x7fa968108520 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.758+0000 7fa96d600700 1 -- 172.21.15.112:0/1702953486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96819cbc0 con 0x7fa968108520 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.758+0000 7fa964c00700 1 -- 172.21.15.112:0/1702953486 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa958019630 con 0x7fa968108520 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.758+0000 7fa96d600700 1 -- 172.21.15.112:0/1702953486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96819f8b0 con 0x7fa968108520 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.758+0000 7fa964c00700 1 -- 172.21.15.112:0/1702953486 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa958019c30 con 0x7fa968108520 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.758+0000 7fa964c00700 1 -- 172.21.15.112:0/1702953486 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa95800b720 con 0x7fa968108520 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.760+0000 7fa96d600700 1 -- 172.21.15.112:0/1702953486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9680640a0 con 0x7fa968108520 2024-04-25T18:33:41.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.762+0000 7fa964c00700 1 -- 172.21.15.112:0/1702953486 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa9580042a0 con 0x7fa968108520 2024-04-25T18:33:41.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.840+0000 7fa96d600700 1 -- 172.21.15.112:0/1702953486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa96819fdf0 con 0x7fa968108520 2024-04-25T18:33:41.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.840+0000 7fa964c00700 1 -- 172.21.15.112:0/1702953486 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fa95801a9c0 con 0x7fa968108520 2024-04-25T18:33:41.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.842+0000 7fa96d600700 1 -- 172.21.15.112:0/1702953486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 msgr2=0x7fa96819c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:33:41.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.842+0000 7fa96d600700 1 --2- 172.21.15.112:0/1702953486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa96819c3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa9580117b0 tx=0x7fa958011b70 comp rx=0 tx=0).stop 2024-04-25T18:33:41.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.842+0000 7fa96d600700 1 -- 172.21.15.112:0/1702953486 shutdown_connections 2024-04-25T18:33:41.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.842+0000 7fa96d600700 1 --2- 172.21.15.112:0/1702953486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa968108520 0x7fa96819c3f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:33:41.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.842+0000 7fa96d600700 1 -- 172.21.15.112:0/1702953486 >> 172.21.15.112:0/1702953486 conn(0x7fa968103b40 msgr2=0x7fa9681046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:33:41.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.842+0000 7fa96d600700 1 -- 172.21.15.112:0/1702953486 shutdown_connections 2024-04-25T18:33:41.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:40.842+0000 7fa96d600700 1 -- 172.21.15.112:0/1702953486 wait complete. 2024-04-25T18:33:41.590 INFO:teuthology.orchestra.run.smithi112.stdout:Restarting the monitor... 2024-04-25T18:33:41.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:41 smithi112 systemd[1]: Stopping Ceph mon.a for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:33:42.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:42 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a[27491]: 2024-04-25T18:33:42.059+0000 7f4b38a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:33:42.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:42 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a[27491]: 2024-04-25T18:33:42.059+0000 7f4b38a00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T18:33:42.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:42 smithi112 podman[27739]: 2024-04-25 18:33:42.295758813 +0000 UTC m=+0.372653934 container died 2947da3e3b884ceda002c565971db213eb149fd58a50da668244e0c30ac31d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T18:33:45.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:45 smithi112 podman[27739]: 2024-04-25 18:33:45.253681811 +0000 UTC m=+3.330576942 container cleanup 2947da3e3b884ceda002c565971db213eb149fd58a50da668244e0c30ac31d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:33:45.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:45 smithi112 bash[27739]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a 2024-04-25T18:33:52.362 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:52 smithi112 podman[27753]: 2024-04-25 18:33:52.08176417 +0000 UTC m=+9.765015615 container remove 2947da3e3b884ceda002c565971db213eb149fd58a50da668244e0c30ac31d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T18:33:52.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:52 smithi112 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T18:33:52.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:52 smithi112 systemd[1]: Stopped Ceph mon.a for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:33:52.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:52 smithi112 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.837s CPU time. 2024-04-25T18:33:52.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:52 smithi112 systemd[1]: Starting Ceph mon.a for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:33:53.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:52 smithi112 podman[27845]: 2024-04-25 18:33:52.878089907 +0000 UTC m=+0.036627864 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:33:53.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 podman[27845]: 2024-04-25 18:33:53.009031508 +0000 UTC m=+0.167569491 container create 8e659435d39e1dc00ddec175623d7d37a80a8826781e84c5c9f9cb4d651ae6e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T18:33:53.445 INFO:teuthology.orchestra.run.smithi112.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T18:33:53.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 podman[27845]: 2024-04-25 18:33:53.277277553 +0000 UTC m=+0.435815575 container init 8e659435d39e1dc00ddec175623d7d37a80a8826781e84c5c9f9cb4d651ae6e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T18:33:53.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 podman[27845]: 2024-04-25 18:33:53.28713543 +0000 UTC m=+0.445673416 container start 8e659435d39e1dc00ddec175623d7d37a80a8826781e84c5c9f9cb4d651ae6e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T18:33:53.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T18:33:53.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T18:33:53.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: pidfile_write: ignore empty --pid-file 2024-04-25T18:33:53.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: load: jerasure load: lrc 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: RocksDB version: 7.9.2 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Git sha 0 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: DB SUMMARY 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: DB Session ID: 8LXXNSVQES0FCM3LJS99 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: CURRENT file: CURRENT 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106058 ; 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.error_if_exists: 0 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.create_if_missing: 0 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.paranoid_checks: 1 2024-04-25T18:33:53.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T18:33:53.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T18:33:53.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T18:33:53.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.env: 0x55aacbcd4cc0 2024-04-25T18:33:53.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T18:33:53.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.info_log: 0x55ab05529c40 2024-04-25T18:33:53.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T18:33:53.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.statistics: (nil) 2024-04-25T18:33:53.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.use_fsync: 0 2024-04-25T18:33:53.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_log_file_size: 0 2024-04-25T18:33:53.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.allow_fallocate: 1 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.use_direct_reads: 0 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.db_log_dir: 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.wal_dir: 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T18:33:53.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.write_buffer_manager: 0x55ab054bd0e0 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.unordered_write: 0 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T18:33:53.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T18:33:53.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T18:33:53.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.row_cache: None 2024-04-25T18:33:53.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.wal_filter: None 2024-04-25T18:33:53.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T18:33:53.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T18:33:53.602 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.two_write_queues: 0 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.wal_compression: 0 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.atomic_flush: 0 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.log_readahead_size: 0 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T18:33:53.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_background_jobs: 2 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_background_compactions: -1 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_subcompactions: 1 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_open_files: -1 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T18:33:53.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_background_flushes: -1 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Compression algorithms supported: 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: kZSTD supported: 0 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: kXpressCompression supported: 0 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: kZlibCompression supported: 1 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: kSnappyCompression supported: 1 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: kLZ4Compression supported: 1 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: kBZip2Compression supported: 0 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.merge_operator: 2024-04-25T18:33:53.605 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_filter: None 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ab05529e00) 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks: 1 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_top_level_index_and_filter: 1 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_type: 0 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_index_type: 0 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_shortening: 1 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T18:33:53.606 INFO:journalctl@ceph.mon.a.smithi112.stdout: checksum: 4 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: no_block_cache: 0 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache: 0x55ab054ef1f0 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_name: BinnedLRUCache 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_options: 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: capacity : 536870912 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_shard_bits : 4 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: strict_capacity_limit : 0 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: high_pri_pool_ratio: 0.000 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_compressed: (nil) 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: persistent_cache: (nil) 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size: 4096 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size_deviation: 10 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_restart_interval: 16 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_block_restart_interval: 1 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: metadata_block_size: 4096 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: partition_filters: 0 2024-04-25T18:33:53.607 INFO:journalctl@ceph.mon.a.smithi112.stdout: use_delta_encoding: 1 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: filter_policy: bloomfilter 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: whole_key_filtering: 1 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: verify_compression: 0 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: read_amp_bytes_per_bit: 0 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: format_version: 5 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_index_compression: 1 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_align: 0 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: max_auto_readahead_size: 262144 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: prepopulate_block_cache: 0 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: initial_auto_readahead_size: 8192 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compression: NoCompression 2024-04-25T18:33:53.608 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.num_levels: 7 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T18:33:53.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T18:33:53.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T18:33:53.611 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T18:33:53.611 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T18:33:53.611 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T18:33:53.611 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T18:33:53.612 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.inplace_update_support: 0 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.bloom_locality: 0 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.max_successive_merges: 0 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T18:33:53.613 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.ttl: 2592000 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.enable_blob_files: false 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.min_blob_size: 0 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T18:33:53.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 199c2cb2-75e8-4457-89af-68e527dab01e 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070033344810, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070033346300, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101110, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98999, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14405, "raw_average_key_size": 49, "raw_value_size": 91178, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714070033, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "199c2cb2-75e8-4457-89af-68e527dab01e", "db_session_id": "8LXXNSVQES0FCM3LJS99", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070033346496, "job": 1, "event": "recovery_finished"} 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ab05604000 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: DB pointer 0x55ab055ee000 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T18:33:53.615 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** DB Stats ** 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: L0 2/0 100.60 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 106.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: Sum 2/0 100.60 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 106.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T18:33:53.616 INFO:journalctl@ceph.mon.a.smithi112.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 106.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 106.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative compaction: 0.00 GB write, 13.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval compaction: 0.00 GB write, 13.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T18:33:53.617 INFO:journalctl@ceph.mon.a.smithi112.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache BinnedLRUCache@0x55ab054ef1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.5e-05 secs_since: 0 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: starting mon.a rank 0 at public addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] at bind addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: mon.a@-1(???) e1 preinit fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: mon.a@-1(???).mds e1 new map 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: mon.a@-1(???).mds e1 print_map 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout: e1 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout: legacy client fscid: -1 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout: No filesystems configured 2024-04-25T18:33:53.618 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: monmap epoch 1 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: last_changed 2024-04-25T18:33:33.098667+0000 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: created 2024-04-25T18:33:33.098667+0000 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: min_mon_release 19 (squid) 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: election_strategy: 1 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: fsmap 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 ceph-mon[27864]: mgrmap e1: no daemons active 2024-04-25T18:33:53.619 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 bash[27845]: 8e659435d39e1dc00ddec175623d7d37a80a8826781e84c5c9f9cb4d651ae6e6 2024-04-25T18:33:53.620 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:53 smithi112 systemd[1]: Started Ceph mon.a for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:33:54.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.085+0000 7f8cce600700 1 Processor -- start 2024-04-25T18:33:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.085+0000 7f8cce600700 1 -- start start 2024-04-25T18:33:54.826 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.086+0000 7f8cce600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc8108ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:33:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.086+0000 7f8cce600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8109170 con 0x7f8cc8108780 2024-04-25T18:33:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.086+0000 7f8cc7400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc8108ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:33:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.086+0000 7f8cc7400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc8108ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35480/0 (socket says 172.21.15.112:35480) 2024-04-25T18:33:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.086+0000 7f8cc7400700 1 -- 172.21.15.112:0/1160439966 learned_addr learned my addr 172.21.15.112:0/1160439966 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:33:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.087+0000 7f8cc7400700 1 -- 172.21.15.112:0/1160439966 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc8109980 con 0x7f8cc8108780 2024-04-25T18:33:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.087+0000 7f8cc7400700 1 --2- 172.21.15.112:0/1160439966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc8108ba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80098a0 tx=0x7f8cb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75280ba15bc232e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:33:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.088+0000 7f8cc6000700 1 -- 172.21.15.112:0/1160439966 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb8004160 con 0x7f8cc8108780 2024-04-25T18:33:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.088+0000 7f8cc6000700 1 -- 172.21.15.112:0/1160439966 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8cb8004300 con 0x7f8cc8108780 2024-04-25T18:33:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.088+0000 7f8cc6000700 1 -- 172.21.15.112:0/1160439966 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb8011af0 con 0x7f8cc8108780 2024-04-25T18:33:54.827 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.088+0000 7f8cce600700 1 -- 172.21.15.112:0/1160439966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 msgr2=0x7f8cc8108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:33:54.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.088+0000 7f8cce600700 1 --2- 172.21.15.112:0/1160439966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc8108ba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80098a0 tx=0x7f8cb8009bb0 comp rx=0 tx=0).stop 2024-04-25T18:33:54.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.089+0000 7f8cce600700 1 -- 172.21.15.112:0/1160439966 shutdown_connections 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.089+0000 7f8cce600700 1 --2- 172.21.15.112:0/1160439966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc8108ba0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.089+0000 7f8cce600700 1 -- 172.21.15.112:0/1160439966 >> 172.21.15.112:0/1160439966 conn(0x7f8cc8103d40 msgr2=0x7f8cc8106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.089+0000 7f8cce600700 1 -- 172.21.15.112:0/1160439966 shutdown_connections 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.089+0000 7f8cce600700 1 -- 172.21.15.112:0/1160439966 wait complete. 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.090+0000 7f8cce600700 1 Processor -- start 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.090+0000 7f8cce600700 1 -- start start 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.090+0000 7f8cce600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.090+0000 7f8cce600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc810cbb0 con 0x7f8cc8108780 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.090+0000 7f8cc7400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.091+0000 7f8cc7400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35496/0 (socket says 172.21.15.112:35496) 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.091+0000 7f8cc7400700 1 -- 172.21.15.112:0/232327990 learned_addr learned my addr 172.21.15.112:0/232327990 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.091+0000 7f8cc7400700 1 -- 172.21.15.112:0/232327990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc819ec80 con 0x7f8cc8108780 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.091+0000 7f8cc7400700 1 --2- 172.21.15.112:0/232327990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc819e740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8011840 tx=0x7f8cb8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:33:54.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.091+0000 7f8cbfe00700 1 -- 172.21.15.112:0/232327990 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb8003af0 con 0x7f8cc8108780 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.092+0000 7f8cbfe00700 1 -- 172.21.15.112:0/232327990 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8cb8020070 con 0x7f8cc8108780 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.092+0000 7f8cbfe00700 1 -- 172.21.15.112:0/232327990 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cb8019630 con 0x7f8cc8108780 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.092+0000 7f8cce600700 1 -- 172.21.15.112:0/232327990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc819ef10 con 0x7f8cc8108780 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.092+0000 7f8cce600700 1 -- 172.21.15.112:0/232327990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc81a1c00 con 0x7f8cc8108780 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.092+0000 7f8cbfe00700 1 -- 172.21.15.112:0/232327990 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8cb8019c30 con 0x7f8cc8108780 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.092+0000 7f8cbfe00700 1 -- 172.21.15.112:0/232327990 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8cb800b720 con 0x7f8cc8108780 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.093+0000 7f8cce600700 1 -- 172.21.15.112:0/232327990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc80640a0 con 0x7f8cc8108780 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.096+0000 7f8cbfe00700 1 -- 172.21.15.112:0/232327990 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8cb80042c0 con 0x7f8cc8108780 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.172+0000 7f8cce600700 1 -- 172.21.15.112:0/232327990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f8cc81a1dd0 con 0x7f8cc8108780 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.176+0000 7f8cbfe00700 1 -- 172.21.15.112:0/232327990 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f8cb801a9c0 con 0x7f8cc8108780 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.178+0000 7f8cce600700 1 -- 172.21.15.112:0/232327990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 msgr2=0x7f8cc819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.178+0000 7f8cce600700 1 --2- 172.21.15.112:0/232327990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc819e740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8011840 tx=0x7f8cb8011b70 comp rx=0 tx=0).stop 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.178+0000 7f8cce600700 1 -- 172.21.15.112:0/232327990 shutdown_connections 2024-04-25T18:33:54.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.178+0000 7f8cce600700 1 --2- 172.21.15.112:0/232327990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cc8108780 0x7f8cc819e740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:33:54.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.178+0000 7f8cce600700 1 -- 172.21.15.112:0/232327990 >> 172.21.15.112:0/232327990 conn(0x7f8cc8103d40 msgr2=0x7f8cc8104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:33:54.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.178+0000 7f8cce600700 1 -- 172.21.15.112:0/232327990 shutdown_connections 2024-04-25T18:33:54.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:54.178+0000 7f8cce600700 1 -- 172.21.15.112:0/232327990 wait complete. 2024-04-25T18:33:54.833 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T18:33:54.866 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:33:54.866 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mgr... 2024-04-25T18:33:54.867 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T18:33:54.867 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T18:33:55.276 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.x 2024-04-25T18:33:55.276 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T18:33:55.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:55 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/232327990' entity='client.admin' 2024-04-25T18:33:55.524 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2fe18a52-0332-11ef-bc93-c7b262605968.target.wants/ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-2fe18a52-0332-11ef-bc93-c7b262605968@.service. 2024-04-25T18:33:55.908 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:55 smithi112 systemd[1]: Starting Ceph mgr.x for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:33:56.190 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:55 smithi112 podman[28088]: 2024-04-25 18:33:55.811891166 +0000 UTC m=+0.035013835 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:33:56.190 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:55 smithi112 podman[28088]: 2024-04-25 18:33:55.959750223 +0000 UTC m=+0.182872918 container create 0472f2a3d65ad6c8c9257de2417ec854a5f73f4697bad68948e80671963f49fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:33:56.190 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:56 smithi112 podman[28088]: 2024-04-25 18:33:56.187971831 +0000 UTC m=+0.411094538 container init 0472f2a3d65ad6c8c9257de2417ec854a5f73f4697bad68948e80671963f49fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T18:33:56.352 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:33:56.352 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-25T18:33:56.367 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:33:56.367 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T18:33:56.367 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-25T18:33:56.367 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T18:33:56.386 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:33:56.386 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-25T18:33:56.407 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:33:56.407 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T18:33:56.407 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-25T18:33:56.408 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T18:33:56.408 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr to start... 2024-04-25T18:33:56.408 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr... 2024-04-25T18:33:56.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:56 smithi112 podman[28088]: 2024-04-25 18:33:56.198363027 +0000 UTC m=+0.421485699 container start 0472f2a3d65ad6c8c9257de2417ec854a5f73f4697bad68948e80671963f49fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T18:33:56.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:56 smithi112 bash[28088]: 0472f2a3d65ad6c8c9257de2417ec854a5f73f4697bad68948e80671963f49fc 2024-04-25T18:33:56.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:56 smithi112 systemd[1]: Started Ceph mgr.x for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:33:56.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:56 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:33:56.429+0000 7f3a535d2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:33:56.839 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:56 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:33:56.693+0000 7f3a535d2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:33:57.148 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:56 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:33:56.837+0000 7f3a535d2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:33:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:33:57 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/152890931' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:33:57.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:57 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:33:57.278+0000 7f3a535d2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:33:57.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:57 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:33:57.391+0000 7f3a535d2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:33:58.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:33:58.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:33:58.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:33:58.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:33:58.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:33:58.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:33:58.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:33:58.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:33:58.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:33:58.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:33:58.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:33:58.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:33:58.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:33:58.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:33:58.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:33:58.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:33:58.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:33:58.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:33:58.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:33:58.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:33:58.350 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:33:58.351 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:33:37.379703+0000", 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.056+0000 7f2eb0800700 1 Processor -- start 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.056+0000 7f2eb0800700 1 -- start start 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.056+0000 7f2eb0800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea8108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.056+0000 7f2eb0800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81090f0 con 0x7f2ea8108700 2024-04-25T18:33:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.057+0000 7f2eade00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea8108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.057+0000 7f2eade00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea8108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35512/0 (socket says 172.21.15.112:35512) 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.057+0000 7f2eade00700 1 -- 172.21.15.112:0/96925803 learned_addr learned my addr 172.21.15.112:0/96925803 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.058+0000 7f2eade00700 1 -- 172.21.15.112:0/96925803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea8109900 con 0x7f2ea8108700 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.058+0000 7f2eade00700 1 --2- 172.21.15.112:0/96925803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea8108b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0098a0 tx=0x7f2e9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ec9a7917723da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.058+0000 7f2eaca00700 1 -- 172.21.15.112:0/96925803 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c004160 con 0x7f2ea8108700 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.059+0000 7f2eaca00700 1 -- 172.21.15.112:0/96925803 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2e9c004300 con 0x7f2ea8108700 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.059+0000 7f2eaca00700 1 -- 172.21.15.112:0/96925803 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c011af0 con 0x7f2ea8108700 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.059+0000 7f2eb0800700 1 -- 172.21.15.112:0/96925803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 msgr2=0x7f2ea8108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.059+0000 7f2eb0800700 1 --2- 172.21.15.112:0/96925803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea8108b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0098a0 tx=0x7f2e9c009bb0 comp rx=0 tx=0).stop 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.060+0000 7f2eb0800700 1 -- 172.21.15.112:0/96925803 shutdown_connections 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.060+0000 7f2eb0800700 1 --2- 172.21.15.112:0/96925803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea8108b20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.060+0000 7f2eb0800700 1 -- 172.21.15.112:0/96925803 >> 172.21.15.112:0/96925803 conn(0x7f2ea8103ce0 msgr2=0x7f2ea8106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.060+0000 7f2eb0800700 1 -- 172.21.15.112:0/96925803 shutdown_connections 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.060+0000 7f2eb0800700 1 -- 172.21.15.112:0/96925803 wait complete. 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.061+0000 7f2eb0800700 1 Processor -- start 2024-04-25T18:33:58.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.062+0000 7f2eb0800700 1 -- start start 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.062+0000 7f2eb0800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.062+0000 7f2eb0800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea810cb30 con 0x7f2ea8108700 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.062+0000 7f2eade00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.062+0000 7f2eade00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35520/0 (socket says 172.21.15.112:35520) 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.062+0000 7f2eade00700 1 -- 172.21.15.112:0/152890931 learned_addr learned my addr 172.21.15.112:0/152890931 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.063+0000 7f2eade00700 1 -- 172.21.15.112:0/152890931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea819ca90 con 0x7f2ea8108700 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.063+0000 7f2eade00700 1 --2- 172.21.15.112:0/152890931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea819c550 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0117b0 tx=0x7f2e9c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.063+0000 7f2ea2a00700 1 -- 172.21.15.112:0/152890931 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c003af0 con 0x7f2ea8108700 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.064+0000 7f2ea2a00700 1 -- 172.21.15.112:0/152890931 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2e9c020070 con 0x7f2ea8108700 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.064+0000 7f2eb0800700 1 -- 172.21.15.112:0/152890931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea819cd20 con 0x7f2ea8108700 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.064+0000 7f2ea2a00700 1 -- 172.21.15.112:0/152890931 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c019630 con 0x7f2ea8108700 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.064+0000 7f2eb0800700 1 -- 172.21.15.112:0/152890931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea819fa10 con 0x7f2ea8108700 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.064+0000 7f2ea2a00700 1 -- 172.21.15.112:0/152890931 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2e9c019c80 con 0x7f2ea8108700 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.064+0000 7f2ea2a00700 1 -- 172.21.15.112:0/152890931 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2e9c00b720 con 0x7f2ea8108700 2024-04-25T18:33:58.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.065+0000 7f2eb0800700 1 -- 172.21.15.112:0/152890931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea80640a0 con 0x7f2ea8108700 2024-04-25T18:33:58.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.068+0000 7f2ea2a00700 1 -- 172.21.15.112:0/152890931 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2e9c0042a0 con 0x7f2ea8108700 2024-04-25T18:33:58.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.146+0000 7f2eb0800700 1 -- 172.21.15.112:0/152890931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2ea819ff60 con 0x7f2ea8108700 2024-04-25T18:33:58.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.147+0000 7f2ea2a00700 1 -- 172.21.15.112:0/152890931 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2e9c01a9c0 con 0x7f2ea8108700 2024-04-25T18:33:58.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.149+0000 7f2eb0800700 1 -- 172.21.15.112:0/152890931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 msgr2=0x7f2ea819c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:33:58.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.149+0000 7f2eb0800700 1 --2- 172.21.15.112:0/152890931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea819c550 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0117b0 tx=0x7f2e9c011b70 comp rx=0 tx=0).stop 2024-04-25T18:33:58.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.149+0000 7f2eb0800700 1 -- 172.21.15.112:0/152890931 shutdown_connections 2024-04-25T18:33:58.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.149+0000 7f2eb0800700 1 --2- 172.21.15.112:0/152890931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea8108700 0x7f2ea819c550 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:33:58.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.149+0000 7f2eb0800700 1 -- 172.21.15.112:0/152890931 >> 172.21.15.112:0/152890931 conn(0x7f2ea8103ce0 msgr2=0x7f2ea81047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:33:58.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.150+0000 7f2eb0800700 1 -- 172.21.15.112:0/152890931 shutdown_connections 2024-04-25T18:33:58.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:33:57.150+0000 7f2eb0800700 1 -- 172.21.15.112:0/152890931 wait complete. 2024-04-25T18:33:58.358 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (1/15)... 2024-04-25T18:33:59.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:58 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:33:58.960+0000 7f3a535d2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:33:59.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:59 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:33:59.182+0000 7f3a535d2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:33:59.723 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:59 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:33:59.389+0000 7f3a535d2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:34:00.128 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:59 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:33:59.759+0000 7f3a535d2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:34:00.128 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:33:59 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:33:59.878+0000 7f3a535d2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:34:00.387 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:00 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:00.126+0000 7f3a535d2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:34:00.659 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:00 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:00.385+0000 7f3a535d2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:34:01.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:01 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3391699788' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:34:01.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:01.190+0000 7f3a535d2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:34:01.851 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:34:01.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:34:01.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:34:01.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:34:01.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:01.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:33:37.379703+0000", 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.955+0000 7f9285e00700 1 Processor -- start 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.955+0000 7f9285e00700 1 -- start start 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.956+0000 7f9285e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f92801073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.956+0000 7f9285e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9280063cc0 con 0x7f9280106fa0 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.956+0000 7f927ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f92801073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.957+0000 7f927ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f92801073c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51816/0 (socket says 172.21.15.112:51816) 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.957+0000 7f927ea00700 1 -- 172.21.15.112:0/2297386429 learned_addr learned my addr 172.21.15.112:0/2297386429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:01.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.957+0000 7f927ea00700 1 -- 172.21.15.112:0/2297386429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9280107900 con 0x7f9280106fa0 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.957+0000 7f927ea00700 1 --2- 172.21.15.112:0/2297386429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f92801073c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92700098a0 tx=0x7f9270009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9457b181c4b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.958+0000 7f927e000700 1 -- 172.21.15.112:0/2297386429 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9270004160 con 0x7f9280106fa0 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.958+0000 7f927e000700 1 -- 172.21.15.112:0/2297386429 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9270004300 con 0x7f9280106fa0 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.958+0000 7f927e000700 1 -- 172.21.15.112:0/2297386429 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9270011af0 con 0x7f9280106fa0 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.959+0000 7f9285e00700 1 -- 172.21.15.112:0/2297386429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 msgr2=0x7f92801073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.959+0000 7f9285e00700 1 --2- 172.21.15.112:0/2297386429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f92801073c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92700098a0 tx=0x7f9270009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.959+0000 7f9285e00700 1 -- 172.21.15.112:0/2297386429 shutdown_connections 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.959+0000 7f9285e00700 1 --2- 172.21.15.112:0/2297386429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f92801073c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.959+0000 7f9285e00700 1 -- 172.21.15.112:0/2297386429 >> 172.21.15.112:0/2297386429 conn(0x7f9280102ba0 msgr2=0x7f9280104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.959+0000 7f9285e00700 1 -- 172.21.15.112:0/2297386429 shutdown_connections 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.959+0000 7f9285e00700 1 -- 172.21.15.112:0/2297386429 wait complete. 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.960+0000 7f9285e00700 1 Processor -- start 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.960+0000 7f9285e00700 1 -- start start 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.960+0000 7f9285e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f928019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.960+0000 7f9285e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9280108850 con 0x7f9280106fa0 2024-04-25T18:34:01.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.961+0000 7f927ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f928019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.961+0000 7f927ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f928019e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51832/0 (socket says 172.21.15.112:51832) 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.961+0000 7f927ea00700 1 -- 172.21.15.112:0/3391699788 learned_addr learned my addr 172.21.15.112:0/3391699788 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.961+0000 7f927ea00700 1 -- 172.21.15.112:0/3391699788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928019ec10 con 0x7f9280106fa0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.961+0000 7f927ea00700 1 --2- 172.21.15.112:0/3391699788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f928019e6d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9270011ac0 tx=0x7f9270011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.962+0000 7f9284a00700 1 -- 172.21.15.112:0/3391699788 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9270003b20 con 0x7f9280106fa0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.962+0000 7f9284a00700 1 -- 172.21.15.112:0/3391699788 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f927001a8c0 con 0x7f9280106fa0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.962+0000 7f9284a00700 1 -- 172.21.15.112:0/3391699788 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92700248d0 con 0x7f9280106fa0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.962+0000 7f9285e00700 1 -- 172.21.15.112:0/3391699788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928019eea0 con 0x7f9280106fa0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.962+0000 7f9285e00700 1 -- 172.21.15.112:0/3391699788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92801a1b90 con 0x7f9280106fa0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.962+0000 7f9284a00700 1 -- 172.21.15.112:0/3391699788 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9270022020 con 0x7f9280106fa0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.962+0000 7f9284a00700 1 -- 172.21.15.112:0/3391699788 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f92700193c0 con 0x7f9280106fa0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.963+0000 7f9285e00700 1 -- 172.21.15.112:0/3391699788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92800640a0 con 0x7f9280106fa0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:00.966+0000 7f9284a00700 1 -- 172.21.15.112:0/3391699788 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9270042e50 con 0x7f9280106fa0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:01.045+0000 7f9285e00700 1 -- 172.21.15.112:0/3391699788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f928002e190 con 0x7f9280106fa0 2024-04-25T18:34:01.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:01.046+0000 7f9284a00700 1 -- 172.21.15.112:0/3391699788 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9270031020 con 0x7f9280106fa0 2024-04-25T18:34:01.866 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:01.047+0000 7f9285e00700 1 -- 172.21.15.112:0/3391699788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 msgr2=0x7f928019e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:01.866 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:01.047+0000 7f9285e00700 1 --2- 172.21.15.112:0/3391699788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f928019e6d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9270011ac0 tx=0x7f9270011ba0 comp rx=0 tx=0).stop 2024-04-25T18:34:01.866 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:01.048+0000 7f9285e00700 1 -- 172.21.15.112:0/3391699788 shutdown_connections 2024-04-25T18:34:01.866 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:01.048+0000 7f9285e00700 1 --2- 172.21.15.112:0/3391699788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9280106fa0 0x7f928019e6d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:01.866 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:01.048+0000 7f9285e00700 1 -- 172.21.15.112:0/3391699788 >> 172.21.15.112:0/3391699788 conn(0x7f9280102ba0 msgr2=0x7f9280195360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:01.866 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:01.048+0000 7f9285e00700 1 -- 172.21.15.112:0/3391699788 shutdown_connections 2024-04-25T18:34:01.866 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:01.048+0000 7f9285e00700 1 -- 172.21.15.112:0/3391699788 wait complete. 2024-04-25T18:34:01.866 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (2/15)... 2024-04-25T18:34:02.148 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:01.760+0000 7f3a535d2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:34:02.148 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:01.884+0000 7f3a535d2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:34:02.148 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:02.146+0000 7f3a535d2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:34:03.220 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:02.778+0000 7f3a535d2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:34:03.221 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:02.898+0000 7f3a535d2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:34:03.221 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:03 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:03.218+0000 7f3a535d2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:34:03.474 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:03 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:03.394+0000 7f3a535d2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:34:03.973 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:03 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:03.518+0000 7f3a535d2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:34:03.973 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:03 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:03.667+0000 7f3a535d2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:34:04.323 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:04 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:04.036+0000 7f3a535d2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:34:04.958 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:04 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:04.826+0000 7f3a535d2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:34:04.959 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:04 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1178996929' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:34:05.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:05.132+0000 7f3a535d2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:34:05.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:34:05.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-25T18:34:05.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:05.393 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:34:05.394 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:05.395 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:33:37.379703+0000", 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.504+0000 7fb7a7600700 1 Processor -- start 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.504+0000 7fb7a7600700 1 -- start start 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.504+0000 7fb7a7600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a007c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.505+0000 7fb7a7600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a007c7d0 con 0x7fb7a007dda0 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.505+0000 7fb7a4c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a007c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.505+0000 7fb7a4c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a007c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51846/0 (socket says 172.21.15.112:51846) 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.505+0000 7fb7a4c00700 1 -- 172.21.15.112:0/1645731445 learned_addr learned my addr 172.21.15.112:0/1645731445 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.505+0000 7fb7a4c00700 1 -- 172.21.15.112:0/1645731445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a007c950 con 0x7fb7a007dda0 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.506+0000 7fb7a4c00700 1 --2- 172.21.15.112:0/1645731445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a007c200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb7940098a0 tx=0x7fb794009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dfd5a21f0961ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.506+0000 7fb79f400700 1 -- 172.21.15.112:0/1645731445 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb794004160 con 0x7fb7a007dda0 2024-04-25T18:34:05.396 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.506+0000 7fb79f400700 1 -- 172.21.15.112:0/1645731445 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb794004300 con 0x7fb7a007dda0 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.506+0000 7fb79f400700 1 -- 172.21.15.112:0/1645731445 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb794011af0 con 0x7fb7a007dda0 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.507+0000 7fb7a7600700 1 -- 172.21.15.112:0/1645731445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 msgr2=0x7fb7a007c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.507+0000 7fb7a7600700 1 --2- 172.21.15.112:0/1645731445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a007c200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb7940098a0 tx=0x7fb794009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.507+0000 7fb7a7600700 1 -- 172.21.15.112:0/1645731445 shutdown_connections 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.507+0000 7fb7a7600700 1 --2- 172.21.15.112:0/1645731445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a007c200 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.507+0000 7fb7a7600700 1 -- 172.21.15.112:0/1645731445 >> 172.21.15.112:0/1645731445 conn(0x7fb7a0103c40 msgr2=0x7fb7a01060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.507+0000 7fb7a7600700 1 -- 172.21.15.112:0/1645731445 shutdown_connections 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.507+0000 7fb7a7600700 1 -- 172.21.15.112:0/1645731445 wait complete. 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.508+0000 7fb7a7600700 1 Processor -- start 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.508+0000 7fb7a7600700 1 -- start start 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.509+0000 7fb7a7600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.509+0000 7fb7a7600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a010eba0 con 0x7fb7a007dda0 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.509+0000 7fb7a4c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:05.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.509+0000 7fb7a4c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a019e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51852/0 (socket says 172.21.15.112:51852) 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.509+0000 7fb7a4c00700 1 -- 172.21.15.112:0/1178996929 learned_addr learned my addr 172.21.15.112:0/1178996929 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.509+0000 7fb7a4c00700 1 -- 172.21.15.112:0/1178996929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a019ec70 con 0x7fb7a007dda0 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.510+0000 7fb7a4c00700 1 --2- 172.21.15.112:0/1178996929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a019e730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb794000c00 tx=0x7fb794011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.510+0000 7fb79d600700 1 -- 172.21.15.112:0/1178996929 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb794003b20 con 0x7fb7a007dda0 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.510+0000 7fb79d600700 1 -- 172.21.15.112:0/1178996929 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb79401a8c0 con 0x7fb7a007dda0 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.510+0000 7fb7a7600700 1 -- 172.21.15.112:0/1178996929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a019ef00 con 0x7fb7a007dda0 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.510+0000 7fb79d600700 1 -- 172.21.15.112:0/1178996929 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb794024940 con 0x7fb7a007dda0 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.510+0000 7fb7a7600700 1 -- 172.21.15.112:0/1178996929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a01a1bf0 con 0x7fb7a007dda0 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.511+0000 7fb79d600700 1 -- 172.21.15.112:0/1178996929 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb794022020 con 0x7fb7a007dda0 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.511+0000 7fb79d600700 1 -- 172.21.15.112:0/1178996929 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb79402dd90 con 0x7fb7a007dda0 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.511+0000 7fb7a7600700 1 -- 172.21.15.112:0/1178996929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a00640a0 con 0x7fb7a007dda0 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.514+0000 7fb79d600700 1 -- 172.21.15.112:0/1178996929 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb794042e50 con 0x7fb7a007dda0 2024-04-25T18:34:05.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.593+0000 7fb7a7600700 1 -- 172.21.15.112:0/1178996929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb7a01a2130 con 0x7fb7a007dda0 2024-04-25T18:34:05.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.593+0000 7fb79d600700 1 -- 172.21.15.112:0/1178996929 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fb794031020 con 0x7fb7a007dda0 2024-04-25T18:34:05.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.595+0000 7fb7a7600700 1 -- 172.21.15.112:0/1178996929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 msgr2=0x7fb7a019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:05.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.595+0000 7fb7a7600700 1 --2- 172.21.15.112:0/1178996929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a019e730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb794000c00 tx=0x7fb794011b70 comp rx=0 tx=0).stop 2024-04-25T18:34:05.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.595+0000 7fb7a7600700 1 -- 172.21.15.112:0/1178996929 shutdown_connections 2024-04-25T18:34:05.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.595+0000 7fb7a7600700 1 --2- 172.21.15.112:0/1178996929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7a007dda0 0x7fb7a019e730 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:05.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.595+0000 7fb7a7600700 1 -- 172.21.15.112:0/1178996929 >> 172.21.15.112:0/1178996929 conn(0x7fb7a0103c40 msgr2=0x7fb7a01047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:05.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.595+0000 7fb7a7600700 1 -- 172.21.15.112:0/1178996929 shutdown_connections 2024-04-25T18:34:05.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:04.595+0000 7fb7a7600700 1 -- 172.21.15.112:0/1178996929 wait complete. 2024-04-25T18:34:05.399 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (3/15)... 2024-04-25T18:34:05.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: Activating manager daemon x 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: mgrmap e2: x(active, starting, since 0.0129847s) 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: from='mgr.14100 172.21.15.112:0/593095893' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: from='mgr.14100 172.21.15.112:0/593095893' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: from='mgr.14100 172.21.15.112:0/593095893' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: from='mgr.14100 172.21.15.112:0/593095893' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: from='mgr.14100 172.21.15.112:0/593095893' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: Manager daemon x is now available 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: from='mgr.14100 172.21.15.112:0/593095893' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: from='mgr.14100 172.21.15.112:0/593095893' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: from='mgr.14100 172.21.15.112:0/593095893' entity='mgr.x' 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: from='mgr.14100 172.21.15.112:0/593095893' entity='mgr.x' 2024-04-25T18:34:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:05 smithi112 ceph-mon[27864]: from='mgr.14100 172.21.15.112:0/593095893' entity='mgr.x' 2024-04-25T18:34:07.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:07 smithi112 ceph-mon[27864]: mgrmap e3: x(active, since 1.02224s) 2024-04-25T18:34:08.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:08 smithi112 ceph-mon[27864]: mgrmap e4: x(active, since 2s) 2024-04-25T18:34:09.113 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:34:09.115 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:34:09.115 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:34:09.115 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:34:09.115 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:34:09.115 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:34:09.115 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:34:09.115 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:09.115 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:34:09.115 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:34:09.116 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:34:09.116 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:34:09.116 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:34:09.116 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:34:09.116 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:34:09.116 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-04-25T18:34:09.116 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:34:09.116 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:09.116 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:34:09.116 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:34:09.117 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:09.118 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:34:09.118 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:09.118 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:34:09.118 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:34:09.118 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:34:09.118 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:34:09.118 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:34:09.118 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:34:09.118 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:09.126 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:09.127 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:33:37.379703+0000", 2024-04-25T18:34:09.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.043+0000 7f96fbc00700 1 Processor -- start 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.044+0000 7f96fbc00700 1 -- start start 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.044+0000 7f96fbc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f4082b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.044+0000 7f96fbc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96f4083110 con 0x7f96f4082720 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.044+0000 7f96f9200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f4082b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.045+0000 7f96f9200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f4082b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51948/0 (socket says 172.21.15.112:51948) 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.045+0000 7f96f9200700 1 -- 172.21.15.112:0/2331398386 learned_addr learned my addr 172.21.15.112:0/2331398386 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.045+0000 7f96f9200700 1 -- 172.21.15.112:0/2331398386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f410fef0 con 0x7f96f4082720 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.045+0000 7f96f9200700 1 --2- 172.21.15.112:0/2331398386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f4082b40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96e80098a0 tx=0x7f96e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961d86880062e0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:09.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.045+0000 7f96f3400700 1 -- 172.21.15.112:0/2331398386 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e8004160 con 0x7f96f4082720 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.046+0000 7f96f3400700 1 -- 172.21.15.112:0/2331398386 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f96e8004300 con 0x7f96f4082720 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.046+0000 7f96f3400700 1 -- 172.21.15.112:0/2331398386 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e8011af0 con 0x7f96f4082720 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.046+0000 7f96fbc00700 1 -- 172.21.15.112:0/2331398386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 msgr2=0x7f96f4082b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.046+0000 7f96fbc00700 1 --2- 172.21.15.112:0/2331398386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f4082b40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96e80098a0 tx=0x7f96e8009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.046+0000 7f96fbc00700 1 -- 172.21.15.112:0/2331398386 shutdown_connections 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.046+0000 7f96fbc00700 1 --2- 172.21.15.112:0/2331398386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f4082b40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.046+0000 7f96fbc00700 1 -- 172.21.15.112:0/2331398386 >> 172.21.15.112:0/2331398386 conn(0x7f96f407dce0 msgr2=0x7f96f4080100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.046+0000 7f96fbc00700 1 -- 172.21.15.112:0/2331398386 shutdown_connections 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.046+0000 7f96fbc00700 1 -- 172.21.15.112:0/2331398386 wait complete. 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.047+0000 7f96fbc00700 1 Processor -- start 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.048+0000 7f96fbc00700 1 -- start start 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.048+0000 7f96fbc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f41a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.048+0000 7f96fbc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96f4113120 con 0x7f96f4082720 2024-04-25T18:34:09.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.048+0000 7f96f9200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f41a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.048+0000 7f96f9200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f41a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51952/0 (socket says 172.21.15.112:51952) 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.048+0000 7f96f9200700 1 -- 172.21.15.112:0/2377908224 learned_addr learned my addr 172.21.15.112:0/2377908224 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.049+0000 7f96f9200700 1 -- 172.21.15.112:0/2377908224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f41a30a0 con 0x7f96f4082720 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.049+0000 7f96f9200700 1 --2- 172.21.15.112:0/2377908224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f41a2b60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f96e80117b0 tx=0x7f96e8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.049+0000 7f96f1600700 1 -- 172.21.15.112:0/2377908224 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e8003af0 con 0x7f96f4082720 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.049+0000 7f96f1600700 1 -- 172.21.15.112:0/2377908224 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f96e8020070 con 0x7f96f4082720 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.049+0000 7f96fbc00700 1 -- 172.21.15.112:0/2377908224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f41a3330 con 0x7f96f4082720 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.049+0000 7f96f1600700 1 -- 172.21.15.112:0/2377908224 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e8019630 con 0x7f96f4082720 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.049+0000 7f96fbc00700 1 -- 172.21.15.112:0/2377908224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f41a6020 con 0x7f96f4082720 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.051+0000 7f96f1600700 1 -- 172.21.15.112:0/2377908224 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f96e801a9c0 con 0x7f96f4082720 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.051+0000 7f96fbc00700 1 -- 172.21.15.112:0/2377908224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f40640a0 con 0x7f96f4082720 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.051+0000 7f96f1600700 1 --2- 172.21.15.112:0/2377908224 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f96dc0395b0 0x7f96dc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.051+0000 7f96f1600700 1 -- 172.21.15.112:0/2377908224 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f96e804da90 con 0x7f96f4082720 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.051+0000 7f96f3e00700 1 --2- 172.21.15.112:0/2377908224 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f96dc0395b0 0x7f96dc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:09.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.052+0000 7f96f3e00700 1 --2- 172.21.15.112:0/2377908224 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f96dc0395b0 0x7f96dc03ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f96e0006fd0 tx=0x7f96e0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.057+0000 7f96f1600700 1 -- 172.21.15.112:0/2377908224 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96e800e260 con 0x7f96f4082720 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.371+0000 7f96fbc00700 1 -- 172.21.15.112:0/2377908224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f96f41a61f0 con 0x7f96f4082720 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.372+0000 7f96f1600700 1 -- 172.21.15.112:0/2377908224 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f96f41a61f0 con 0x7f96f4082720 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.375+0000 7f96fbc00700 1 -- 172.21.15.112:0/2377908224 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f96dc0395b0 msgr2=0x7f96dc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.375+0000 7f96fbc00700 1 --2- 172.21.15.112:0/2377908224 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f96dc0395b0 0x7f96dc03ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f96e0006fd0 tx=0x7f96e0006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.375+0000 7f96fbc00700 1 -- 172.21.15.112:0/2377908224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 msgr2=0x7f96f41a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.375+0000 7f96fbc00700 1 --2- 172.21.15.112:0/2377908224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f41a2b60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f96e80117b0 tx=0x7f96e8011b70 comp rx=0 tx=0).stop 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.375+0000 7f96fbc00700 1 -- 172.21.15.112:0/2377908224 shutdown_connections 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.375+0000 7f96fbc00700 1 --2- 172.21.15.112:0/2377908224 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f96dc0395b0 0x7f96dc03ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.375+0000 7f96fbc00700 1 --2- 172.21.15.112:0/2377908224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96f4082720 0x7f96f41a2b60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.375+0000 7f96fbc00700 1 -- 172.21.15.112:0/2377908224 >> 172.21.15.112:0/2377908224 conn(0x7f96f407dce0 msgr2=0x7f96f407e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.376+0000 7f96fbc00700 1 -- 172.21.15.112:0/2377908224 shutdown_connections 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:08.376+0000 7f96fbc00700 1 -- 172.21.15.112:0/2377908224 wait complete. 2024-04-25T18:34:09.132 INFO:teuthology.orchestra.run.smithi112.stdout:mgr is available 2024-04-25T18:34:09.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:09 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2377908224' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:34:10.456 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:10 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1009138511' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T18:34:10.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:34:10.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T18:34:10.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:34:10.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T18:34:10.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T18:34:10.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T18:34:10.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T18:34:10.824 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T18:34:10.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:34:10.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T18:34:10.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T18:34:10.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T18:34:10.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T18:34:10.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T18:34:10.825 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.795+0000 7ff308000700 1 Processor -- start 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.795+0000 7ff308000700 1 -- start start 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.795+0000 7ff308000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff300082b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.795+0000 7ff308000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff300083110 con 0x7ff300082720 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.796+0000 7ff305600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff300082b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.796+0000 7ff305600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff300082b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51956/0 (socket says 172.21.15.112:51956) 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.796+0000 7ff305600700 1 -- 172.21.15.112:0/2022675442 learned_addr learned my addr 172.21.15.112:0/2022675442 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.796+0000 7ff305600700 1 -- 172.21.15.112:0/2022675442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30010fef0 con 0x7ff300082720 2024-04-25T18:34:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.797+0000 7ff305600700 1 --2- 172.21.15.112:0/2022675442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff300082b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40098a0 tx=0x7ff2f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00121e5b4afa2b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.797+0000 7ff2ffe00700 1 -- 172.21.15.112:0/2022675442 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f4004160 con 0x7ff300082720 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.797+0000 7ff2ffe00700 1 -- 172.21.15.112:0/2022675442 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff2f4004300 con 0x7ff300082720 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.797+0000 7ff2ffe00700 1 -- 172.21.15.112:0/2022675442 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f4011af0 con 0x7ff300082720 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.798+0000 7ff308000700 1 -- 172.21.15.112:0/2022675442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 msgr2=0x7ff300082b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.798+0000 7ff308000700 1 --2- 172.21.15.112:0/2022675442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff300082b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40098a0 tx=0x7ff2f4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.798+0000 7ff308000700 1 -- 172.21.15.112:0/2022675442 shutdown_connections 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.798+0000 7ff308000700 1 --2- 172.21.15.112:0/2022675442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff300082b40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.798+0000 7ff308000700 1 -- 172.21.15.112:0/2022675442 >> 172.21.15.112:0/2022675442 conn(0x7ff30007dce0 msgr2=0x7ff300080100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.798+0000 7ff308000700 1 -- 172.21.15.112:0/2022675442 shutdown_connections 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.798+0000 7ff308000700 1 -- 172.21.15.112:0/2022675442 wait complete. 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.799+0000 7ff308000700 1 Processor -- start 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.799+0000 7ff308000700 1 -- start start 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.800+0000 7ff308000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff3001a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:10.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.800+0000 7ff308000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff300113120 con 0x7ff300082720 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.800+0000 7ff305600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff3001a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.800+0000 7ff305600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff3001a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51964/0 (socket says 172.21.15.112:51964) 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.800+0000 7ff305600700 1 -- 172.21.15.112:0/1009138511 learned_addr learned my addr 172.21.15.112:0/1009138511 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.800+0000 7ff305600700 1 -- 172.21.15.112:0/1009138511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3001a30a0 con 0x7ff300082720 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.801+0000 7ff305600700 1 --2- 172.21.15.112:0/1009138511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff3001a2b60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40117b0 tx=0x7ff2f4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.801+0000 7ff2fe000700 1 -- 172.21.15.112:0/1009138511 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f4003af0 con 0x7ff300082720 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.801+0000 7ff2fe000700 1 -- 172.21.15.112:0/1009138511 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff2f4020070 con 0x7ff300082720 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.801+0000 7ff308000700 1 -- 172.21.15.112:0/1009138511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3001a3330 con 0x7ff300082720 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.801+0000 7ff2fe000700 1 -- 172.21.15.112:0/1009138511 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f4019630 con 0x7ff300082720 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.801+0000 7ff308000700 1 -- 172.21.15.112:0/1009138511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3001a6020 con 0x7ff300082720 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.802+0000 7ff2fe000700 1 -- 172.21.15.112:0/1009138511 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff2f401a9c0 con 0x7ff300082720 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.802+0000 7ff308000700 1 -- 172.21.15.112:0/1009138511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3000640a0 con 0x7ff300082720 2024-04-25T18:34:10.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.803+0000 7ff2fe000700 1 --2- 172.21.15.112:0/1009138511 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7ff2e80395b0 0x7ff2e803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.803+0000 7ff2fe000700 1 -- 172.21.15.112:0/1009138511 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff2f404da90 con 0x7ff300082720 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.803+0000 7ff304c00700 1 --2- 172.21.15.112:0/1009138511 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7ff2e80395b0 0x7ff2e803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.804+0000 7ff304c00700 1 --2- 172.21.15.112:0/1009138511 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7ff2e80395b0 0x7ff2e803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec006fd0 tx=0x7ff2ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:09.808+0000 7ff2fe000700 1 -- 172.21.15.112:0/1009138511 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2f400e260 con 0x7ff300082720 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.031+0000 7ff308000700 1 -- 172.21.15.112:0/1009138511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff3001a61f0 con 0x7ff300082720 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.032+0000 7ff2fe000700 1 -- 172.21.15.112:0/1009138511 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7ff3001a61f0 con 0x7ff300082720 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.036+0000 7ff308000700 1 -- 172.21.15.112:0/1009138511 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7ff2e80395b0 msgr2=0x7ff2e803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.036+0000 7ff308000700 1 --2- 172.21.15.112:0/1009138511 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7ff2e80395b0 0x7ff2e803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec006fd0 tx=0x7ff2ec006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.036+0000 7ff308000700 1 -- 172.21.15.112:0/1009138511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 msgr2=0x7ff3001a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.036+0000 7ff308000700 1 --2- 172.21.15.112:0/1009138511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff3001a2b60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40117b0 tx=0x7ff2f4011b70 comp rx=0 tx=0).stop 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.036+0000 7ff308000700 1 -- 172.21.15.112:0/1009138511 shutdown_connections 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.036+0000 7ff308000700 1 --2- 172.21.15.112:0/1009138511 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7ff2e80395b0 0x7ff2e803ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.036+0000 7ff308000700 1 --2- 172.21.15.112:0/1009138511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff300082720 0x7ff3001a2b60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:10.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.036+0000 7ff308000700 1 -- 172.21.15.112:0/1009138511 >> 172.21.15.112:0/1009138511 conn(0x7ff30007dce0 msgr2=0x7ff30007e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:10.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.036+0000 7ff308000700 1 -- 172.21.15.112:0/1009138511 shutdown_connections 2024-04-25T18:34:10.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:10.036+0000 7ff308000700 1 -- 172.21.15.112:0/1009138511 wait complete. 2024-04-25T18:34:10.836 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling cephadm module... 2024-04-25T18:34:12.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:11 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3849365973' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T18:34:12.095 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:11 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: ignoring --setuser ceph since I am not root 2024-04-25T18:34:12.095 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:11 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: ignoring --setgroup ceph since I am not root 2024-04-25T18:34:12.095 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:12 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:12.094+0000 7fb21f28c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:34:12.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:12 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:12.343+0000 7fb21f28c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:34:12.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.443+0000 7f087c200700 1 Processor -- start 2024-04-25T18:34:12.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.443+0000 7f087c200700 1 -- start start 2024-04-25T18:34:12.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.443+0000 7f087c200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f0874108b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:12.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.443+0000 7f087c200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0874109100 con 0x7f0874108710 2024-04-25T18:34:12.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.444+0000 7f0879800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f0874108b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:12.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.444+0000 7f0879800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f0874108b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46270/0 (socket says 172.21.15.112:46270) 2024-04-25T18:34:12.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.444+0000 7f0879800700 1 -- 172.21.15.112:0/2777099841 learned_addr learned my addr 172.21.15.112:0/2777099841 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:12.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.444+0000 7f0879800700 1 -- 172.21.15.112:0/2777099841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0874109910 con 0x7f0874108710 2024-04-25T18:34:12.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.444+0000 7f0879800700 1 --2- 172.21.15.112:0/2777099841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f0874108b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f08680098a0 tx=0x7f0868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb16a8848d06bef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:12.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.445+0000 7f0873e00700 1 -- 172.21.15.112:0/2777099841 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0868004160 con 0x7f0874108710 2024-04-25T18:34:12.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.445+0000 7f0873e00700 1 -- 172.21.15.112:0/2777099841 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0868004300 con 0x7f0874108710 2024-04-25T18:34:12.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.445+0000 7f0873e00700 1 -- 172.21.15.112:0/2777099841 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0868011af0 con 0x7f0874108710 2024-04-25T18:34:12.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.445+0000 7f087c200700 1 -- 172.21.15.112:0/2777099841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 msgr2=0x7f0874108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:12.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.445+0000 7f087c200700 1 --2- 172.21.15.112:0/2777099841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f0874108b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f08680098a0 tx=0x7f0868009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:12.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.446+0000 7f087c200700 1 -- 172.21.15.112:0/2777099841 shutdown_connections 2024-04-25T18:34:12.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.446+0000 7f087c200700 1 --2- 172.21.15.112:0/2777099841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f0874108b30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:12.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.446+0000 7f087c200700 1 -- 172.21.15.112:0/2777099841 >> 172.21.15.112:0/2777099841 conn(0x7f0874103cf0 msgr2=0x7f0874106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:12.598 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.446+0000 7f087c200700 1 -- 172.21.15.112:0/2777099841 shutdown_connections 2024-04-25T18:34:12.598 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.446+0000 7f087c200700 1 -- 172.21.15.112:0/2777099841 wait complete. 2024-04-25T18:34:12.598 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.447+0000 7f087c200700 1 Processor -- start 2024-04-25T18:34:12.598 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.448+0000 7f087c200700 1 -- start start 2024-04-25T18:34:12.598 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.448+0000 7f087c200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f087419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:12.598 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.448+0000 7f087c200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f087410cb40 con 0x7f0874108710 2024-04-25T18:34:12.598 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.448+0000 7f0879800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f087419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:12.598 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.448+0000 7f0879800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f087419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46280/0 (socket says 172.21.15.112:46280) 2024-04-25T18:34:12.598 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.448+0000 7f0879800700 1 -- 172.21.15.112:0/3849365973 learned_addr learned my addr 172.21.15.112:0/3849365973 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:12.598 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.449+0000 7f0879800700 1 -- 172.21.15.112:0/3849365973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087419cac0 con 0x7f0874108710 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.449+0000 7f0879800700 1 --2- 172.21.15.112:0/3849365973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f087419c580 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f08680117b0 tx=0x7f0868011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.449+0000 7f0872000700 1 -- 172.21.15.112:0/3849365973 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0868003af0 con 0x7f0874108710 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.449+0000 7f0872000700 1 -- 172.21.15.112:0/3849365973 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0868020070 con 0x7f0874108710 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.449+0000 7f0872000700 1 -- 172.21.15.112:0/3849365973 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0868019630 con 0x7f0874108710 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.449+0000 7f087c200700 1 -- 172.21.15.112:0/3849365973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087419cd50 con 0x7f0874108710 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.450+0000 7f087c200700 1 -- 172.21.15.112:0/3849365973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087419fa40 con 0x7f0874108710 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.451+0000 7f0872000700 1 -- 172.21.15.112:0/3849365973 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f086801a9c0 con 0x7f0874108710 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.451+0000 7f087c200700 1 -- 172.21.15.112:0/3849365973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08740640a0 con 0x7f0874108710 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.451+0000 7f0872000700 1 --2- 172.21.15.112:0/3849365973 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f085c0395b0 0x7f085c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.451+0000 7f0872000700 1 -- 172.21.15.112:0/3849365973 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f086804dae0 con 0x7f0874108710 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.452+0000 7f0878e00700 1 --2- 172.21.15.112:0/3849365973 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f085c0395b0 0x7f085c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:12.599 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.452+0000 7f0878e00700 1 --2- 172.21.15.112:0/3849365973 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f085c0395b0 0x7f085c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0860006fd0 tx=0x7f0860006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.457+0000 7f0872000700 1 -- 172.21.15.112:0/3849365973 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08680042a0 con 0x7f0874108710 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.742+0000 7f087c200700 1 -- 172.21.15.112:0/3849365973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f087419fc10 con 0x7f0874108710 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.795+0000 7f0872000700 1 -- 172.21.15.112:0/3849365973 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f087419fc10 con 0x7f0874108710 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.798+0000 7f0872000700 1 -- 172.21.15.112:0/3849365973 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f086801a430 con 0x7f0874108710 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.799+0000 7f087c200700 1 -- 172.21.15.112:0/3849365973 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f085c0395b0 msgr2=0x7f085c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.799+0000 7f087c200700 1 --2- 172.21.15.112:0/3849365973 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f085c0395b0 0x7f085c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0860006fd0 tx=0x7f0860006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.799+0000 7f087c200700 1 -- 172.21.15.112:0/3849365973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 msgr2=0x7f087419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.799+0000 7f087c200700 1 --2- 172.21.15.112:0/3849365973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f087419c580 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f08680117b0 tx=0x7f0868011b70 comp rx=0 tx=0).stop 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.801+0000 7f087c200700 1 -- 172.21.15.112:0/3849365973 shutdown_connections 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.801+0000 7f087c200700 1 --2- 172.21.15.112:0/3849365973 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f085c0395b0 0x7f085c03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.801+0000 7f087c200700 1 --2- 172.21.15.112:0/3849365973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0874108710 0x7f087419c580 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.801+0000 7f087c200700 1 -- 172.21.15.112:0/3849365973 >> 172.21.15.112:0/3849365973 conn(0x7f0874103cf0 msgr2=0x7f0874104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.801+0000 7f087c200700 1 -- 172.21.15.112:0/3849365973 shutdown_connections 2024-04-25T18:34:12.600 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:11.801+0000 7f087c200700 1 -- 172.21.15.112:0/3849365973 wait complete. 2024-04-25T18:34:12.749 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:12 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:12.473+0000 7fb21f28c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:34:13.136 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:12 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:12.881+0000 7fb21f28c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:34:13.136 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:12 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:12.995+0000 7fb21f28c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:34:13.137 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:12 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3849365973' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T18:34:13.137 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:12 smithi112 ceph-mon[27864]: mgrmap e5: x(active, since 6s) 2024-04-25T18:34:14.059 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:13 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/457068914' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T18:34:14.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:34:14.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T18:34:14.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T18:34:14.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T18:34:14.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T18:34:14.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:34:14.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.289+0000 7fcb0ae00700 1 Processor -- start 2024-04-25T18:34:14.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.289+0000 7fcb0ae00700 1 -- start start 2024-04-25T18:34:14.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.289+0000 7fcb0ae00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0407c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:14.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.289+0000 7fcb0ae00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0407c7d0 con 0x7fcb0407dda0 2024-04-25T18:34:14.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.289+0000 7fcb03e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0407c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:14.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.290+0000 7fcb03e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0407c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46310/0 (socket says 172.21.15.112:46310) 2024-04-25T18:34:14.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.290+0000 7fcb03e00700 1 -- 172.21.15.112:0/2032547925 learned_addr learned my addr 172.21.15.112:0/2032547925 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:14.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.290+0000 7fcb03e00700 1 -- 172.21.15.112:0/2032547925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0407c950 con 0x7fcb0407dda0 2024-04-25T18:34:14.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.290+0000 7fcb03e00700 1 --2- 172.21.15.112:0/2032547925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0407c200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40098a0 tx=0x7fcaf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb028d4edd49491 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:14.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.291+0000 7fcb03400700 1 -- 172.21.15.112:0/2032547925 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaf4004160 con 0x7fcb0407dda0 2024-04-25T18:34:14.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.291+0000 7fcb03400700 1 -- 172.21.15.112:0/2032547925 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcaf4004300 con 0x7fcb0407dda0 2024-04-25T18:34:14.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.291+0000 7fcb03400700 1 -- 172.21.15.112:0/2032547925 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaf4011af0 con 0x7fcb0407dda0 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.291+0000 7fcb0ae00700 1 -- 172.21.15.112:0/2032547925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 msgr2=0x7fcb0407c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.291+0000 7fcb0ae00700 1 --2- 172.21.15.112:0/2032547925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0407c200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40098a0 tx=0x7fcaf4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.291+0000 7fcb0ae00700 1 -- 172.21.15.112:0/2032547925 shutdown_connections 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.291+0000 7fcb0ae00700 1 --2- 172.21.15.112:0/2032547925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0407c200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.291+0000 7fcb0ae00700 1 -- 172.21.15.112:0/2032547925 >> 172.21.15.112:0/2032547925 conn(0x7fcb04103c40 msgr2=0x7fcb041060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.292+0000 7fcb0ae00700 1 -- 172.21.15.112:0/2032547925 shutdown_connections 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.292+0000 7fcb0ae00700 1 -- 172.21.15.112:0/2032547925 wait complete. 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.293+0000 7fcb0ae00700 1 Processor -- start 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.293+0000 7fcb0ae00700 1 -- start start 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.293+0000 7fcb0ae00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.293+0000 7fcb0ae00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0410eba0 con 0x7fcb0407dda0 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.293+0000 7fcb03e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:14.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.293+0000 7fcb03e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46322/0 (socket says 172.21.15.112:46322) 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.293+0000 7fcb03e00700 1 -- 172.21.15.112:0/457068914 learned_addr learned my addr 172.21.15.112:0/457068914 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.294+0000 7fcb03e00700 1 -- 172.21.15.112:0/457068914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0419ec70 con 0x7fcb0407dda0 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.294+0000 7fcb03e00700 1 --2- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0419e730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4000c00 tx=0x7fcaf4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.294+0000 7fcb01600700 1 -- 172.21.15.112:0/457068914 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaf4003b20 con 0x7fcb0407dda0 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.294+0000 7fcb01600700 1 -- 172.21.15.112:0/457068914 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcaf401a8c0 con 0x7fcb0407dda0 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.294+0000 7fcb01600700 1 -- 172.21.15.112:0/457068914 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaf4024940 con 0x7fcb0407dda0 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.294+0000 7fcb0ae00700 1 -- 172.21.15.112:0/457068914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0419ef00 con 0x7fcb0407dda0 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.294+0000 7fcb0ae00700 1 -- 172.21.15.112:0/457068914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb041a1bf0 con 0x7fcb0407dda0 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.295+0000 7fcb01600700 1 -- 172.21.15.112:0/457068914 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fcaf4022020 con 0x7fcb0407dda0 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.295+0000 7fcb0ae00700 1 -- 172.21.15.112:0/457068914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb040640a0 con 0x7fcb0407dda0 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.296+0000 7fcb01600700 1 --2- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7fcae8039600 0x7fcae803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.296+0000 7fcb01600700 1 -- 172.21.15.112:0/457068914 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcaf404d490 con 0x7fcb0407dda0 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.296+0000 7fcafb400700 1 -- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7fcae8039600 msgr2=0x7fcae803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3468460974 2024-04-25T18:34:14.425 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.296+0000 7fcafb400700 1 --2- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7fcae8039600 0x7fcae803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.301+0000 7fcb01600700 1 -- 172.21.15.112:0/457068914 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcaf400be50 con 0x7fcb0407dda0 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.497+0000 7fcafb400700 1 -- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7fcae8039600 msgr2=0x7fcae803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3468460974 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.497+0000 7fcafb400700 1 --2- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7fcae8039600 0x7fcae803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.623+0000 7fcb0ae00700 1 -- 172.21.15.112:0/457068914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcb041a1e10 con 0x7fcb0407dda0 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.624+0000 7fcb01600700 1 -- 172.21.15.112:0/457068914 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fcaf404d1f0 con 0x7fcb0407dda0 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.626+0000 7fcb0ae00700 1 -- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7fcae8039600 msgr2=0x7fcae803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.626+0000 7fcb0ae00700 1 --2- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7fcae8039600 0x7fcae803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.626+0000 7fcb0ae00700 1 -- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 msgr2=0x7fcb0419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.626+0000 7fcb0ae00700 1 --2- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0419e730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4000c00 tx=0x7fcaf4011b70 comp rx=0 tx=0).stop 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.626+0000 7fcb0ae00700 1 -- 172.21.15.112:0/457068914 shutdown_connections 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.626+0000 7fcb0ae00700 1 --2- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7fcae8039600 0x7fcae803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.626+0000 7fcb0ae00700 1 --2- 172.21.15.112:0/457068914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb0407dda0 0x7fcb0419e730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.626+0000 7fcb0ae00700 1 -- 172.21.15.112:0/457068914 >> 172.21.15.112:0/457068914 conn(0x7fcb04103c40 msgr2=0x7fcb041047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:14.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.626+0000 7fcb0ae00700 1 -- 172.21.15.112:0/457068914 shutdown_connections 2024-04-25T18:34:14.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:13.627+0000 7fcb0ae00700 1 -- 172.21.15.112:0/457068914 wait complete. 2024-04-25T18:34:14.427 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-04-25T18:34:14.427 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 5... 2024-04-25T18:34:14.721 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:14 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:14.549+0000 7fb21f28c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:34:14.973 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:14 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:14.780+0000 7fb21f28c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:34:15.376 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:14 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:14.996+0000 7fb21f28c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:34:15.376 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:15.375+0000 7fb21f28c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:34:15.641 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:15.496+0000 7fb21f28c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:34:15.973 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:15.744+0000 7fb21f28c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:34:16.441 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:15.995+0000 7fb21f28c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:34:17.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:16 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:16.764+0000 7fb21f28c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:34:17.700 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:17 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:17.309+0000 7fb21f28c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:34:17.700 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:17 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:17.435+0000 7fb21f28c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:34:17.700 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:17 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:17.699+0000 7fb21f28c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:34:18.723 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:18 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:18.340+0000 7fb21f28c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:34:18.723 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:18 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:18.464+0000 7fb21f28c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:34:19.080 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:18 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:18.782+0000 7fb21f28c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:34:19.080 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:18 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:18.955+0000 7fb21f28c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:34:19.080 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:19 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:19.079+0000 7fb21f28c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:34:19.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:19 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:19.225+0000 7fb21f28c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:34:19.973 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:19 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:19.586+0000 7fb21f28c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:34:20.604 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:20.324+0000 7fb21f28c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:34:20.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:20.603+0000 7fb21f28c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:34:20.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: Active manager daemon x restarted 2024-04-25T18:34:20.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: Activating manager daemon x 2024-04-25T18:34:20.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T18:34:20.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: mgrmap e6: x(active, starting, since 0.00493098s) 2024-04-25T18:34:20.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:34:20.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:34:20.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:34:20.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:34:20.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:34:20.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: Manager daemon x is now available 2024-04-25T18:34:20.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:20.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:20 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:21.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:21 smithi112 ceph-mon[27864]: Found migration_current of "None". Setting to last migration. 2024-04-25T18:34:21.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:21 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:34:21.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:21 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:34:21.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:21 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:34:21.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:21 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:34:21.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:21 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:21.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:21 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:21.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:21 smithi112 ceph-mon[27864]: mgrmap e7: x(active, since 1.01049s) 2024-04-25T18:34:22.411 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:34:22.412 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T18:34:22.412 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T18:34:22.412 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:34:22.412 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.030+0000 7f49f6c00700 1 Processor -- start 2024-04-25T18:34:22.412 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.031+0000 7f49f6c00700 1 -- start start 2024-04-25T18:34:22.412 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.031+0000 7f49f6c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f0108ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:22.413 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.031+0000 7f49f6c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0109080 con 0x7f49f0108690 2024-04-25T18:34:22.413 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.031+0000 7f49efe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f0108ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:22.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.031+0000 7f49efe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f0108ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46326/0 (socket says 172.21.15.112:46326) 2024-04-25T18:34:22.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.031+0000 7f49efe00700 1 -- 172.21.15.112:0/2288244134 learned_addr learned my addr 172.21.15.112:0/2288244134 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:22.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.032+0000 7f49efe00700 1 -- 172.21.15.112:0/2288244134 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f0109890 con 0x7f49f0108690 2024-04-25T18:34:22.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.032+0000 7f49efe00700 1 --2- 172.21.15.112:0/2288244134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f0108ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49e00098a0 tx=0x7f49e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667043600eef4b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:22.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.032+0000 7f49eea00700 1 -- 172.21.15.112:0/2288244134 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0004160 con 0x7f49f0108690 2024-04-25T18:34:22.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.032+0000 7f49eea00700 1 -- 172.21.15.112:0/2288244134 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f49e0004300 con 0x7f49f0108690 2024-04-25T18:34:22.414 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.032+0000 7f49eea00700 1 -- 172.21.15.112:0/2288244134 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0011af0 con 0x7f49f0108690 2024-04-25T18:34:22.416 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.033+0000 7f49f6c00700 1 -- 172.21.15.112:0/2288244134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 msgr2=0x7f49f0108ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.033+0000 7f49f6c00700 1 --2- 172.21.15.112:0/2288244134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f0108ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49e00098a0 tx=0x7f49e0009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.033+0000 7f49f6c00700 1 -- 172.21.15.112:0/2288244134 shutdown_connections 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.033+0000 7f49f6c00700 1 --2- 172.21.15.112:0/2288244134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f0108ab0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.033+0000 7f49f6c00700 1 -- 172.21.15.112:0/2288244134 >> 172.21.15.112:0/2288244134 conn(0x7f49f0103c50 msgr2=0x7f49f0106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.033+0000 7f49f6c00700 1 -- 172.21.15.112:0/2288244134 shutdown_connections 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.034+0000 7f49f6c00700 1 -- 172.21.15.112:0/2288244134 wait complete. 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.035+0000 7f49f6c00700 1 Processor -- start 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.035+0000 7f49f6c00700 1 -- start start 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.035+0000 7f49f6c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f007da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.035+0000 7f49f6c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49f010cac0 con 0x7f49f0108690 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.035+0000 7f49efe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f007da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.035+0000 7f49efe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f007da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46334/0 (socket says 172.21.15.112:46334) 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.035+0000 7f49efe00700 1 -- 172.21.15.112:0/3909077902 learned_addr learned my addr 172.21.15.112:0/3909077902 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:22.417 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.036+0000 7f49efe00700 1 -- 172.21.15.112:0/3909077902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f007dfb0 con 0x7f49f0108690 2024-04-25T18:34:22.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.036+0000 7f49efe00700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f007da70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f49e00117b0 tx=0x7f49e0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:22.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.036+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0003af0 con 0x7f49f0108690 2024-04-25T18:34:22.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.036+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f49e0020070 con 0x7f49f0108690 2024-04-25T18:34:22.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.036+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0019630 con 0x7f49f0108690 2024-04-25T18:34:22.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.036+0000 7f49f6c00700 1 -- 172.21.15.112:0/3909077902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f007c1a0 con 0x7f49f0108690 2024-04-25T18:34:22.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.036+0000 7f49f6c00700 1 -- 172.21.15.112:0/3909077902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f007c6e0 con 0x7f49f0108690 2024-04-25T18:34:22.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.038+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f49e001a9c0 con 0x7f49f0108690 2024-04-25T18:34:22.418 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.038+0000 7f49ecc00700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 0x7f49d403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:22.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.038+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 --> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f49d403c1d0 con 0x7f49d4039600 2024-04-25T18:34:22.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.038+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f49e004db20 con 0x7f49f0108690 2024-04-25T18:34:22.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.038+0000 7f49ef400700 1 -- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 msgr2=0x7f49d403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3468460974 2024-04-25T18:34:22.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.038+0000 7f49ef400700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 0x7f49d403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:34:22.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.239+0000 7f49ef400700 1 -- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 msgr2=0x7f49d403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3468460974 2024-04-25T18:34:22.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.239+0000 7f49ef400700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 0x7f49d403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:34:22.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.640+0000 7f49ef400700 1 -- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 msgr2=0x7f49d403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3468460974 2024-04-25T18:34:22.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:15.640+0000 7f49ef400700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 0x7f49d403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T18:34:22.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:16.441+0000 7f49ef400700 1 -- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 msgr2=0x7f49d403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3468460974 2024-04-25T18:34:22.422 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:16.441+0000 7f49ef400700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 0x7f49d403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:18.043+0000 7f49ef400700 1 -- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 msgr2=0x7f49d403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3468460974 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:18.043+0000 7f49ef400700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 0x7f49d403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:20.610+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f49e001a430 con 0x7f49f0108690 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:20.611+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 msgr2=0x7f49d403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:20.611+0000 7f49ecc00700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 0x7f49d403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.614+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f49e004f7d0 con 0x7f49f0108690 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.614+0000 7f49ecc00700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f49d403cef0 0x7f49d403f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.614+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 --> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f49d403c1d0 con 0x7f49d403cef0 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.614+0000 7f49ef400700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f49d403cef0 0x7f49d403f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.615+0000 7f49ef400700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f49d403cef0 0x7f49d403f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f49dc003a10 tx=0x7f49dc0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.616+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 <== mgr.14120 v2:172.21.15.112:6800/733062511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f49d403c1d0 con 0x7f49d403cef0 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.623+0000 7f49f6c00700 1 -- 172.21.15.112:0/3909077902 --> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f49f00640a0 con 0x7f49d403cef0 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49ecc00700 1 -- 172.21.15.112:0/3909077902 <== mgr.14120 v2:172.21.15.112:6800/733062511 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f49f00640a0 con 0x7f49d403cef0 2024-04-25T18:34:22.423 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49f6c00700 1 -- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f49d403cef0 msgr2=0x7f49d403f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49f6c00700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f49d403cef0 0x7f49d403f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f49dc003a10 tx=0x7f49dc0092b0 comp rx=0 tx=0).stop 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49f6c00700 1 -- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 msgr2=0x7f49f007da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49f6c00700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f007da70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f49e00117b0 tx=0x7f49e0011b70 comp rx=0 tx=0).stop 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49f6c00700 1 -- 172.21.15.112:0/3909077902 shutdown_connections 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49f6c00700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f49d403cef0 0x7f49d403f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49f6c00700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f0108690 0x7f49f007da70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49f6c00700 1 --2- 172.21.15.112:0/3909077902 >> [v2:172.21.15.112:6800/3468460974,v1:172.21.15.112:6801/3468460974] conn(0x7f49d4039600 0x7f49d403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49f6c00700 1 -- 172.21.15.112:0/3909077902 >> 172.21.15.112:0/3909077902 conn(0x7f49f0103c50 msgr2=0x7f49f0104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49f6c00700 1 -- 172.21.15.112:0/3909077902 shutdown_connections 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:21.624+0000 7f49f6c00700 1 -- 172.21.15.112:0/3909077902 wait complete. 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 5 is available 2024-04-25T18:34:22.424 INFO:teuthology.orchestra.run.smithi112.stdout:Setting orchestrator backend to cephadm... 2024-04-25T18:34:22.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:22 smithi112 ceph-mon[27864]: [25/Apr/2024:18:34:21] ENGINE Bus STARTING 2024-04-25T18:34:22.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:22 smithi112 ceph-mon[27864]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T18:34:22.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:22 smithi112 ceph-mon[27864]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T18:34:22.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:22 smithi112 ceph-mon[27864]: [25/Apr/2024:18:34:21] ENGINE Serving on http://172.21.15.112:8765 2024-04-25T18:34:22.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:22 smithi112 ceph-mon[27864]: [25/Apr/2024:18:34:21] ENGINE Serving on https://172.21.15.112:7150 2024-04-25T18:34:22.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:22 smithi112 ceph-mon[27864]: [25/Apr/2024:18:34:21] ENGINE Bus STARTED 2024-04-25T18:34:22.967 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:22 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:34:23.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:23 smithi112 ceph-mon[27864]: mgrmap e8: x(active, since 2s) 2024-04-25T18:34:23.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:23 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:23.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:23 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:34:24.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.132+0000 7f3fb1800700 1 Processor -- start 2024-04-25T18:34:24.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.132+0000 7f3fb1800700 1 -- start start 2024-04-25T18:34:24.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.132+0000 7f3fb1800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac108b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:24.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.132+0000 7f3fb1800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac109100 con 0x7f3fac108710 2024-04-25T18:34:24.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.133+0000 7f3faaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac108b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:24.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.133+0000 7f3faaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac108b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42046/0 (socket says 172.21.15.112:42046) 2024-04-25T18:34:24.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.133+0000 7f3faaa00700 1 -- 172.21.15.112:0/3306272106 learned_addr learned my addr 172.21.15.112:0/3306272106 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:24.139 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.133+0000 7f3faaa00700 1 -- 172.21.15.112:0/3306272106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fac109910 con 0x7f3fac108710 2024-04-25T18:34:24.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.134+0000 7f3faaa00700 1 --2- 172.21.15.112:0/3306272106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac108b30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0098a0 tx=0x7f3f9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8259922683b043e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:24.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.134+0000 7f3fa9600700 1 -- 172.21.15.112:0/3306272106 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f9c004160 con 0x7f3fac108710 2024-04-25T18:34:24.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.134+0000 7f3fa9600700 1 -- 172.21.15.112:0/3306272106 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3f9c004300 con 0x7f3fac108710 2024-04-25T18:34:24.140 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.134+0000 7f3fa9600700 1 -- 172.21.15.112:0/3306272106 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f9c011af0 con 0x7f3fac108710 2024-04-25T18:34:24.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.135+0000 7f3fb1800700 1 -- 172.21.15.112:0/3306272106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 msgr2=0x7f3fac108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:24.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.135+0000 7f3fb1800700 1 --2- 172.21.15.112:0/3306272106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac108b30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0098a0 tx=0x7f3f9c009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:24.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.135+0000 7f3fb1800700 1 -- 172.21.15.112:0/3306272106 shutdown_connections 2024-04-25T18:34:24.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.135+0000 7f3fb1800700 1 --2- 172.21.15.112:0/3306272106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac108b30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:24.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.135+0000 7f3fb1800700 1 -- 172.21.15.112:0/3306272106 >> 172.21.15.112:0/3306272106 conn(0x7f3fac103cf0 msgr2=0x7f3fac106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:24.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.135+0000 7f3fb1800700 1 -- 172.21.15.112:0/3306272106 shutdown_connections 2024-04-25T18:34:24.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.135+0000 7f3fb1800700 1 -- 172.21.15.112:0/3306272106 wait complete. 2024-04-25T18:34:24.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.136+0000 7f3fb1800700 1 Processor -- start 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.136+0000 7f3fb1800700 1 -- start start 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.137+0000 7f3fb1800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.137+0000 7f3fb1800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac10cb40 con 0x7f3fac108710 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.137+0000 7f3faaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.137+0000 7f3faaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42058/0 (socket says 172.21.15.112:42058) 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.137+0000 7f3faaa00700 1 -- 172.21.15.112:0/1780040387 learned_addr learned my addr 172.21.15.112:0/1780040387 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.137+0000 7f3faaa00700 1 -- 172.21.15.112:0/1780040387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fac19cac0 con 0x7f3fac108710 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.138+0000 7f3faaa00700 1 --2- 172.21.15.112:0/1780040387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac19c580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0117b0 tx=0x7f3f9c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.138+0000 7f3fa3400700 1 -- 172.21.15.112:0/1780040387 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f9c003af0 con 0x7f3fac108710 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.138+0000 7f3fa3400700 1 -- 172.21.15.112:0/1780040387 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3f9c020070 con 0x7f3fac108710 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.138+0000 7f3fa3400700 1 -- 172.21.15.112:0/1780040387 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f9c019630 con 0x7f3fac108710 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.138+0000 7f3fb1800700 1 -- 172.21.15.112:0/1780040387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fac19cd50 con 0x7f3fac108710 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.138+0000 7f3fb1800700 1 -- 172.21.15.112:0/1780040387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fac19fa40 con 0x7f3fac108710 2024-04-25T18:34:24.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.139+0000 7f3fa3400700 1 -- 172.21.15.112:0/1780040387 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3f9c01a9c0 con 0x7f3fac108710 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.139+0000 7f3fb1800700 1 -- 172.21.15.112:0/1780040387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fac0640a0 con 0x7f3fac108710 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.140+0000 7f3fa3400700 1 --2- 172.21.15.112:0/1780040387 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f3f900395b0 0x7f3f9003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.140+0000 7f3fa3400700 1 -- 172.21.15.112:0/1780040387 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3f9c04dc00 con 0x7f3fac108710 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.140+0000 7f3faa000700 1 --2- 172.21.15.112:0/1780040387 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f3f900395b0 0x7f3f9003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.141+0000 7f3faa000700 1 --2- 172.21.15.112:0/1780040387 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f3f900395b0 0x7f3f9003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f98006fd0 tx=0x7f3f98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.146+0000 7f3fa3400700 1 -- 172.21.15.112:0/1780040387 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f9c00e260 con 0x7f3fac108710 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.389+0000 7f3fb1800700 1 -- 172.21.15.112:0/1780040387 --> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3fac104f30 con 0x7f3f900395b0 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.400+0000 7f3fa3400700 1 -- 172.21.15.112:0/1780040387 <== mgr.14120 v2:172.21.15.112:6800/733062511 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3fac104f30 con 0x7f3f900395b0 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.403+0000 7f3fb1800700 1 -- 172.21.15.112:0/1780040387 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f3f900395b0 msgr2=0x7f3f9003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.403+0000 7f3fb1800700 1 --2- 172.21.15.112:0/1780040387 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f3f900395b0 0x7f3f9003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f98006fd0 tx=0x7f3f98006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.403+0000 7f3fb1800700 1 -- 172.21.15.112:0/1780040387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 msgr2=0x7f3fac19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.403+0000 7f3fb1800700 1 --2- 172.21.15.112:0/1780040387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac19c580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0117b0 tx=0x7f3f9c011b70 comp rx=0 tx=0).stop 2024-04-25T18:34:24.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.403+0000 7f3fb1800700 1 -- 172.21.15.112:0/1780040387 shutdown_connections 2024-04-25T18:34:24.147 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.403+0000 7f3fb1800700 1 --2- 172.21.15.112:0/1780040387 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f3f900395b0 0x7f3f9003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:24.147 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.403+0000 7f3fb1800700 1 --2- 172.21.15.112:0/1780040387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108710 0x7f3fac19c580 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:24.147 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.403+0000 7f3fb1800700 1 -- 172.21.15.112:0/1780040387 >> 172.21.15.112:0/1780040387 conn(0x7f3fac103cf0 msgr2=0x7f3fac104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:24.147 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.403+0000 7f3fb1800700 1 -- 172.21.15.112:0/1780040387 shutdown_connections 2024-04-25T18:34:24.147 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:23.404+0000 7f3fb1800700 1 -- 172.21.15.112:0/1780040387 wait complete. 2024-04-25T18:34:24.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:24 smithi112 ceph-mon[27864]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:34:25.800 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T18:34:25.800 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.778+0000 7f9109c00700 1 Processor -- start 2024-04-25T18:34:25.800 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.778+0000 7f9109c00700 1 -- start start 2024-04-25T18:34:25.801 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.778+0000 7f9109c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f9104108920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:25.801 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.778+0000 7f9109c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9104108ef0 con 0x7f9104108500 2024-04-25T18:34:25.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.779+0000 7f9103e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f9104108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:25.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.779+0000 7f9103e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f9104108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42072/0 (socket says 172.21.15.112:42072) 2024-04-25T18:34:25.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.779+0000 7f9103e00700 1 -- 172.21.15.112:0/3978985854 learned_addr learned my addr 172.21.15.112:0/3978985854 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:25.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.779+0000 7f9103e00700 1 -- 172.21.15.112:0/3978985854 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9104109700 con 0x7f9104108500 2024-04-25T18:34:25.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.780+0000 7f9103e00700 1 --2- 172.21.15.112:0/3978985854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f9104108920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f90f40098a0 tx=0x7f90f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcc37ccc998d615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:25.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.780+0000 7f9102a00700 1 -- 172.21.15.112:0/3978985854 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90f4004160 con 0x7f9104108500 2024-04-25T18:34:25.806 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.780+0000 7f9102a00700 1 -- 172.21.15.112:0/3978985854 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f90f4004300 con 0x7f9104108500 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.780+0000 7f9102a00700 1 -- 172.21.15.112:0/3978985854 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90f4011af0 con 0x7f9104108500 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.781+0000 7f9109c00700 1 -- 172.21.15.112:0/3978985854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 msgr2=0x7f9104108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.781+0000 7f9109c00700 1 --2- 172.21.15.112:0/3978985854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f9104108920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f90f40098a0 tx=0x7f90f4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.781+0000 7f9109c00700 1 -- 172.21.15.112:0/3978985854 shutdown_connections 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.781+0000 7f9109c00700 1 --2- 172.21.15.112:0/3978985854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f9104108920 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.781+0000 7f9109c00700 1 -- 172.21.15.112:0/3978985854 >> 172.21.15.112:0/3978985854 conn(0x7f9104103aa0 msgr2=0x7f9104105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.781+0000 7f9109c00700 1 -- 172.21.15.112:0/3978985854 shutdown_connections 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.781+0000 7f9109c00700 1 -- 172.21.15.112:0/3978985854 wait complete. 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.782+0000 7f9109c00700 1 Processor -- start 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.782+0000 7f9109c00700 1 -- start start 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.782+0000 7f9109c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f910419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.782+0000 7f9109c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f910410c930 con 0x7f9104108500 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.783+0000 7f9103e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f910419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:25.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.783+0000 7f9103e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f910419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42082/0 (socket says 172.21.15.112:42082) 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.783+0000 7f9103e00700 1 -- 172.21.15.112:0/1380133916 learned_addr learned my addr 172.21.15.112:0/1380133916 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.783+0000 7f9103e00700 1 -- 172.21.15.112:0/1380133916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910419eb20 con 0x7f9104108500 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.783+0000 7f9103e00700 1 --2- 172.21.15.112:0/1380133916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f910419e5e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f90f40117b0 tx=0x7f90f4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.783+0000 7f9100c00700 1 -- 172.21.15.112:0/1380133916 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90f4003af0 con 0x7f9104108500 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.783+0000 7f9100c00700 1 -- 172.21.15.112:0/1380133916 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f90f4020070 con 0x7f9104108500 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.783+0000 7f9100c00700 1 -- 172.21.15.112:0/1380133916 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90f4019630 con 0x7f9104108500 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.784+0000 7f9109c00700 1 -- 172.21.15.112:0/1380133916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910419edb0 con 0x7f9104108500 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.784+0000 7f9109c00700 1 -- 172.21.15.112:0/1380133916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91041a1aa0 con 0x7f9104108500 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.785+0000 7f9100c00700 1 -- 172.21.15.112:0/1380133916 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f90f401a9c0 con 0x7f9104108500 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.785+0000 7f90fa000700 1 -- 172.21.15.112:0/1380133916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90dc005330 con 0x7f9104108500 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.785+0000 7f9100c00700 1 --2- 172.21.15.112:0/1380133916 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f90e8039600 0x7f90e803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.785+0000 7f9100c00700 1 -- 172.21.15.112:0/1380133916 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f90f404dfa0 con 0x7f9104108500 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.786+0000 7f9103400700 1 --2- 172.21.15.112:0/1380133916 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f90e8039600 0x7f90e803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:25.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.786+0000 7f9103400700 1 --2- 172.21.15.112:0/1380133916 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f90e8039600 0x7f90e803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f90f0006fd0 tx=0x7f90f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:24.796+0000 7f9100c00700 1 -- 172.21.15.112:0/1380133916 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90f4023370 con 0x7f9104108500 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.023+0000 7f90fa000700 1 -- 172.21.15.112:0/1380133916 --> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f90dc000bc0 con 0x7f90e8039600 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.024+0000 7f9100c00700 1 -- 172.21.15.112:0/1380133916 <== mgr.14120 v2:172.21.15.112:6800/733062511 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f90dc000bc0 con 0x7f90e8039600 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.027+0000 7f90fa000700 1 -- 172.21.15.112:0/1380133916 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f90e8039600 msgr2=0x7f90e803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.027+0000 7f90fa000700 1 --2- 172.21.15.112:0/1380133916 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f90e8039600 0x7f90e803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f90f0006fd0 tx=0x7f90f0006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.027+0000 7f90fa000700 1 -- 172.21.15.112:0/1380133916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 msgr2=0x7f910419e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.027+0000 7f90fa000700 1 --2- 172.21.15.112:0/1380133916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f910419e5e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f90f40117b0 tx=0x7f90f4011b70 comp rx=0 tx=0).stop 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.027+0000 7f90fa000700 1 -- 172.21.15.112:0/1380133916 shutdown_connections 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.027+0000 7f90fa000700 1 --2- 172.21.15.112:0/1380133916 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f90e8039600 0x7f90e803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.027+0000 7f90fa000700 1 --2- 172.21.15.112:0/1380133916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9104108500 0x7f910419e5e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.027+0000 7f90fa000700 1 -- 172.21.15.112:0/1380133916 >> 172.21.15.112:0/1380133916 conn(0x7f9104103aa0 msgr2=0x7f91041045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.027+0000 7f90fa000700 1 -- 172.21.15.112:0/1380133916 shutdown_connections 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:25.027+0000 7f90fa000700 1 -- 172.21.15.112:0/1380133916 wait complete. 2024-04-25T18:34:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:Generating ssh key... 2024-04-25T18:34:26.262 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:25 smithi112 ceph-mon[27864]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:34:26.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: Generating public/private rsa key pair. 2024-04-25T18:34:26.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: Your identification has been saved in /tmp/tmpc20291g9/key. 2024-04-25T18:34:26.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: Your public key has been saved in /tmp/tmpc20291g9/key.pub. 2024-04-25T18:34:26.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: The key fingerprint is: 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: SHA256:37Muf63asYG3J2+ue5hDJEv7c3cMckzX66DHHYNsEI0 ceph-2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: The key's randomart image is: 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: +---[RSA 3072]----+ 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: | o | 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: | E . | 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: | . .| 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: | .o o o| 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: | S .oB...| 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: | . .+*=+ | 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: | . B=B*o| 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: | .. BOBX| 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: | +*oB#=| 2024-04-25T18:34:26.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: +----[SHA256]-----+ 2024-04-25T18:34:27.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.423+0000 7f80c4a00700 1 Processor -- start 2024-04-25T18:34:27.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.423+0000 7f80c4a00700 1 -- start start 2024-04-25T18:34:27.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.423+0000 7f80c4a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c010b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:27.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.423+0000 7f80c4a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0063d60 con 0x7f80c0109110 2024-04-25T18:34:27.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.424+0000 7f80bde00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c010b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:27.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.424+0000 7f80bde00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c010b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42096/0 (socket says 172.21.15.112:42096) 2024-04-25T18:34:27.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.424+0000 7f80bde00700 1 -- 172.21.15.112:0/437681578 learned_addr learned my addr 172.21.15.112:0/437681578 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:27.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.424+0000 7f80bde00700 1 -- 172.21.15.112:0/437681578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c010ba40 con 0x7f80c0109110 2024-04-25T18:34:27.546 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.425+0000 7f80bde00700 1 --2- 172.21.15.112:0/437681578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c010b500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f80b00098a0 tx=0x7f80b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b8dd55deb5b71e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:27.547 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.425+0000 7f80bd400700 1 -- 172.21.15.112:0/437681578 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b0004160 con 0x7f80c0109110 2024-04-25T18:34:27.547 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.425+0000 7f80bd400700 1 -- 172.21.15.112:0/437681578 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f80b0004300 con 0x7f80c0109110 2024-04-25T18:34:27.547 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.425+0000 7f80bd400700 1 -- 172.21.15.112:0/437681578 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b0011af0 con 0x7f80c0109110 2024-04-25T18:34:27.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.426+0000 7f80c4a00700 1 -- 172.21.15.112:0/437681578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 msgr2=0x7f80c010b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:27.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.426+0000 7f80c4a00700 1 --2- 172.21.15.112:0/437681578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c010b500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f80b00098a0 tx=0x7f80b0009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:27.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.426+0000 7f80c4a00700 1 -- 172.21.15.112:0/437681578 shutdown_connections 2024-04-25T18:34:27.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.426+0000 7f80c4a00700 1 --2- 172.21.15.112:0/437681578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c010b500 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:27.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.426+0000 7f80c4a00700 1 -- 172.21.15.112:0/437681578 >> 172.21.15.112:0/437681578 conn(0x7f80c0102ba0 msgr2=0x7f80c0104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:27.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.426+0000 7f80c4a00700 1 -- 172.21.15.112:0/437681578 shutdown_connections 2024-04-25T18:34:27.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.426+0000 7f80c4a00700 1 -- 172.21.15.112:0/437681578 wait complete. 2024-04-25T18:34:27.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.427+0000 7f80c4a00700 1 Processor -- start 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.427+0000 7f80c4a00700 1 -- start start 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.428+0000 7f80c4a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.428+0000 7f80c4a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80c010c930 con 0x7f80c0109110 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.428+0000 7f80bde00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.428+0000 7f80bde00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42100/0 (socket says 172.21.15.112:42100) 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.428+0000 7f80bde00700 1 -- 172.21.15.112:0/4215666461 learned_addr learned my addr 172.21.15.112:0/4215666461 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.428+0000 7f80bde00700 1 -- 172.21.15.112:0/4215666461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c019ec30 con 0x7f80c0109110 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.428+0000 7f80bde00700 1 --2- 172.21.15.112:0/4215666461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c019e6f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f80b0011ac0 tx=0x7f80b0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.429+0000 7f80b6a00700 1 -- 172.21.15.112:0/4215666461 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b0003b20 con 0x7f80c0109110 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.429+0000 7f80b6a00700 1 -- 172.21.15.112:0/4215666461 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f80b001a8c0 con 0x7f80c0109110 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.429+0000 7f80b6a00700 1 -- 172.21.15.112:0/4215666461 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b00248d0 con 0x7f80c0109110 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.429+0000 7f80c4a00700 1 -- 172.21.15.112:0/4215666461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c019eec0 con 0x7f80c0109110 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.429+0000 7f80c4a00700 1 -- 172.21.15.112:0/4215666461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c01a1bb0 con 0x7f80c0109110 2024-04-25T18:34:27.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.430+0000 7f80b6a00700 1 -- 172.21.15.112:0/4215666461 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f80b0022020 con 0x7f80c0109110 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.430+0000 7f80c4a00700 1 -- 172.21.15.112:0/4215666461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c00640a0 con 0x7f80c0109110 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.430+0000 7f80b6a00700 1 --2- 172.21.15.112:0/4215666461 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f80a4039600 0x7f80a403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.431+0000 7f80b6a00700 1 -- 172.21.15.112:0/4215666461 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f80b004d820 con 0x7f80c0109110 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.431+0000 7f80b7e00700 1 --2- 172.21.15.112:0/4215666461 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f80a4039600 0x7f80a403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.432+0000 7f80b7e00700 1 --2- 172.21.15.112:0/4215666461 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f80a4039600 0x7f80a403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f80a8006fd0 tx=0x7f80a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.436+0000 7f80b6a00700 1 -- 172.21.15.112:0/4215666461 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80b001ab20 con 0x7f80c0109110 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.663+0000 7f80c4a00700 1 -- 172.21.15.112:0/4215666461 --> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f80c0103de0 con 0x7f80a4039600 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.774+0000 7f80b6a00700 1 -- 172.21.15.112:0/4215666461 <== mgr.14120 v2:172.21.15.112:6800/733062511 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f80c0103de0 con 0x7f80a4039600 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.777+0000 7f80c4a00700 1 -- 172.21.15.112:0/4215666461 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f80a4039600 msgr2=0x7f80a403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.777+0000 7f80c4a00700 1 --2- 172.21.15.112:0/4215666461 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f80a4039600 0x7f80a403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f80a8006fd0 tx=0x7f80a8006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.777+0000 7f80c4a00700 1 -- 172.21.15.112:0/4215666461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 msgr2=0x7f80c019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.777+0000 7f80c4a00700 1 --2- 172.21.15.112:0/4215666461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c019e6f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f80b0011ac0 tx=0x7f80b0011ba0 comp rx=0 tx=0).stop 2024-04-25T18:34:27.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.777+0000 7f80c4a00700 1 -- 172.21.15.112:0/4215666461 shutdown_connections 2024-04-25T18:34:27.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.777+0000 7f80c4a00700 1 --2- 172.21.15.112:0/4215666461 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f80a4039600 0x7f80a403bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:27.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.777+0000 7f80c4a00700 1 --2- 172.21.15.112:0/4215666461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c0109110 0x7f80c019e6f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:27.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.777+0000 7f80c4a00700 1 -- 172.21.15.112:0/4215666461 >> 172.21.15.112:0/4215666461 conn(0x7f80c0102ba0 msgr2=0x7f80c01036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:27.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.778+0000 7f80c4a00700 1 -- 172.21.15.112:0/4215666461 shutdown_connections 2024-04-25T18:34:27.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:26.778+0000 7f80c4a00700 1 -- 172.21.15.112:0/4215666461 wait complete. 2024-04-25T18:34:27.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:27 smithi112 ceph-mon[27864]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:34:27.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:27 smithi112 ceph-mon[27864]: Generating ssh key... 2024-04-25T18:34:27.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:27 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:27.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:27 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:29.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:28 smithi112 ceph-mon[27864]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2DUIksrtLuNu5/Ru0zw7LxtRazCt70+VyRCF/CW7c84qYHKocH66GkeRtAKsCfii+8WBzkTjxsajwST1wcQgX7rcRvvoqEY8hwmuyNV/lijnIH2bpWcvtve9CKm140lDmuEHfGe/2SXH7e3qhin1fywglYBz3UDNbkBslOjm6C48mIlGUxRZtflOol1vMfqiUeaFD3n0b9tciLdjBzhOEB/monULPGjaF+EGePzY2BXy4FSIIZ6SRoLHxRSn355NeETdOxhl8Yl6i+90ZWREtbugZr2CaIw9TWmTXYqji5TNAB6d9lAMrIXnSKudxJ8VQMyJAlAAiVgodbMdbMn9m7KMh2+U01H4NzpA4HqPuInQkzZj8VmyKqq3+ymXsSA+aWPOPto83BgkpjnyG8Fev+oULcI6gUuCMvoZviZXWBI4Vc045HsirFeNbXx9ZzR/XLtRmooVlEnQDiSmYcZt0UwE5EbujnavqR31CmgfzwA9bXj99Fsg3kKKPsGsbNSs= ceph-2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.190+0000 7fe79d400700 1 Processor -- start 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.190+0000 7fe79d400700 1 -- start start 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.190+0000 7fe79d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe79807fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.190+0000 7fe79d400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7980800a0 con 0x7fe79807f6b0 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.191+0000 7fe796800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe79807fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.191+0000 7fe796800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe79807fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42110/0 (socket says 172.21.15.112:42110) 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.191+0000 7fe796800700 1 -- 172.21.15.112:0/2819415500 learned_addr learned my addr 172.21.15.112:0/2819415500 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.191+0000 7fe796800700 1 -- 172.21.15.112:0/2819415500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe798080900 con 0x7fe79807f6b0 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.192+0000 7fe796800700 1 --2- 172.21.15.112:0/2819415500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe79807fad0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe7880098a0 tx=0x7fe788009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d678bcc7c7a7b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.192+0000 7fe795400700 1 -- 172.21.15.112:0/2819415500 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe788004160 con 0x7fe79807f6b0 2024-04-25T18:34:29.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.192+0000 7fe795400700 1 -- 172.21.15.112:0/2819415500 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe788004300 con 0x7fe79807f6b0 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.192+0000 7fe795400700 1 -- 172.21.15.112:0/2819415500 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe788011af0 con 0x7fe79807f6b0 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.193+0000 7fe79d400700 1 -- 172.21.15.112:0/2819415500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 msgr2=0x7fe79807fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.193+0000 7fe79d400700 1 --2- 172.21.15.112:0/2819415500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe79807fad0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe7880098a0 tx=0x7fe788009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.193+0000 7fe79d400700 1 -- 172.21.15.112:0/2819415500 shutdown_connections 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.193+0000 7fe79d400700 1 --2- 172.21.15.112:0/2819415500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe79807fad0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.193+0000 7fe79d400700 1 -- 172.21.15.112:0/2819415500 >> 172.21.15.112:0/2819415500 conn(0x7fe79807dce0 msgr2=0x7fe79807e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.193+0000 7fe79d400700 1 -- 172.21.15.112:0/2819415500 shutdown_connections 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.193+0000 7fe79d400700 1 -- 172.21.15.112:0/2819415500 wait complete. 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.194+0000 7fe79d400700 1 Processor -- start 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.194+0000 7fe79d400700 1 -- start start 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.194+0000 7fe79d400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe7981a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.194+0000 7fe79d400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe798081b40 con 0x7fe79807f6b0 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.195+0000 7fe796800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe7981a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:29.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.195+0000 7fe796800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe7981a2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42112/0 (socket says 172.21.15.112:42112) 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.195+0000 7fe796800700 1 -- 172.21.15.112:0/2174015253 learned_addr learned my addr 172.21.15.112:0/2174015253 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.195+0000 7fe796800700 1 -- 172.21.15.112:0/2174015253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7981a3050 con 0x7fe79807f6b0 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.195+0000 7fe796800700 1 --2- 172.21.15.112:0/2174015253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe7981a2b10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe788009fb0 tx=0x7fe788011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.196+0000 7fe78f400700 1 -- 172.21.15.112:0/2174015253 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7880036a0 con 0x7fe79807f6b0 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.196+0000 7fe78f400700 1 -- 172.21.15.112:0/2174015253 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe78801a430 con 0x7fe79807f6b0 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.196+0000 7fe79d400700 1 -- 172.21.15.112:0/2174015253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7981a32e0 con 0x7fe79807f6b0 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.196+0000 7fe78f400700 1 -- 172.21.15.112:0/2174015253 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe788019550 con 0x7fe79807f6b0 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.196+0000 7fe79d400700 1 -- 172.21.15.112:0/2174015253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7981a5fd0 con 0x7fe79807f6b0 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.197+0000 7fe78f400700 1 -- 172.21.15.112:0/2174015253 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe788003840 con 0x7fe79807f6b0 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.197+0000 7fe79d400700 1 -- 172.21.15.112:0/2174015253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7980640a0 con 0x7fe79807f6b0 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.197+0000 7fe78f400700 1 --2- 172.21.15.112:0/2174015253 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fe77c0395b0 0x7fe77c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.198+0000 7fe78f400700 1 -- 172.21.15.112:0/2174015253 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe78804d940 con 0x7fe79807f6b0 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.198+0000 7fe795e00700 1 --2- 172.21.15.112:0/2174015253 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fe77c0395b0 0x7fe77c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:29.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.198+0000 7fe795e00700 1 --2- 172.21.15.112:0/2174015253 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fe77c0395b0 0x7fe77c03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe784006fd0 tx=0x7fe784006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.203+0000 7fe78f400700 1 -- 172.21.15.112:0/2174015253 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7880172b0 con 0x7fe79807f6b0 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.433+0000 7fe79d400700 1 -- 172.21.15.112:0/2174015253 --> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fe7980008d0 con 0x7fe77c0395b0 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.435+0000 7fe78f400700 1 -- 172.21.15.112:0/2174015253 <== mgr.14120 v2:172.21.15.112:6800/733062511 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fe7980008d0 con 0x7fe77c0395b0 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.439+0000 7fe79d400700 1 -- 172.21.15.112:0/2174015253 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fe77c0395b0 msgr2=0x7fe77c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.439+0000 7fe79d400700 1 --2- 172.21.15.112:0/2174015253 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fe77c0395b0 0x7fe77c03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe784006fd0 tx=0x7fe784006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.439+0000 7fe79d400700 1 -- 172.21.15.112:0/2174015253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 msgr2=0x7fe7981a2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.439+0000 7fe79d400700 1 --2- 172.21.15.112:0/2174015253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe7981a2b10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe788009fb0 tx=0x7fe788011940 comp rx=0 tx=0).stop 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.439+0000 7fe79d400700 1 -- 172.21.15.112:0/2174015253 shutdown_connections 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.439+0000 7fe79d400700 1 --2- 172.21.15.112:0/2174015253 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fe77c0395b0 0x7fe77c03ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.439+0000 7fe79d400700 1 --2- 172.21.15.112:0/2174015253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe79807f6b0 0x7fe7981a2b10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.439+0000 7fe79d400700 1 -- 172.21.15.112:0/2174015253 >> 172.21.15.112:0/2174015253 conn(0x7fe79807dce0 msgr2=0x7fe798109240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.439+0000 7fe79d400700 1 -- 172.21.15.112:0/2174015253 shutdown_connections 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:28.439+0000 7fe79d400700 1 -- 172.21.15.112:0/2174015253 wait complete. 2024-04-25T18:34:29.174 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T18:34:29.175 INFO:teuthology.orchestra.run.smithi112.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T18:34:29.176 INFO:teuthology.orchestra.run.smithi112.stdout:Adding host smithi112... 2024-04-25T18:34:31.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:30 smithi112 ceph-mon[27864]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:34:32.143 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:31 smithi112 ceph-mon[27864]: Deploying cephadm binary to smithi112 2024-04-25T18:34:36.775 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Added host 'smithi112' with addr '172.21.15.112' 2024-04-25T18:34:36.775 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.830+0000 7fce6e800700 1 Processor -- start 2024-04-25T18:34:36.775 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.830+0000 7fce6e800700 1 -- start start 2024-04-25T18:34:36.775 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.831+0000 7fce6e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6810ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:36.775 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.831+0000 7fce6e800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce6810b310 con 0x7fce6810a920 2024-04-25T18:34:36.776 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.831+0000 7fce67400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6810ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:36.776 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.831+0000 7fce67400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6810ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43996/0 (socket says 172.21.15.112:43996) 2024-04-25T18:34:36.776 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.831+0000 7fce67400700 1 -- 172.21.15.112:0/961320650 learned_addr learned my addr 172.21.15.112:0/961320650 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:36.776 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.832+0000 7fce67400700 1 -- 172.21.15.112:0/961320650 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6810bb10 con 0x7fce6810a920 2024-04-25T18:34:36.776 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.832+0000 7fce67400700 1 --2- 172.21.15.112:0/961320650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6810ad40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fce580098a0 tx=0x7fce58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2a10c5c0c14a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:36.776 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.832+0000 7fce66000700 1 -- 172.21.15.112:0/961320650 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce58004160 con 0x7fce6810a920 2024-04-25T18:34:36.776 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.832+0000 7fce66000700 1 -- 172.21.15.112:0/961320650 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fce58004300 con 0x7fce6810a920 2024-04-25T18:34:36.776 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.832+0000 7fce66000700 1 -- 172.21.15.112:0/961320650 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce58011af0 con 0x7fce6810a920 2024-04-25T18:34:36.776 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.833+0000 7fce6e800700 1 -- 172.21.15.112:0/961320650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 msgr2=0x7fce6810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:36.776 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.833+0000 7fce6e800700 1 --2- 172.21.15.112:0/961320650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6810ad40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fce580098a0 tx=0x7fce58009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:36.776 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.833+0000 7fce6e800700 1 -- 172.21.15.112:0/961320650 shutdown_connections 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.833+0000 7fce6e800700 1 --2- 172.21.15.112:0/961320650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6810ad40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.833+0000 7fce6e800700 1 -- 172.21.15.112:0/961320650 >> 172.21.15.112:0/961320650 conn(0x7fce6807dce0 msgr2=0x7fce6807e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.833+0000 7fce6e800700 1 -- 172.21.15.112:0/961320650 shutdown_connections 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.833+0000 7fce6e800700 1 -- 172.21.15.112:0/961320650 wait complete. 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.834+0000 7fce6e800700 1 Processor -- start 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.835+0000 7fce6e800700 1 -- start start 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.835+0000 7fce6e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.835+0000 7fce6e800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce6810ed40 con 0x7fce6810a920 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.835+0000 7fce67400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.835+0000 7fce67400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6819e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43998/0 (socket says 172.21.15.112:43998) 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.835+0000 7fce67400700 1 -- 172.21.15.112:0/3646345971 learned_addr learned my addr 172.21.15.112:0/3646345971 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.835+0000 7fce67400700 1 -- 172.21.15.112:0/3646345971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6819ec60 con 0x7fce6810a920 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.836+0000 7fce67400700 1 --2- 172.21.15.112:0/3646345971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6819e720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fce58011ac0 tx=0x7fce58011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:36.777 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.836+0000 7fce5fe00700 1 -- 172.21.15.112:0/3646345971 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce58003970 con 0x7fce6810a920 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.836+0000 7fce5fe00700 1 -- 172.21.15.112:0/3646345971 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fce58003b10 con 0x7fce6810a920 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.836+0000 7fce5fe00700 1 -- 172.21.15.112:0/3646345971 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce58024610 con 0x7fce6810a920 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.836+0000 7fce6e800700 1 -- 172.21.15.112:0/3646345971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6819eef0 con 0x7fce6810a920 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.836+0000 7fce6e800700 1 -- 172.21.15.112:0/3646345971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce681a1be0 con 0x7fce6810a920 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.837+0000 7fce5fe00700 1 -- 172.21.15.112:0/3646345971 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fce5801a9e0 con 0x7fce6810a920 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.837+0000 7fce6e800700 1 -- 172.21.15.112:0/3646345971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce680640a0 con 0x7fce6810a920 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.838+0000 7fce5fe00700 1 --2- 172.21.15.112:0/3646345971 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fce4c039600 0x7fce4c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.838+0000 7fce5fe00700 1 -- 172.21.15.112:0/3646345971 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fce5804d970 con 0x7fce6810a920 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.838+0000 7fce66a00700 1 --2- 172.21.15.112:0/3646345971 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fce4c039600 0x7fce4c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.839+0000 7fce66a00700 1 --2- 172.21.15.112:0/3646345971 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fce4c039600 0x7fce4c03bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fce54006fd0 tx=0x7fce54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:29.843+0000 7fce5fe00700 1 -- 172.21.15.112:0/3646345971 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce5800b3e0 con 0x7fce6810a920 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:30.071+0000 7fce6e800700 1 -- 172.21.15.112:0/3646345971 --> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}) -- 0x7fce68108bf0 con 0x7fce4c039600 2024-04-25T18:34:36.778 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:36.025+0000 7fce5fe00700 1 -- 172.21.15.112:0/3646345971 <== mgr.14120 v2:172.21.15.112:6800/733062511 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fce68108bf0 con 0x7fce4c039600 2024-04-25T18:34:36.779 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:36.030+0000 7fce6e800700 1 -- 172.21.15.112:0/3646345971 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fce4c039600 msgr2=0x7fce4c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:36.779 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:36.030+0000 7fce6e800700 1 --2- 172.21.15.112:0/3646345971 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fce4c039600 0x7fce4c03bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fce54006fd0 tx=0x7fce54006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:36.779 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:36.030+0000 7fce6e800700 1 -- 172.21.15.112:0/3646345971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 msgr2=0x7fce6819e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:36.779 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:36.030+0000 7fce6e800700 1 --2- 172.21.15.112:0/3646345971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6819e720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fce58011ac0 tx=0x7fce58011ba0 comp rx=0 tx=0).stop 2024-04-25T18:34:36.779 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:36.030+0000 7fce6e800700 1 -- 172.21.15.112:0/3646345971 shutdown_connections 2024-04-25T18:34:36.779 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:36.030+0000 7fce6e800700 1 --2- 172.21.15.112:0/3646345971 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fce4c039600 0x7fce4c03bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:36.779 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:36.030+0000 7fce6e800700 1 --2- 172.21.15.112:0/3646345971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce6810a920 0x7fce6819e720 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:36.779 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:36.030+0000 7fce6e800700 1 -- 172.21.15.112:0/3646345971 >> 172.21.15.112:0/3646345971 conn(0x7fce6807dce0 msgr2=0x7fce681084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:36.779 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:36.031+0000 7fce6e800700 1 -- 172.21.15.112:0/3646345971 shutdown_connections 2024-04-25T18:34:36.779 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:36.031+0000 7fce6e800700 1 -- 172.21.15.112:0/3646345971 wait complete. 2024-04-25T18:34:36.779 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mon service... 2024-04-25T18:34:37.308 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:37 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:37.308 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:37 smithi112 ceph-mon[27864]: Added host smithi112 2024-04-25T18:34:37.308 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:37 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:34:38.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T18:34:38.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.476+0000 7fae50a00700 1 Processor -- start 2024-04-25T18:34:38.742 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.476+0000 7fae50a00700 1 -- start start 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.477+0000 7fae50a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c108920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.477+0000 7fae50a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c108ef0 con 0x7fae4c108500 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.477+0000 7fae4a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.478+0000 7fae4a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44014/0 (socket says 172.21.15.112:44014) 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.478+0000 7fae4a800700 1 -- 172.21.15.112:0/1597823845 learned_addr learned my addr 172.21.15.112:0/1597823845 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.478+0000 7fae4a800700 1 -- 172.21.15.112:0/1597823845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c109700 con 0x7fae4c108500 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.478+0000 7fae4a800700 1 --2- 172.21.15.112:0/1597823845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c108920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009e80 tx=0x7fae3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7dc95d9a68b4a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.479+0000 7fae49e00700 1 -- 172.21.15.112:0/1597823845 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3c017070 con 0x7fae4c108500 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.479+0000 7fae49e00700 1 -- 172.21.15.112:0/1597823845 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fae3c00b8c0 con 0x7fae4c108500 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.479+0000 7fae49e00700 1 -- 172.21.15.112:0/1597823845 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3c004490 con 0x7fae4c108500 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.479+0000 7fae50a00700 1 -- 172.21.15.112:0/1597823845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 msgr2=0x7fae4c108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:38.744 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.479+0000 7fae50a00700 1 --2- 172.21.15.112:0/1597823845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c108920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009e80 tx=0x7fae3c00b330 comp rx=0 tx=0).stop 2024-04-25T18:34:38.749 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.480+0000 7fae50a00700 1 -- 172.21.15.112:0/1597823845 shutdown_connections 2024-04-25T18:34:38.749 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.480+0000 7fae50a00700 1 --2- 172.21.15.112:0/1597823845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c108920 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:38.749 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.480+0000 7fae50a00700 1 -- 172.21.15.112:0/1597823845 >> 172.21.15.112:0/1597823845 conn(0x7fae4c103aa0 msgr2=0x7fae4c105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:38.749 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.480+0000 7fae50a00700 1 -- 172.21.15.112:0/1597823845 shutdown_connections 2024-04-25T18:34:38.749 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.480+0000 7fae50a00700 1 -- 172.21.15.112:0/1597823845 wait complete. 2024-04-25T18:34:38.749 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.481+0000 7fae50a00700 1 Processor -- start 2024-04-25T18:34:38.749 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.481+0000 7fae50a00700 1 -- start start 2024-04-25T18:34:38.749 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.481+0000 7fae50a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:38.749 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.481+0000 7fae50a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c10c930 con 0x7fae4c108500 2024-04-25T18:34:38.750 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.482+0000 7fae4a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:38.750 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.482+0000 7fae4a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c19e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44020/0 (socket says 172.21.15.112:44020) 2024-04-25T18:34:38.750 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.482+0000 7fae4a800700 1 -- 172.21.15.112:0/372885552 learned_addr learned my addr 172.21.15.112:0/372885552 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:38.750 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.482+0000 7fae4a800700 1 -- 172.21.15.112:0/372885552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c19eab0 con 0x7fae4c108500 2024-04-25T18:34:38.750 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.482+0000 7fae4a800700 1 --2- 172.21.15.112:0/372885552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c19e570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00b7b0 tx=0x7fae3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:38.750 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.483+0000 7fae43400700 1 -- 172.21.15.112:0/372885552 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3c020020 con 0x7fae4c108500 2024-04-25T18:34:38.750 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.483+0000 7fae43400700 1 -- 172.21.15.112:0/372885552 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fae3c012430 con 0x7fae4c108500 2024-04-25T18:34:38.750 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.483+0000 7fae50a00700 1 -- 172.21.15.112:0/372885552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4c19ed40 con 0x7fae4c108500 2024-04-25T18:34:38.750 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.483+0000 7fae43400700 1 -- 172.21.15.112:0/372885552 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae3c017040 con 0x7fae4c108500 2024-04-25T18:34:38.750 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.483+0000 7fae50a00700 1 -- 172.21.15.112:0/372885552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4c1a1a30 con 0x7fae4c108500 2024-04-25T18:34:38.750 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.484+0000 7fae43400700 1 -- 172.21.15.112:0/372885552 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fae3c023020 con 0x7fae4c108500 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.484+0000 7fae50a00700 1 -- 172.21.15.112:0/372885552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4c0640a0 con 0x7fae4c108500 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.484+0000 7fae43400700 1 --2- 172.21.15.112:0/372885552 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fae300395b0 0x7fae3003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.484+0000 7fae43400700 1 -- 172.21.15.112:0/372885552 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fae3c00f050 con 0x7fae4c108500 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.485+0000 7fae43e00700 1 --2- 172.21.15.112:0/372885552 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fae300395b0 0x7fae3003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.485+0000 7fae43e00700 1 --2- 172.21.15.112:0/372885552 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fae300395b0 0x7fae3003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae34006fd0 tx=0x7fae34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.490+0000 7fae43400700 1 -- 172.21.15.112:0/372885552 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae3c02a020 con 0x7fae4c108500 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.769+0000 7fae50a00700 1 -- 172.21.15.112:0/372885552 --> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fae4c104c70 con 0x7fae300395b0 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.780+0000 7fae43400700 1 -- 172.21.15.112:0/372885552 <== mgr.14120 v2:172.21.15.112:6800/733062511 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fae4c104c70 con 0x7fae300395b0 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.783+0000 7fae50a00700 1 -- 172.21.15.112:0/372885552 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fae300395b0 msgr2=0x7fae3003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.783+0000 7fae50a00700 1 --2- 172.21.15.112:0/372885552 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fae300395b0 0x7fae3003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae34006fd0 tx=0x7fae34006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.783+0000 7fae50a00700 1 -- 172.21.15.112:0/372885552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 msgr2=0x7fae4c19e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.783+0000 7fae50a00700 1 --2- 172.21.15.112:0/372885552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c19e570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00b7b0 tx=0x7fae3c00be20 comp rx=0 tx=0).stop 2024-04-25T18:34:38.751 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.783+0000 7fae50a00700 1 -- 172.21.15.112:0/372885552 shutdown_connections 2024-04-25T18:34:38.752 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.783+0000 7fae50a00700 1 --2- 172.21.15.112:0/372885552 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7fae300395b0 0x7fae3003ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:38.752 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.783+0000 7fae50a00700 1 --2- 172.21.15.112:0/372885552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae4c108500 0x7fae4c19e570 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:38.752 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.783+0000 7fae50a00700 1 -- 172.21.15.112:0/372885552 >> 172.21.15.112:0/372885552 conn(0x7fae4c103aa0 msgr2=0x7fae4c104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:38.752 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.784+0000 7fae50a00700 1 -- 172.21.15.112:0/372885552 shutdown_connections 2024-04-25T18:34:38.752 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:37.784+0000 7fae50a00700 1 -- 172.21.15.112:0/372885552 wait complete. 2024-04-25T18:34:38.752 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mgr service... 2024-04-25T18:34:39.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:38 smithi112 ceph-mon[27864]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:34:39.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:38 smithi112 ceph-mon[27864]: Saving service mon spec with placement count:5 2024-04-25T18:34:39.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:38 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:41.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T18:34:41.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.902+0000 7f95f9600700 1 Processor -- start 2024-04-25T18:34:41.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.902+0000 7f95f9600700 1 -- start start 2024-04-25T18:34:41.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.902+0000 7f95f9600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f41089a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:41.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.902+0000 7f95f9600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95f4108f70 con 0x7f95f4108580 2024-04-25T18:34:41.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.903+0000 7f95f3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f41089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:41.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.903+0000 7f95f3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f41089a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48366/0 (socket says 172.21.15.112:48366) 2024-04-25T18:34:41.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.903+0000 7f95f3e00700 1 -- 172.21.15.112:0/2809103270 learned_addr learned my addr 172.21.15.112:0/2809103270 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:41.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.903+0000 7f95f3e00700 1 -- 172.21.15.112:0/2809103270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f4109780 con 0x7f95f4108580 2024-04-25T18:34:41.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.904+0000 7f95f3e00700 1 --2- 172.21.15.112:0/2809103270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f41089a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f95e40098a0 tx=0x7f95e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b036a87ee80873a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.904+0000 7f95f2a00700 1 -- 172.21.15.112:0/2809103270 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95e4004160 con 0x7f95f4108580 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.904+0000 7f95f2a00700 1 -- 172.21.15.112:0/2809103270 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f95e4004300 con 0x7f95f4108580 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.904+0000 7f95f2a00700 1 -- 172.21.15.112:0/2809103270 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95e4011af0 con 0x7f95f4108580 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.904+0000 7f95f9600700 1 -- 172.21.15.112:0/2809103270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 msgr2=0x7f95f41089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.904+0000 7f95f9600700 1 --2- 172.21.15.112:0/2809103270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f41089a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f95e40098a0 tx=0x7f95e4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.905+0000 7f95f9600700 1 -- 172.21.15.112:0/2809103270 shutdown_connections 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.905+0000 7f95f9600700 1 --2- 172.21.15.112:0/2809103270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f41089a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.905+0000 7f95f9600700 1 -- 172.21.15.112:0/2809103270 >> 172.21.15.112:0/2809103270 conn(0x7f95f4103b40 msgr2=0x7f95f4105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.905+0000 7f95f9600700 1 -- 172.21.15.112:0/2809103270 shutdown_connections 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.905+0000 7f95f9600700 1 -- 172.21.15.112:0/2809103270 wait complete. 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.906+0000 7f95f9600700 1 Processor -- start 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.906+0000 7f95f9600700 1 -- start start 2024-04-25T18:34:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.906+0000 7f95f9600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.906+0000 7f95f9600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410c9b0 con 0x7f95f4108580 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.906+0000 7f95f3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.906+0000 7f95f3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48382/0 (socket says 172.21.15.112:48382) 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.906+0000 7f95f3e00700 1 -- 172.21.15.112:0/3646118905 learned_addr learned my addr 172.21.15.112:0/3646118905 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.907+0000 7f95f3e00700 1 -- 172.21.15.112:0/3646118905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f419eb60 con 0x7f95f4108580 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.907+0000 7f95f3e00700 1 --2- 172.21.15.112:0/3646118905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f419e620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009fb0 tx=0x7f95e4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.907+0000 7f95f0c00700 1 -- 172.21.15.112:0/3646118905 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95e40036a0 con 0x7f95f4108580 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.907+0000 7f95f0c00700 1 -- 172.21.15.112:0/3646118905 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f95e401a430 con 0x7f95f4108580 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.907+0000 7f95f0c00700 1 -- 172.21.15.112:0/3646118905 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95e4019550 con 0x7f95f4108580 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.907+0000 7f95f9600700 1 -- 172.21.15.112:0/3646118905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f419edf0 con 0x7f95f4108580 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.908+0000 7f95f9600700 1 -- 172.21.15.112:0/3646118905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f41a1ae0 con 0x7f95f4108580 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.909+0000 7f95f0c00700 1 -- 172.21.15.112:0/3646118905 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f95e401a8e0 con 0x7f95f4108580 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.909+0000 7f95f9600700 1 -- 172.21.15.112:0/3646118905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f40640a0 con 0x7f95f4108580 2024-04-25T18:34:41.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.909+0000 7f95f0c00700 1 --2- 172.21.15.112:0/3646118905 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f95d80395b0 0x7f95d803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:41.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.909+0000 7f95f0c00700 1 -- 172.21.15.112:0/3646118905 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f95e404d990 con 0x7f95f4108580 2024-04-25T18:34:41.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.909+0000 7f95f3400700 1 --2- 172.21.15.112:0/3646118905 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f95d80395b0 0x7f95d803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:41.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.910+0000 7f95f3400700 1 --2- 172.21.15.112:0/3646118905 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f95d80395b0 0x7f95d803ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f95e0006fd0 tx=0x7f95e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:41.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:39.915+0000 7f95f0c00700 1 -- 172.21.15.112:0/3646118905 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95e401a690 con 0x7f95f4108580 2024-04-25T18:34:41.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.155+0000 7f95f9600700 1 -- 172.21.15.112:0/3646118905 --> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f95f4104d20 con 0x7f95d80395b0 2024-04-25T18:34:41.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.163+0000 7f95f0c00700 1 -- 172.21.15.112:0/3646118905 <== mgr.14120 v2:172.21.15.112:6800/733062511 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f95f4104d20 con 0x7f95d80395b0 2024-04-25T18:34:41.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.166+0000 7f95f9600700 1 -- 172.21.15.112:0/3646118905 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f95d80395b0 msgr2=0x7f95d803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:41.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.166+0000 7f95f9600700 1 --2- 172.21.15.112:0/3646118905 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f95d80395b0 0x7f95d803ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f95e0006fd0 tx=0x7f95e0006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:41.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.166+0000 7f95f9600700 1 -- 172.21.15.112:0/3646118905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 msgr2=0x7f95f419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:41.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.166+0000 7f95f9600700 1 --2- 172.21.15.112:0/3646118905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f419e620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009fb0 tx=0x7f95e4011940 comp rx=0 tx=0).stop 2024-04-25T18:34:41.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.166+0000 7f95f9600700 1 -- 172.21.15.112:0/3646118905 shutdown_connections 2024-04-25T18:34:41.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.166+0000 7f95f9600700 1 --2- 172.21.15.112:0/3646118905 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f95d80395b0 0x7f95d803ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:41.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.166+0000 7f95f9600700 1 --2- 172.21.15.112:0/3646118905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95f4108580 0x7f95f419e620 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:41.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.166+0000 7f95f9600700 1 -- 172.21.15.112:0/3646118905 >> 172.21.15.112:0/3646118905 conn(0x7f95f4103b40 msgr2=0x7f95f4104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:41.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.166+0000 7f95f9600700 1 -- 172.21.15.112:0/3646118905 shutdown_connections 2024-04-25T18:34:41.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:40.166+0000 7f95f9600700 1 -- 172.21.15.112:0/3646118905 wait complete. 2024-04-25T18:34:41.327 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:41 smithi112 ceph-mon[27864]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:34:41.327 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:41 smithi112 ceph-mon[27864]: Saving service mgr spec with placement count:2 2024-04-25T18:34:41.327 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:41 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:41.327 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:41 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:41.327 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:41 smithi112 ceph-mon[27864]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:34:41.327 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:41 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:42.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.672+0000 7f09e7e00700 1 Processor -- start 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.672+0000 7f09e7e00700 1 -- start start 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.673+0000 7f09e7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e807c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.673+0000 7f09e7e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09e807c7b0 con 0x7f09e807dd80 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.673+0000 7f09e6a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e807c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.673+0000 7f09e6a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e807c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48394/0 (socket says 172.21.15.112:48394) 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.673+0000 7f09e6a00700 1 -- 172.21.15.112:0/2665727667 learned_addr learned my addr 172.21.15.112:0/2665727667 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.674+0000 7f09e6a00700 1 -- 172.21.15.112:0/2665727667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e807c930 con 0x7f09e807dd80 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.674+0000 7f09e6a00700 1 --2- 172.21.15.112:0/2665727667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e807c1e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09d80098a0 tx=0x7f09d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aba8e94b0e5e054 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.674+0000 7f09e5600700 1 -- 172.21.15.112:0/2665727667 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09d8004160 con 0x7f09e807dd80 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.674+0000 7f09e5600700 1 -- 172.21.15.112:0/2665727667 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d8004300 con 0x7f09e807dd80 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.674+0000 7f09e5600700 1 -- 172.21.15.112:0/2665727667 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09d8011a20 con 0x7f09e807dd80 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.675+0000 7f09e7e00700 1 -- 172.21.15.112:0/2665727667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 msgr2=0x7f09e807c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:42.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.675+0000 7f09e7e00700 1 --2- 172.21.15.112:0/2665727667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e807c1e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09d80098a0 tx=0x7f09d8009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.675+0000 7f09e7e00700 1 -- 172.21.15.112:0/2665727667 shutdown_connections 2024-04-25T18:34:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.675+0000 7f09e7e00700 1 --2- 172.21.15.112:0/2665727667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e807c1e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.675+0000 7f09e7e00700 1 -- 172.21.15.112:0/2665727667 >> 172.21.15.112:0/2665727667 conn(0x7f09e8103aa0 msgr2=0x7f09e8105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:42.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.675+0000 7f09e7e00700 1 -- 172.21.15.112:0/2665727667 shutdown_connections 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.675+0000 7f09e7e00700 1 -- 172.21.15.112:0/2665727667 wait complete. 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.676+0000 7f09e7e00700 1 Processor -- start 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.676+0000 7f09e7e00700 1 -- start start 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.676+0000 7f09e7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.676+0000 7f09e7e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09e810eb20 con 0x7f09e807dd80 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.677+0000 7f09e6a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.677+0000 7f09e6a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48410/0 (socket says 172.21.15.112:48410) 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.677+0000 7f09e6a00700 1 -- 172.21.15.112:0/325764731 learned_addr learned my addr 172.21.15.112:0/325764731 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.677+0000 7f09e6a00700 1 -- 172.21.15.112:0/325764731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e819ea80 con 0x7f09e807dd80 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.677+0000 7f09e6a00700 1 --2- 172.21.15.112:0/325764731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e819e540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f09d8000c00 tx=0x7f09d8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.678+0000 7f09df400700 1 -- 172.21.15.112:0/325764731 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09d8003aa0 con 0x7f09e807dd80 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.678+0000 7f09df400700 1 -- 172.21.15.112:0/325764731 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d801b8f0 con 0x7f09e807dd80 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.678+0000 7f09df400700 1 -- 172.21.15.112:0/325764731 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09d8023870 con 0x7f09e807dd80 2024-04-25T18:34:42.721 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.678+0000 7f09e7e00700 1 -- 172.21.15.112:0/325764731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e819ed10 con 0x7f09e807dd80 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.678+0000 7f09e7e00700 1 -- 172.21.15.112:0/325764731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e81a1a00 con 0x7f09e807dd80 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.679+0000 7f09e7e00700 1 -- 172.21.15.112:0/325764731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e80640a0 con 0x7f09e807dd80 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.679+0000 7f09df400700 1 -- 172.21.15.112:0/325764731 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f09d801b440 con 0x7f09e807dd80 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.680+0000 7f09df400700 1 --2- 172.21.15.112:0/325764731 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f09cc039600 0x7f09cc03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.680+0000 7f09df400700 1 -- 172.21.15.112:0/325764731 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f09d804dea0 con 0x7f09e807dd80 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.680+0000 7f09e6000700 1 --2- 172.21.15.112:0/325764731 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f09cc039600 0x7f09cc03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.681+0000 7f09e6000700 1 --2- 172.21.15.112:0/325764731 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f09cc039600 0x7f09cc03bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f09d4006fd0 tx=0x7f09d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.685+0000 7f09df400700 1 -- 172.21.15.112:0/325764731 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09d801f030 con 0x7f09e807dd80 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.956+0000 7f09e7e00700 1 -- 172.21.15.112:0/325764731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f09e81a2000 con 0x7f09e807dd80 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.959+0000 7f09df400700 1 -- 172.21.15.112:0/325764731 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f09d800e260 con 0x7f09e807dd80 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.963+0000 7f09e7e00700 1 -- 172.21.15.112:0/325764731 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f09cc039600 msgr2=0x7f09cc03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.963+0000 7f09e7e00700 1 --2- 172.21.15.112:0/325764731 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f09cc039600 0x7f09cc03bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f09d4006fd0 tx=0x7f09d4006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:42.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.963+0000 7f09e7e00700 1 -- 172.21.15.112:0/325764731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 msgr2=0x7f09e819e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.963+0000 7f09e7e00700 1 --2- 172.21.15.112:0/325764731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e819e540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f09d8000c00 tx=0x7f09d8011aa0 comp rx=0 tx=0).stop 2024-04-25T18:34:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.964+0000 7f09e7e00700 1 -- 172.21.15.112:0/325764731 shutdown_connections 2024-04-25T18:34:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.964+0000 7f09e7e00700 1 --2- 172.21.15.112:0/325764731 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f09cc039600 0x7f09cc03bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.964+0000 7f09e7e00700 1 --2- 172.21.15.112:0/325764731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09e807dd80 0x7f09e819e540 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.964+0000 7f09e7e00700 1 -- 172.21.15.112:0/325764731 >> 172.21.15.112:0/325764731 conn(0x7f09e8103aa0 msgr2=0x7f09e81045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.964+0000 7f09e7e00700 1 -- 172.21.15.112:0/325764731 shutdown_connections 2024-04-25T18:34:42.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:41.964+0000 7f09e7e00700 1 -- 172.21.15.112:0/325764731 wait complete. 2024-04-25T18:34:43.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:42 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/325764731' entity='client.admin' 2024-04-25T18:34:43.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:42 smithi112 ceph-mon[27864]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:34:44.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.569+0000 7f4dc3200700 1 Processor -- start 2024-04-25T18:34:44.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.570+0000 7f4dc3200700 1 -- start start 2024-04-25T18:34:44.807 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.570+0000 7f4dc3200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc10ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.570+0000 7f4dc3200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc10b310 con 0x7f4dbc10a920 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.570+0000 7f4dbbe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc10ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.571+0000 7f4dbbe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc10ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48422/0 (socket says 172.21.15.112:48422) 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.571+0000 7f4dbbe00700 1 -- 172.21.15.112:0/426778896 learned_addr learned my addr 172.21.15.112:0/426778896 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.571+0000 7f4dbbe00700 1 -- 172.21.15.112:0/426778896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc10bb10 con 0x7f4dbc10a920 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.571+0000 7f4dbbe00700 1 --2- 172.21.15.112:0/426778896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc10ad40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0098a0 tx=0x7f4dac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3a7c5ad68c4afa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.572+0000 7f4dbaa00700 1 -- 172.21.15.112:0/426778896 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dac004160 con 0x7f4dbc10a920 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.572+0000 7f4dbaa00700 1 -- 172.21.15.112:0/426778896 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dac004300 con 0x7f4dbc10a920 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.572+0000 7f4dbaa00700 1 -- 172.21.15.112:0/426778896 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dac011a20 con 0x7f4dbc10a920 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.572+0000 7f4dc3200700 1 -- 172.21.15.112:0/426778896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 msgr2=0x7f4dbc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.572+0000 7f4dc3200700 1 --2- 172.21.15.112:0/426778896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc10ad40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0098a0 tx=0x7f4dac009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.572+0000 7f4dc3200700 1 -- 172.21.15.112:0/426778896 shutdown_connections 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.572+0000 7f4dc3200700 1 --2- 172.21.15.112:0/426778896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc10ad40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:44.808 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.572+0000 7f4dc3200700 1 -- 172.21.15.112:0/426778896 >> 172.21.15.112:0/426778896 conn(0x7f4dbc07dd20 msgr2=0x7f4dbc07e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.573+0000 7f4dc3200700 1 -- 172.21.15.112:0/426778896 shutdown_connections 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.573+0000 7f4dc3200700 1 -- 172.21.15.112:0/426778896 wait complete. 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.574+0000 7f4dc3200700 1 Processor -- start 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.574+0000 7f4dc3200700 1 -- start start 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.574+0000 7f4dc3200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.574+0000 7f4dc3200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc10eec0 con 0x7f4dbc10a920 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.574+0000 7f4dbbe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.574+0000 7f4dbbe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48424/0 (socket says 172.21.15.112:48424) 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.574+0000 7f4dbbe00700 1 -- 172.21.15.112:0/1946612399 learned_addr learned my addr 172.21.15.112:0/1946612399 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.575+0000 7f4dbbe00700 1 -- 172.21.15.112:0/1946612399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc19ec60 con 0x7f4dbc10a920 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.575+0000 7f4dbbe00700 1 --2- 172.21.15.112:0/1946612399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc19e720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4dac000c00 tx=0x7f4dac011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.575+0000 7f4db8c00700 1 -- 172.21.15.112:0/1946612399 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dac003aa0 con 0x7f4dbc10a920 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.575+0000 7f4db8c00700 1 -- 172.21.15.112:0/1946612399 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dac01b8f0 con 0x7f4dbc10a920 2024-04-25T18:34:44.809 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.575+0000 7f4dc3200700 1 -- 172.21.15.112:0/1946612399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dbc19eef0 con 0x7f4dbc10a920 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.575+0000 7f4dc3200700 1 -- 172.21.15.112:0/1946612399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dbc1a1be0 con 0x7f4dbc10a920 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.575+0000 7f4db8c00700 1 -- 172.21.15.112:0/1946612399 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dac023870 con 0x7f4dbc10a920 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.577+0000 7f4db8c00700 1 -- 172.21.15.112:0/1946612399 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4dac01b440 con 0x7f4dbc10a920 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.577+0000 7f4dc3200700 1 -- 172.21.15.112:0/1946612399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dbc0640a0 con 0x7f4dbc10a920 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.577+0000 7f4db8c00700 1 --2- 172.21.15.112:0/1946612399 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f4da0039600 0x7f4da003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.577+0000 7f4db8c00700 1 -- 172.21.15.112:0/1946612399 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4dac04def0 con 0x7f4dbc10a920 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.578+0000 7f4dbb400700 1 --2- 172.21.15.112:0/1946612399 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f4da0039600 0x7f4da003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.578+0000 7f4dbb400700 1 --2- 172.21.15.112:0/1946612399 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f4da0039600 0x7f4da003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4da8006fd0 tx=0x7f4da8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.583+0000 7f4db8c00700 1 -- 172.21.15.112:0/1946612399 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4dac01f030 con 0x7f4dbc10a920 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.808+0000 7f4dc3200700 1 -- 172.21.15.112:0/1946612399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f4dbc1a2240 con 0x7f4dbc10a920 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.811+0000 7f4db8c00700 1 -- 172.21.15.112:0/1946612399 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4dac01a3d0 con 0x7f4dbc10a920 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.815+0000 7f4dc3200700 1 -- 172.21.15.112:0/1946612399 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f4da0039600 msgr2=0x7f4da003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:44.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.815+0000 7f4dc3200700 1 --2- 172.21.15.112:0/1946612399 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f4da0039600 0x7f4da003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4da8006fd0 tx=0x7f4da8006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:44.811 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.815+0000 7f4dc3200700 1 -- 172.21.15.112:0/1946612399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 msgr2=0x7f4dbc19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:44.811 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.815+0000 7f4dc3200700 1 --2- 172.21.15.112:0/1946612399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc19e720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4dac000c00 tx=0x7f4dac011aa0 comp rx=0 tx=0).stop 2024-04-25T18:34:44.811 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.815+0000 7f4dc3200700 1 -- 172.21.15.112:0/1946612399 shutdown_connections 2024-04-25T18:34:44.811 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.815+0000 7f4dc3200700 1 --2- 172.21.15.112:0/1946612399 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f4da0039600 0x7f4da003bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:44.811 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.815+0000 7f4dc3200700 1 --2- 172.21.15.112:0/1946612399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dbc10a920 0x7f4dbc19e720 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:44.811 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.815+0000 7f4dc3200700 1 -- 172.21.15.112:0/1946612399 >> 172.21.15.112:0/1946612399 conn(0x7f4dbc07dd20 msgr2=0x7f4dbc108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:44.811 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.815+0000 7f4dc3200700 1 -- 172.21.15.112:0/1946612399 shutdown_connections 2024-04-25T18:34:44.811 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:43.815+0000 7f4dc3200700 1 -- 172.21.15.112:0/1946612399 wait complete. 2024-04-25T18:34:44.811 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling the dashboard module... 2024-04-25T18:34:45.175 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:44 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1946612399' entity='client.admin' 2024-04-25T18:34:46.074 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:45 smithi112 ceph-mon[27864]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:34:46.074 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:45 smithi112 ceph-mon[27864]: from='mgr.14120 172.21.15.112:0/1592212559' entity='mgr.x' 2024-04-25T18:34:46.074 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:45 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2275736397' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T18:34:46.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: ignoring --setuser ceph since I am not root 2024-04-25T18:34:46.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: ignoring --setgroup ceph since I am not root 2024-04-25T18:34:46.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:46.276+0000 7f6e7ceb7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:34:46.723 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:46.539+0000 7f6e7ceb7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:34:46.723 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:46.678+0000 7f6e7ceb7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:34:46.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.511+0000 7ff0bac00700 1 Processor -- start 2024-04-25T18:34:46.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.511+0000 7ff0bac00700 1 -- start start 2024-04-25T18:34:46.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.512+0000 7ff0bac00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b4108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:46.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.512+0000 7ff0bac00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b41090f0 con 0x7ff0b4108700 2024-04-25T18:34:46.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.512+0000 7ff0b3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b4108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:46.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.512+0000 7ff0b3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b4108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48430/0 (socket says 172.21.15.112:48430) 2024-04-25T18:34:46.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.512+0000 7ff0b3e00700 1 -- 172.21.15.112:0/2190042766 learned_addr learned my addr 172.21.15.112:0/2190042766 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.513+0000 7ff0b3e00700 1 -- 172.21.15.112:0/2190042766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b4109900 con 0x7ff0b4108700 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.513+0000 7ff0b3e00700 1 --2- 172.21.15.112:0/2190042766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b4108b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40098a0 tx=0x7ff0a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a09828644c8245 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.513+0000 7ff0b2a00700 1 -- 172.21.15.112:0/2190042766 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a4004160 con 0x7ff0b4108700 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.513+0000 7ff0b2a00700 1 -- 172.21.15.112:0/2190042766 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a4004300 con 0x7ff0b4108700 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.513+0000 7ff0b2a00700 1 -- 172.21.15.112:0/2190042766 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a4011a20 con 0x7ff0b4108700 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.514+0000 7ff0bac00700 1 -- 172.21.15.112:0/2190042766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 msgr2=0x7ff0b4108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.514+0000 7ff0bac00700 1 --2- 172.21.15.112:0/2190042766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b4108b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40098a0 tx=0x7ff0a4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.514+0000 7ff0bac00700 1 -- 172.21.15.112:0/2190042766 shutdown_connections 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.514+0000 7ff0bac00700 1 --2- 172.21.15.112:0/2190042766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b4108b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.514+0000 7ff0bac00700 1 -- 172.21.15.112:0/2190042766 >> 172.21.15.112:0/2190042766 conn(0x7ff0b4103ce0 msgr2=0x7ff0b4106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.514+0000 7ff0bac00700 1 -- 172.21.15.112:0/2190042766 shutdown_connections 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.514+0000 7ff0bac00700 1 -- 172.21.15.112:0/2190042766 wait complete. 2024-04-25T18:34:46.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.515+0000 7ff0bac00700 1 Processor -- start 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.515+0000 7ff0bac00700 1 -- start start 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.516+0000 7ff0bac00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b419c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.516+0000 7ff0bac00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b410ccb0 con 0x7ff0b4108700 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.516+0000 7ff0b3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b419c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.516+0000 7ff0b3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b419c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48434/0 (socket says 172.21.15.112:48434) 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.516+0000 7ff0b3e00700 1 -- 172.21.15.112:0/2275736397 learned_addr learned my addr 172.21.15.112:0/2275736397 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.516+0000 7ff0b3e00700 1 -- 172.21.15.112:0/2275736397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b419cab0 con 0x7ff0b4108700 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.517+0000 7ff0b3e00700 1 --2- 172.21.15.112:0/2275736397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b419c570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009520 tx=0x7ff0a4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.517+0000 7ff0b0c00700 1 -- 172.21.15.112:0/2275736397 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a40039e0 con 0x7ff0b4108700 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.517+0000 7ff0b0c00700 1 -- 172.21.15.112:0/2275736397 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a401b8f0 con 0x7ff0b4108700 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.517+0000 7ff0bac00700 1 -- 172.21.15.112:0/2275736397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b419cd40 con 0x7ff0b4108700 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.517+0000 7ff0b0c00700 1 -- 172.21.15.112:0/2275736397 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a4019770 con 0x7ff0b4108700 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.517+0000 7ff0bac00700 1 -- 172.21.15.112:0/2275736397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b419fa30 con 0x7ff0b4108700 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.519+0000 7ff0b0c00700 1 -- 172.21.15.112:0/2275736397 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff0a401b440 con 0x7ff0b4108700 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.519+0000 7ff0bac00700 1 -- 172.21.15.112:0/2275736397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b40640a0 con 0x7ff0b4108700 2024-04-25T18:34:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.519+0000 7ff0b0c00700 1 --2- 172.21.15.112:0/2275736397 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7ff0980395b0 0x7ff09803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.519+0000 7ff0b0c00700 1 -- 172.21.15.112:0/2275736397 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff0a404df60 con 0x7ff0b4108700 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.520+0000 7ff0b3400700 1 --2- 172.21.15.112:0/2275736397 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7ff0980395b0 0x7ff09803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.520+0000 7ff0b3400700 1 --2- 172.21.15.112:0/2275736397 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7ff0980395b0 0x7ff09803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0006fd0 tx=0x7ff0a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.525+0000 7ff0b0c00700 1 -- 172.21.15.112:0/2275736397 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0a401f030 con 0x7ff0b4108700 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.817+0000 7ff0bac00700 1 -- 172.21.15.112:0/2275736397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ff0b41a0020 con 0x7ff0b4108700 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.942+0000 7ff0b0c00700 1 -- 172.21.15.112:0/2275736397 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ff0a404d8b0 con 0x7ff0b4108700 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.946+0000 7ff0bac00700 1 -- 172.21.15.112:0/2275736397 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7ff0980395b0 msgr2=0x7ff09803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.946+0000 7ff0bac00700 1 --2- 172.21.15.112:0/2275736397 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7ff0980395b0 0x7ff09803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0006fd0 tx=0x7ff0a0006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.946+0000 7ff0bac00700 1 -- 172.21.15.112:0/2275736397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 msgr2=0x7ff0b419c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.947+0000 7ff0b0c00700 1 -- 172.21.15.112:0/2275736397 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ff0a401a3d0 con 0x7ff0b4108700 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.947+0000 7ff0bac00700 1 --2- 172.21.15.112:0/2275736397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b419c570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009520 tx=0x7ff0a4011aa0 comp rx=0 tx=0).stop 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.947+0000 7ff0bac00700 1 -- 172.21.15.112:0/2275736397 shutdown_connections 2024-04-25T18:34:46.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.947+0000 7ff0bac00700 1 --2- 172.21.15.112:0/2275736397 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7ff0980395b0 0x7ff09803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:46.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.947+0000 7ff0bac00700 1 --2- 172.21.15.112:0/2275736397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b4108700 0x7ff0b419c570 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009520 tx=0x7ff0a4011aa0 comp rx=0 tx=0).stop 2024-04-25T18:34:46.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.947+0000 7ff0bac00700 1 -- 172.21.15.112:0/2275736397 >> 172.21.15.112:0/2275736397 conn(0x7ff0b4103ce0 msgr2=0x7ff0b41047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:46.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.947+0000 7ff0bac00700 1 -- 172.21.15.112:0/2275736397 shutdown_connections 2024-04-25T18:34:46.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:45.948+0000 7ff0bac00700 1 -- 172.21.15.112:0/2275736397 wait complete. 2024-04-25T18:34:47.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:46 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2275736397' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T18:34:47.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:46 smithi112 ceph-mon[27864]: mgrmap e9: x(active, since 25s) 2024-04-25T18:34:47.424 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:47 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:47.093+0000 7f6e7ceb7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:34:47.424 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:47 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:47.214+0000 7f6e7ceb7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:34:47.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:47 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2026413271' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T18:34:48.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.488+0000 7f03b1a00700 1 Processor -- start 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.488+0000 7f03b1a00700 1 -- start start 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.489+0000 7f03b1a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac10b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.489+0000 7f03b1a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac063d20 con 0x7f03ac109150 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.489+0000 7f03aaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac10b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.489+0000 7f03aaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac10b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48452/0 (socket says 172.21.15.112:48452) 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.489+0000 7f03aaa00700 1 -- 172.21.15.112:0/1169123875 learned_addr learned my addr 172.21.15.112:0/1169123875 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.490+0000 7f03aaa00700 1 -- 172.21.15.112:0/1169123875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac10ba80 con 0x7f03ac109150 2024-04-25T18:34:48.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.490+0000 7f03aaa00700 1 --2- 172.21.15.112:0/1169123875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac10b540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f039c009e80 tx=0x7f039c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d64696580f7204b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.490+0000 7f03a9600700 1 -- 172.21.15.112:0/1169123875 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f039c017070 con 0x7f03ac109150 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.490+0000 7f03a9600700 1 -- 172.21.15.112:0/1169123875 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039c00b8c0 con 0x7f03ac109150 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.491+0000 7f03a9600700 1 -- 172.21.15.112:0/1169123875 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f039c004490 con 0x7f03ac109150 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.491+0000 7f03b1a00700 1 -- 172.21.15.112:0/1169123875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 msgr2=0x7f03ac10b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.491+0000 7f03b1a00700 1 --2- 172.21.15.112:0/1169123875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac10b540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f039c009e80 tx=0x7f039c00b330 comp rx=0 tx=0).stop 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.491+0000 7f03b1a00700 1 -- 172.21.15.112:0/1169123875 shutdown_connections 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.491+0000 7f03b1a00700 1 --2- 172.21.15.112:0/1169123875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac10b540 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.491+0000 7f03b1a00700 1 -- 172.21.15.112:0/1169123875 >> 172.21.15.112:0/1169123875 conn(0x7f03ac102ba0 msgr2=0x7f03ac104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.492+0000 7f03b1a00700 1 -- 172.21.15.112:0/1169123875 shutdown_connections 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.492+0000 7f03b1a00700 1 -- 172.21.15.112:0/1169123875 wait complete. 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.493+0000 7f03b1a00700 1 Processor -- start 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.493+0000 7f03b1a00700 1 -- start start 2024-04-25T18:34:48.604 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.493+0000 7f03b1a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.493+0000 7f03b1a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10ca80 con 0x7f03ac109150 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.494+0000 7f03aaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.494+0000 7f03aaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48468/0 (socket says 172.21.15.112:48468) 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.494+0000 7f03aaa00700 1 -- 172.21.15.112:0/2026413271 learned_addr learned my addr 172.21.15.112:0/2026413271 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.494+0000 7f03aaa00700 1 -- 172.21.15.112:0/2026413271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac19ec30 con 0x7f03ac109150 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.494+0000 7f03aaa00700 1 --2- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac19e6f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f039c00b7b0 tx=0x7f039c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.495+0000 7f03a3400700 1 -- 172.21.15.112:0/2026413271 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f039c020020 con 0x7f03ac109150 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.495+0000 7f03a3400700 1 -- 172.21.15.112:0/2026413271 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039c003c60 con 0x7f03ac109150 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.495+0000 7f03b1a00700 1 -- 172.21.15.112:0/2026413271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac19eec0 con 0x7f03ac109150 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.495+0000 7f03a3400700 1 -- 172.21.15.112:0/2026413271 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f039c017040 con 0x7f03ac109150 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.495+0000 7f03b1a00700 1 -- 172.21.15.112:0/2026413271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac1a1bb0 con 0x7f03ac109150 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.496+0000 7f03b1a00700 1 -- 172.21.15.112:0/2026413271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac0640a0 con 0x7f03ac109150 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.496+0000 7f03a3400700 1 -- 172.21.15.112:0/2026413271 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f039c023020 con 0x7f03ac109150 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.496+0000 7f03a3400700 1 --2- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0390039660 0x7f039003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.496+0000 7f03a3400700 1 -- 172.21.15.112:0/2026413271 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f039c00f050 con 0x7f03ac109150 2024-04-25T18:34:48.605 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.496+0000 7f03aa000700 1 -- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0390039660 msgr2=0x7f039003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/733062511 2024-04-25T18:34:48.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.497+0000 7f03aa000700 1 --2- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0390039660 0x7f039003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:34:48.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.501+0000 7f03a3400700 1 -- 172.21.15.112:0/2026413271 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f039c02a020 con 0x7f03ac109150 2024-04-25T18:34:48.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.698+0000 7f03aa000700 1 -- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0390039660 msgr2=0x7f039003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/733062511 2024-04-25T18:34:48.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.698+0000 7f03aa000700 1 --2- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0390039660 0x7f039003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:34:48.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.823+0000 7f03b1a00700 1 -- 172.21.15.112:0/2026413271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f03ac1a2180 con 0x7f03ac109150 2024-04-25T18:34:48.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.823+0000 7f03a3400700 1 -- 172.21.15.112:0/2026413271 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f039c02a200 con 0x7f03ac109150 2024-04-25T18:34:48.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.825+0000 7f03b1a00700 1 -- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0390039660 msgr2=0x7f039003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:34:48.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.825+0000 7f03b1a00700 1 --2- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0390039660 0x7f039003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:48.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.825+0000 7f03b1a00700 1 -- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 msgr2=0x7f03ac19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:48.606 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.825+0000 7f03b1a00700 1 --2- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac19e6f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f039c00b7b0 tx=0x7f039c00bd50 comp rx=0 tx=0).stop 2024-04-25T18:34:48.607 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.826+0000 7f03b1a00700 1 -- 172.21.15.112:0/2026413271 shutdown_connections 2024-04-25T18:34:48.607 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.826+0000 7f03b1a00700 1 --2- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0390039660 0x7f039003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:48.607 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.826+0000 7f03b1a00700 1 --2- 172.21.15.112:0/2026413271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03ac109150 0x7f03ac19e6f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:48.607 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.826+0000 7f03b1a00700 1 -- 172.21.15.112:0/2026413271 >> 172.21.15.112:0/2026413271 conn(0x7f03ac102ba0 msgr2=0x7f03ac1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:48.607 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.826+0000 7f03b1a00700 1 -- 172.21.15.112:0/2026413271 shutdown_connections 2024-04-25T18:34:48.607 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:47.826+0000 7f03b1a00700 1 -- 172.21.15.112:0/2026413271 wait complete. 2024-04-25T18:34:48.607 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-04-25T18:34:48.607 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 9... 2024-04-25T18:34:48.962 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:48.771+0000 7f6e7ceb7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:34:49.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:49.005+0000 7f6e7ceb7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:34:49.609 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:49.227+0000 7f6e7ceb7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:34:49.609 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:49.608+0000 7f6e7ceb7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:34:49.973 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:49.729+0000 7f6e7ceb7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:34:50.236 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:49.980+0000 7f6e7ceb7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:34:50.236 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:50.235+0000 7f6e7ceb7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:34:51.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:51 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:51.011+0000 7f6e7ceb7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:34:51.952 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:51 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:51.558+0000 7f6e7ceb7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:34:51.952 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:51 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:51.684+0000 7f6e7ceb7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:34:52.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:51 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:51.949+0000 7f6e7ceb7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:34:52.973 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:52 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:52.606+0000 7f6e7ceb7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:34:52.973 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:52 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:52.729+0000 7f6e7ceb7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:34:53.359 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:53 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:53.049+0000 7f6e7ceb7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:34:53.359 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:53 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:53.228+0000 7f6e7ceb7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:34:53.359 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:53 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:53.358+0000 7f6e7ceb7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:34:53.723 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:53 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:53.509+0000 7f6e7ceb7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:34:54.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:53 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:53.886+0000 7f6e7ceb7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:34:54.946 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:54 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:54.649+0000 7f6e7ceb7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:34:55.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:34:54 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:34:54.945+0000 7f6e7ceb7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:34:55.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:55 smithi112 ceph-mon[27864]: Active manager daemon x restarted 2024-04-25T18:34:55.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:55 smithi112 ceph-mon[27864]: Activating manager daemon x 2024-04-25T18:34:55.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:55 smithi112 ceph-mon[27864]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T18:34:55.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:55 smithi112 ceph-mon[27864]: mgrmap e10: x(active, starting, since 0.00904528s) 2024-04-25T18:34:55.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:34:55.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:34:55.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:34:55.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:34:55.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:34:55.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:55 smithi112 ceph-mon[27864]: Manager daemon x is now available 2024-04-25T18:34:56.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:34:56.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:34:56.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:34:56.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:56 smithi112 ceph-mon[27864]: mgrmap e11: x(active, since 1.00983s) 2024-04-25T18:34:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:34:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T18:34:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T18:34:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:34:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.243+0000 7f0c11200700 1 Processor -- start 2024-04-25T18:34:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.244+0000 7f0c11200700 1 -- start start 2024-04-25T18:34:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.244+0000 7f0c11200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c108b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.244+0000 7f0c11200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c1090d0 con 0x7f0c0c1086e0 2024-04-25T18:34:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.244+0000 7f0c09e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c108b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:56.718 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.244+0000 7f0c09e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c108b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48478/0 (socket says 172.21.15.112:48478) 2024-04-25T18:34:56.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.244+0000 7f0c09e00700 1 -- 172.21.15.112:0/538159822 learned_addr learned my addr 172.21.15.112:0/538159822 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:56.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.246+0000 7f0c09e00700 1 -- 172.21.15.112:0/538159822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c1098e0 con 0x7f0c0c1086e0 2024-04-25T18:34:56.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.246+0000 7f0c09e00700 1 --2- 172.21.15.112:0/538159822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c108b00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0098a0 tx=0x7f0bfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81804b5c15decc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:56.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.246+0000 7f0c08a00700 1 -- 172.21.15.112:0/538159822 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bfc004160 con 0x7f0c0c1086e0 2024-04-25T18:34:56.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.246+0000 7f0c08a00700 1 -- 172.21.15.112:0/538159822 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc004300 con 0x7f0c0c1086e0 2024-04-25T18:34:56.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.246+0000 7f0c08a00700 1 -- 172.21.15.112:0/538159822 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bfc011a20 con 0x7f0c0c1086e0 2024-04-25T18:34:56.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.247+0000 7f0c11200700 1 -- 172.21.15.112:0/538159822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 msgr2=0x7f0c0c108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:56.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.247+0000 7f0c11200700 1 --2- 172.21.15.112:0/538159822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c108b00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0098a0 tx=0x7f0bfc009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:56.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.247+0000 7f0c11200700 1 -- 172.21.15.112:0/538159822 shutdown_connections 2024-04-25T18:34:56.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.247+0000 7f0c11200700 1 --2- 172.21.15.112:0/538159822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c108b00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:56.719 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.247+0000 7f0c11200700 1 -- 172.21.15.112:0/538159822 >> 172.21.15.112:0/538159822 conn(0x7f0c0c103ca0 msgr2=0x7f0c0c1060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:56.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.247+0000 7f0c11200700 1 -- 172.21.15.112:0/538159822 shutdown_connections 2024-04-25T18:34:56.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.247+0000 7f0c11200700 1 -- 172.21.15.112:0/538159822 wait complete. 2024-04-25T18:34:56.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.248+0000 7f0c11200700 1 Processor -- start 2024-04-25T18:34:56.720 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.248+0000 7f0c11200700 1 -- start start 2024-04-25T18:34:56.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.249+0000 7f0c11200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:56.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.249+0000 7f0c11200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c10cc90 con 0x7f0c0c1086e0 2024-04-25T18:34:56.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.249+0000 7f0c09e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:56.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.249+0000 7f0c09e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48480/0 (socket says 172.21.15.112:48480) 2024-04-25T18:34:56.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.249+0000 7f0c09e00700 1 -- 172.21.15.112:0/1261926365 learned_addr learned my addr 172.21.15.112:0/1261926365 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:56.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.249+0000 7f0c09e00700 1 -- 172.21.15.112:0/1261926365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c19eca0 con 0x7f0c0c1086e0 2024-04-25T18:34:56.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.250+0000 7f0c09e00700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c19e760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009520 tx=0x7f0bfc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:56.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.250+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bfc0039e0 con 0x7f0c0c1086e0 2024-04-25T18:34:56.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.250+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bfc01b8f0 con 0x7f0c0c1086e0 2024-04-25T18:34:56.722 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.250+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bfc019770 con 0x7f0c0c1086e0 2024-04-25T18:34:56.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.250+0000 7f0c11200700 1 -- 172.21.15.112:0/1261926365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0c19ef30 con 0x7f0c0c1086e0 2024-04-25T18:34:56.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.250+0000 7f0c11200700 1 -- 172.21.15.112:0/1261926365 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0c1a1c20 con 0x7f0c0c1086e0 2024-04-25T18:34:56.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.251+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f0bfc01b440 con 0x7f0c0c1086e0 2024-04-25T18:34:56.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.251+0000 7f0c02a00700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 0x7f0bf003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:56.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.251+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 --> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0bf003c220 con 0x7f0bf0039650 2024-04-25T18:34:56.723 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.251+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0bfc04df90 con 0x7f0c0c1086e0 2024-04-25T18:34:56.724 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.251+0000 7f0c09400700 1 -- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 msgr2=0x7f0bf003bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/733062511 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.251+0000 7f0c09400700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 0x7f0bf003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.452+0000 7f0c09400700 1 -- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 msgr2=0x7f0bf003bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/733062511 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.452+0000 7f0c09400700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 0x7f0bf003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.853+0000 7f0c09400700 1 -- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 msgr2=0x7f0bf003bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/733062511 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:49.853+0000 7f0c09400700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 0x7f0bf003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:50.654+0000 7f0c09400700 1 -- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 msgr2=0x7f0bf003bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/733062511 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:50.654+0000 7f0c09400700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 0x7f0bf003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:52.256+0000 7f0c09400700 1 -- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 msgr2=0x7f0bf003bb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/733062511 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:52.256+0000 7f0c09400700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 0x7f0bf003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:54.954+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f0bfc01a3d0 con 0x7f0c0c1086e0 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:54.954+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 msgr2=0x7f0bf003bb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:34:56.725 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:54.955+0000 7f0c02a00700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 0x7f0bf003bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:56.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.955+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f0bfc04faf0 con 0x7f0c0c1086e0 2024-04-25T18:34:56.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.955+0000 7f0c02a00700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f0bf003cfe0 0x7f0bf003f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:56.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.955+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0bf003c220 con 0x7f0bf003cfe0 2024-04-25T18:34:56.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.956+0000 7f0c09400700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f0bf003cfe0 0x7f0bf003f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:56.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.957+0000 7f0c09400700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f0bf003cfe0 0x7f0bf003f3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8003a10 tx=0x7f0bf8009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:56.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.958+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f0bf003c220 con 0x7f0bf003cfe0 2024-04-25T18:34:56.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.968+0000 7f0c11200700 1 -- 172.21.15.112:0/1261926365 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0c0c0640a0 con 0x7f0bf003cfe0 2024-04-25T18:34:56.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c02a00700 1 -- 172.21.15.112:0/1261926365 <== mgr.14152 v2:172.21.15.112:6800/605196912 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f0c0c0640a0 con 0x7f0bf003cfe0 2024-04-25T18:34:56.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c11200700 1 -- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f0bf003cfe0 msgr2=0x7f0bf003f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:56.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c11200700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f0bf003cfe0 0x7f0bf003f3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8003a10 tx=0x7f0bf8009270 comp rx=0 tx=0).stop 2024-04-25T18:34:56.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c11200700 1 -- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 msgr2=0x7f0c0c19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:56.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c11200700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c19e760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009520 tx=0x7f0bfc011aa0 comp rx=0 tx=0).stop 2024-04-25T18:34:56.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c11200700 1 -- 172.21.15.112:0/1261926365 shutdown_connections 2024-04-25T18:34:56.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c11200700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f0bf003cfe0 0x7f0bf003f3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:56.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c11200700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c1086e0 0x7f0c0c19e760 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:56.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c11200700 1 --2- 172.21.15.112:0/1261926365 >> [v2:172.21.15.112:6800/733062511,v1:172.21.15.112:6801/733062511] conn(0x7f0bf0039650 0x7f0bf003bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:56.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c11200700 1 -- 172.21.15.112:0/1261926365 >> 172.21.15.112:0/1261926365 conn(0x7f0c0c103ca0 msgr2=0x7f0c0c104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:56.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c11200700 1 -- 172.21.15.112:0/1261926365 shutdown_connections 2024-04-25T18:34:56.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:55.969+0000 7f0c11200700 1 -- 172.21.15.112:0/1261926365 wait complete. 2024-04-25T18:34:56.729 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 9 is available 2024-04-25T18:34:56.729 INFO:teuthology.orchestra.run.smithi112.stdout:Generating a dashboard self-signed certificate... 2024-04-25T18:34:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:57 smithi112 ceph-mon[27864]: [25/Apr/2024:18:34:55] ENGINE Bus STARTING 2024-04-25T18:34:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:57 smithi112 ceph-mon[27864]: [25/Apr/2024:18:34:55] ENGINE Serving on http://172.21.15.112:8765 2024-04-25T18:34:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:57 smithi112 ceph-mon[27864]: [25/Apr/2024:18:34:55] ENGINE Serving on https://172.21.15.112:7150 2024-04-25T18:34:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:57 smithi112 ceph-mon[27864]: [25/Apr/2024:18:34:55] ENGINE Bus STARTED 2024-04-25T18:34:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:57 smithi112 ceph-mon[27864]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T18:34:57.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:57 smithi112 ceph-mon[27864]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T18:34:58.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:58 smithi112 ceph-mon[27864]: mgrmap e12: x(active, since 2s) 2024-04-25T18:34:58.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:58 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:34:59.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T18:34:59.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.121+0000 7f82f8400700 1 Processor -- start 2024-04-25T18:34:59.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.121+0000 7f82f8400700 1 -- start start 2024-04-25T18:34:59.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.121+0000 7f82f8400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f007c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:59.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.121+0000 7f82f8400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82f007c7d0 con 0x7f82f007dda0 2024-04-25T18:34:59.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.122+0000 7f82f5a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f007c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:59.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.122+0000 7f82f5a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f007c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54072/0 (socket says 172.21.15.112:54072) 2024-04-25T18:34:59.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.122+0000 7f82f5a00700 1 -- 172.21.15.112:0/966780690 learned_addr learned my addr 172.21.15.112:0/966780690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:59.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.122+0000 7f82f5a00700 1 -- 172.21.15.112:0/966780690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f007c950 con 0x7f82f007dda0 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.123+0000 7f82f5a00700 1 --2- 172.21.15.112:0/966780690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f007c200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f82e40098a0 tx=0x7f82e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0083472553e892 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.123+0000 7f82efe00700 1 -- 172.21.15.112:0/966780690 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82e4004160 con 0x7f82f007dda0 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.123+0000 7f82efe00700 1 -- 172.21.15.112:0/966780690 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e4004300 con 0x7f82f007dda0 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.123+0000 7f82efe00700 1 -- 172.21.15.112:0/966780690 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82e4011a20 con 0x7f82f007dda0 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.124+0000 7f82f8400700 1 -- 172.21.15.112:0/966780690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 msgr2=0x7f82f007c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.124+0000 7f82f8400700 1 --2- 172.21.15.112:0/966780690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f007c200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f82e40098a0 tx=0x7f82e4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.124+0000 7f82f8400700 1 -- 172.21.15.112:0/966780690 shutdown_connections 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.124+0000 7f82f8400700 1 --2- 172.21.15.112:0/966780690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f007c200 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.124+0000 7f82f8400700 1 -- 172.21.15.112:0/966780690 >> 172.21.15.112:0/966780690 conn(0x7f82f0103c40 msgr2=0x7f82f01060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.124+0000 7f82f8400700 1 -- 172.21.15.112:0/966780690 shutdown_connections 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.124+0000 7f82f8400700 1 -- 172.21.15.112:0/966780690 wait complete. 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.125+0000 7f82f8400700 1 Processor -- start 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.125+0000 7f82f8400700 1 -- start start 2024-04-25T18:34:59.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.125+0000 7f82f8400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.125+0000 7f82f8400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82f010ecc0 con 0x7f82f007dda0 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.126+0000 7f82f5a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.126+0000 7f82f5a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f019e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54084/0 (socket says 172.21.15.112:54084) 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.126+0000 7f82f5a00700 1 -- 172.21.15.112:0/484804323 learned_addr learned my addr 172.21.15.112:0/484804323 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.126+0000 7f82f5a00700 1 -- 172.21.15.112:0/484804323 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f019ec70 con 0x7f82f007dda0 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.126+0000 7f82f5a00700 1 --2- 172.21.15.112:0/484804323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f019e730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82e40119f0 tx=0x7f82e4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.127+0000 7f82ee000700 1 -- 172.21.15.112:0/484804323 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82e40039e0 con 0x7f82f007dda0 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.127+0000 7f82ee000700 1 -- 172.21.15.112:0/484804323 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82e401b8f0 con 0x7f82f007dda0 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.127+0000 7f82ee000700 1 -- 172.21.15.112:0/484804323 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82e40237a0 con 0x7f82f007dda0 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.127+0000 7f82f8400700 1 -- 172.21.15.112:0/484804323 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f019ef00 con 0x7f82f007dda0 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.127+0000 7f82f8400700 1 -- 172.21.15.112:0/484804323 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f01a1bf0 con 0x7f82f007dda0 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.128+0000 7f82ee000700 1 -- 172.21.15.112:0/484804323 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f82e401b440 con 0x7f82f007dda0 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.128+0000 7f82f8400700 1 -- 172.21.15.112:0/484804323 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f00640a0 con 0x7f82f007dda0 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.129+0000 7f82ee000700 1 --2- 172.21.15.112:0/484804323 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f82d8039610 0x7f82d803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:34:59.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.129+0000 7f82ee000700 1 -- 172.21.15.112:0/484804323 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f82e404e060 con 0x7f82f007dda0 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.129+0000 7f82f5000700 1 --2- 172.21.15.112:0/484804323 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f82d8039610 0x7f82d803bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.130+0000 7f82f5000700 1 --2- 172.21.15.112:0/484804323 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f82d8039610 0x7f82d803bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f82dc006fd0 tx=0x7f82dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.135+0000 7f82ee000700 1 -- 172.21.15.112:0/484804323 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82e401f030 con 0x7f82f007dda0 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.368+0000 7f82f8400700 1 -- 172.21.15.112:0/484804323 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f82f0104e60 con 0x7f82d8039610 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.510+0000 7f82ee000700 1 -- 172.21.15.112:0/484804323 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f82f0104e60 con 0x7f82d8039610 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.514+0000 7f82f8400700 1 -- 172.21.15.112:0/484804323 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f82d8039610 msgr2=0x7f82d803bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.514+0000 7f82f8400700 1 --2- 172.21.15.112:0/484804323 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f82d8039610 0x7f82d803bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f82dc006fd0 tx=0x7f82dc006e40 comp rx=0 tx=0).stop 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.515+0000 7f82f8400700 1 -- 172.21.15.112:0/484804323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 msgr2=0x7f82f019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.515+0000 7f82f8400700 1 --2- 172.21.15.112:0/484804323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f019e730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82e40119f0 tx=0x7f82e4011ad0 comp rx=0 tx=0).stop 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.515+0000 7f82f8400700 1 -- 172.21.15.112:0/484804323 shutdown_connections 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.515+0000 7f82f8400700 1 --2- 172.21.15.112:0/484804323 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f82d8039610 0x7f82d803bad0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.515+0000 7f82f8400700 1 --2- 172.21.15.112:0/484804323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82f007dda0 0x7f82f019e730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.515+0000 7f82f8400700 1 -- 172.21.15.112:0/484804323 >> 172.21.15.112:0/484804323 conn(0x7f82f0103c40 msgr2=0x7f82f0104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.515+0000 7f82f8400700 1 -- 172.21.15.112:0/484804323 shutdown_connections 2024-04-25T18:34:59.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:58.515+0000 7f82f8400700 1 -- 172.21.15.112:0/484804323 wait complete. 2024-04-25T18:34:59.290 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial admin user... 2024-04-25T18:34:59.623 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:59 smithi112 ceph-mon[27864]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:34:59.623 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:59 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:34:59.623 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:34:59 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:01.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:01.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8qxF91pv4DsQgtR13cEaJufvuKjKhjVvgIk26Duoo.38awXn67oJS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714070100, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T18:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.929+0000 7f41d4000700 1 Processor -- start 2024-04-25T18:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.929+0000 7f41d4000700 1 -- start start 2024-04-25T18:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.930+0000 7f41d4000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.930+0000 7f41d4000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1090b0 con 0x7f41cc1086c0 2024-04-25T18:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.930+0000 7f41d1600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:01.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.931+0000 7f41d1600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc108ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54824/0 (socket says 172.21.15.112:54824) 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.931+0000 7f41d1600700 1 -- 172.21.15.112:0/378291788 learned_addr learned my addr 172.21.15.112:0/378291788 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.931+0000 7f41d1600700 1 -- 172.21.15.112:0/378291788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc1098c0 con 0x7f41cc1086c0 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.931+0000 7f41d1600700 1 --2- 172.21.15.112:0/378291788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc108ae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41c00098a0 tx=0x7f41c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10d7d6d7518c17d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.932+0000 7f41cbe00700 1 -- 172.21.15.112:0/378291788 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41c0004160 con 0x7f41cc1086c0 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.932+0000 7f41cbe00700 1 -- 172.21.15.112:0/378291788 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41c0004300 con 0x7f41cc1086c0 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.932+0000 7f41cbe00700 1 -- 172.21.15.112:0/378291788 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41c0011a20 con 0x7f41cc1086c0 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.933+0000 7f41d4000700 1 -- 172.21.15.112:0/378291788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 msgr2=0x7f41cc108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.933+0000 7f41d4000700 1 --2- 172.21.15.112:0/378291788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc108ae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41c00098a0 tx=0x7f41c0009bb0 comp rx=0 tx=0).stop 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.933+0000 7f41d4000700 1 -- 172.21.15.112:0/378291788 shutdown_connections 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.933+0000 7f41d4000700 1 --2- 172.21.15.112:0/378291788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc108ae0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.933+0000 7f41d4000700 1 -- 172.21.15.112:0/378291788 >> 172.21.15.112:0/378291788 conn(0x7f41cc103c40 msgr2=0x7f41cc1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.933+0000 7f41d4000700 1 -- 172.21.15.112:0/378291788 shutdown_connections 2024-04-25T18:35:01.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.933+0000 7f41d4000700 1 -- 172.21.15.112:0/378291788 wait complete. 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.934+0000 7f41d4000700 1 Processor -- start 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.935+0000 7f41d4000700 1 -- start start 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.935+0000 7f41d4000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.935+0000 7f41d4000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc10cc70 con 0x7f41cc1086c0 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.935+0000 7f41d1600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.935+0000 7f41d1600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54836/0 (socket says 172.21.15.112:54836) 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.935+0000 7f41d1600700 1 -- 172.21.15.112:0/3339395714 learned_addr learned my addr 172.21.15.112:0/3339395714 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.936+0000 7f41d1600700 1 -- 172.21.15.112:0/3339395714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc19ca50 con 0x7f41cc1086c0 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.936+0000 7f41d1600700 1 --2- 172.21.15.112:0/3339395714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc19c510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41c00119f0 tx=0x7f41c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.936+0000 7f41ca000700 1 -- 172.21.15.112:0/3339395714 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41c00039e0 con 0x7f41cc1086c0 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.936+0000 7f41ca000700 1 -- 172.21.15.112:0/3339395714 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41c001b8f0 con 0x7f41cc1086c0 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.936+0000 7f41ca000700 1 -- 172.21.15.112:0/3339395714 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41c00237a0 con 0x7f41cc1086c0 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.936+0000 7f41d4000700 1 -- 172.21.15.112:0/3339395714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41cc19cce0 con 0x7f41cc1086c0 2024-04-25T18:35:01.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.937+0000 7f41d4000700 1 -- 172.21.15.112:0/3339395714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41cc19f9d0 con 0x7f41cc1086c0 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.938+0000 7f41ca000700 1 -- 172.21.15.112:0/3339395714 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f41c001b440 con 0x7f41cc1086c0 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.938+0000 7f41ca000700 1 --2- 172.21.15.112:0/3339395714 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f41b4039610 0x7f41b403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.938+0000 7f41ca000700 1 -- 172.21.15.112:0/3339395714 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f41c004df60 con 0x7f41cc1086c0 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.938+0000 7f41d4000700 1 -- 172.21.15.112:0/3339395714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41cc0640a0 con 0x7f41cc1086c0 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.939+0000 7f41d0c00700 1 --2- 172.21.15.112:0/3339395714 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f41b4039610 0x7f41b403bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.940+0000 7f41d0c00700 1 --2- 172.21.15.112:0/3339395714 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f41b4039610 0x7f41b403bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006fd0 tx=0x7f41b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:34:59.945+0000 7f41ca000700 1 -- 172.21.15.112:0/3339395714 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41c001f030 con 0x7f41cc1086c0 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.197+0000 7f41d4000700 1 -- 172.21.15.112:0/3339395714 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f41cc19fba0 con 0x7f41b4039610 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.455+0000 7f41ca000700 1 -- 172.21.15.112:0/3339395714 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f41cc19fba0 con 0x7f41b4039610 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.458+0000 7f41d4000700 1 -- 172.21.15.112:0/3339395714 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f41b4039610 msgr2=0x7f41b403bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.458+0000 7f41d4000700 1 --2- 172.21.15.112:0/3339395714 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f41b4039610 0x7f41b403bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006fd0 tx=0x7f41b8006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.458+0000 7f41d4000700 1 -- 172.21.15.112:0/3339395714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 msgr2=0x7f41cc19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.458+0000 7f41d4000700 1 --2- 172.21.15.112:0/3339395714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc19c510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41c00119f0 tx=0x7f41c0011ad0 comp rx=0 tx=0).stop 2024-04-25T18:35:01.291 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.459+0000 7f41d4000700 1 -- 172.21.15.112:0/3339395714 shutdown_connections 2024-04-25T18:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.459+0000 7f41d4000700 1 --2- 172.21.15.112:0/3339395714 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f41b4039610 0x7f41b403bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.459+0000 7f41d4000700 1 --2- 172.21.15.112:0/3339395714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41cc1086c0 0x7f41cc19c510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.459+0000 7f41d4000700 1 -- 172.21.15.112:0/3339395714 >> 172.21.15.112:0/3339395714 conn(0x7f41cc103c40 msgr2=0x7f41cc104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.459+0000 7f41d4000700 1 -- 172.21.15.112:0/3339395714 shutdown_connections 2024-04-25T18:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:00.459+0000 7f41d4000700 1 -- 172.21.15.112:0/3339395714 wait complete. 2024-04-25T18:35:01.292 INFO:teuthology.orchestra.run.smithi112.stdout:Fetching dashboard port number... 2024-04-25T18:35:02.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:02 smithi112 ceph-mon[27864]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:35:02.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:02 smithi112 ceph-mon[27864]: mgrmap e13: x(active, since 6s) 2024-04-25T18:35:03.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:03 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3025472753' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T18:35:03.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T18:35:03.904 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.942+0000 7fa713200700 1 Processor -- start 2024-04-25T18:35:03.904 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.942+0000 7fa713200700 1 -- start start 2024-04-25T18:35:03.904 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.942+0000 7fa713200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:03.904 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.942+0000 7fa713200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c1090b0 con 0x7fa70c1086c0 2024-04-25T18:35:03.904 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.943+0000 7fa70be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:03.904 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.943+0000 7fa70be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c108ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54852/0 (socket says 172.21.15.112:54852) 2024-04-25T18:35:03.904 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.943+0000 7fa70be00700 1 -- 172.21.15.112:0/3651062680 learned_addr learned my addr 172.21.15.112:0/3651062680 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:03.904 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.943+0000 7fa70be00700 1 -- 172.21.15.112:0/3651062680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70c1098c0 con 0x7fa70c1086c0 2024-04-25T18:35:03.904 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.943+0000 7fa70be00700 1 --2- 172.21.15.112:0/3651062680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c108ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0098a0 tx=0x7fa6fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4883211c0fda3712 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:03.904 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.944+0000 7fa70aa00700 1 -- 172.21.15.112:0/3651062680 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6fc004160 con 0x7fa70c1086c0 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.944+0000 7fa70aa00700 1 -- 172.21.15.112:0/3651062680 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6fc004300 con 0x7fa70c1086c0 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.944+0000 7fa70aa00700 1 -- 172.21.15.112:0/3651062680 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6fc011a20 con 0x7fa70c1086c0 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.944+0000 7fa713200700 1 -- 172.21.15.112:0/3651062680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 msgr2=0x7fa70c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.944+0000 7fa713200700 1 --2- 172.21.15.112:0/3651062680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c108ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0098a0 tx=0x7fa6fc009bb0 comp rx=0 tx=0).stop 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.945+0000 7fa713200700 1 -- 172.21.15.112:0/3651062680 shutdown_connections 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.945+0000 7fa713200700 1 --2- 172.21.15.112:0/3651062680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c108ae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.945+0000 7fa713200700 1 -- 172.21.15.112:0/3651062680 >> 172.21.15.112:0/3651062680 conn(0x7fa70c103c40 msgr2=0x7fa70c1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.945+0000 7fa713200700 1 -- 172.21.15.112:0/3651062680 shutdown_connections 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.945+0000 7fa713200700 1 -- 172.21.15.112:0/3651062680 wait complete. 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.946+0000 7fa713200700 1 Processor -- start 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.946+0000 7fa713200700 1 -- start start 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.946+0000 7fa713200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.946+0000 7fa713200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c10cc70 con 0x7fa70c1086c0 2024-04-25T18:35:03.905 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.947+0000 7fa70be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.947+0000 7fa70be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54862/0 (socket says 172.21.15.112:54862) 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.947+0000 7fa70be00700 1 -- 172.21.15.112:0/3025472753 learned_addr learned my addr 172.21.15.112:0/3025472753 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.947+0000 7fa70be00700 1 -- 172.21.15.112:0/3025472753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70c19ca50 con 0x7fa70c1086c0 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.947+0000 7fa70be00700 1 --2- 172.21.15.112:0/3025472753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c19c510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0119f0 tx=0x7fa6fc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.948+0000 7fa708c00700 1 -- 172.21.15.112:0/3025472753 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6fc0039e0 con 0x7fa70c1086c0 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.948+0000 7fa708c00700 1 -- 172.21.15.112:0/3025472753 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6fc01b8f0 con 0x7fa70c1086c0 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.948+0000 7fa708c00700 1 -- 172.21.15.112:0/3025472753 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6fc0237a0 con 0x7fa70c1086c0 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.948+0000 7fa713200700 1 -- 172.21.15.112:0/3025472753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70c19cce0 con 0x7fa70c1086c0 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.948+0000 7fa713200700 1 -- 172.21.15.112:0/3025472753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa70c19f9d0 con 0x7fa70c1086c0 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.949+0000 7fa713200700 1 -- 172.21.15.112:0/3025472753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa70c0640a0 con 0x7fa70c1086c0 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.949+0000 7fa708c00700 1 -- 172.21.15.112:0/3025472753 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fa6fc01b440 con 0x7fa70c1086c0 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.950+0000 7fa708c00700 1 --2- 172.21.15.112:0/3025472753 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6f00396b0 0x7fa6f003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.950+0000 7fa708c00700 1 -- 172.21.15.112:0/3025472753 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa6fc04e270 con 0x7fa70c1086c0 2024-04-25T18:35:03.906 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.950+0000 7fa70b400700 1 --2- 172.21.15.112:0/3025472753 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6f00396b0 0x7fa6f003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.951+0000 7fa70b400700 1 --2- 172.21.15.112:0/3025472753 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6f00396b0 0x7fa6f003bb70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8006fd0 tx=0x7fa6f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:01.955+0000 7fa708c00700 1 -- 172.21.15.112:0/3025472753 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6fc01f030 con 0x7fa70c1086c0 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.185+0000 7fa713200700 1 -- 172.21.15.112:0/3025472753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa70c19fba0 con 0x7fa70c1086c0 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.186+0000 7fa708c00700 1 -- 172.21.15.112:0/3025472753 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa6fc0187b0 con 0x7fa70c1086c0 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.189+0000 7fa713200700 1 -- 172.21.15.112:0/3025472753 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6f00396b0 msgr2=0x7fa6f003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.189+0000 7fa713200700 1 --2- 172.21.15.112:0/3025472753 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6f00396b0 0x7fa6f003bb70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8006fd0 tx=0x7fa6f8006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.189+0000 7fa713200700 1 -- 172.21.15.112:0/3025472753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 msgr2=0x7fa70c19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.189+0000 7fa713200700 1 --2- 172.21.15.112:0/3025472753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c19c510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0119f0 tx=0x7fa6fc011ad0 comp rx=0 tx=0).stop 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.189+0000 7fa713200700 1 -- 172.21.15.112:0/3025472753 shutdown_connections 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.189+0000 7fa713200700 1 --2- 172.21.15.112:0/3025472753 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6f00396b0 0x7fa6f003bb70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.189+0000 7fa713200700 1 --2- 172.21.15.112:0/3025472753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa70c1086c0 0x7fa70c19c510 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.189+0000 7fa713200700 1 -- 172.21.15.112:0/3025472753 >> 172.21.15.112:0/3025472753 conn(0x7fa70c103c40 msgr2=0x7fa70c104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.190+0000 7fa713200700 1 -- 172.21.15.112:0/3025472753 shutdown_connections 2024-04-25T18:35:03.907 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:02.190+0000 7fa713200700 1 -- 172.21.15.112:0/3025472753 wait complete. 2024-04-25T18:35:03.934 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:35:03.934 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-25T18:35:03.956 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:35:03.956 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T18:35:03.956 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-25T18:35:03.956 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T18:35:03.963 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph Dashboard is now available at: 2024-04-25T18:35:03.963 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:03.963 INFO:teuthology.orchestra.run.smithi112.stdout: URL: https://smithi112.front.sepia.ceph.com:8443/ 2024-04-25T18:35:03.963 INFO:teuthology.orchestra.run.smithi112.stdout: User: admin 2024-04-25T18:35:03.963 INFO:teuthology.orchestra.run.smithi112.stdout: Password: y4q9s32b0r 2024-04-25T18:35:03.963 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:03.964 INFO:teuthology.orchestra.run.smithi112.stdout:Saving cluster configuration to /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config directory 2024-04-25T18:35:03.966 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling autotune for osd_memory_target 2024-04-25T18:35:05.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.668+0000 7f79a2a00700 1 Processor -- start 2024-04-25T18:35:05.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.668+0000 7f79a2a00700 1 -- start start 2024-04-25T18:35:05.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.668+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c082b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:05.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.668+0000 7f79a2a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f799c083110 con 0x7f799c082720 2024-04-25T18:35:05.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.669+0000 7f799be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c082b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:05.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.669+0000 7f799be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c082b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54878/0 (socket says 172.21.15.112:54878) 2024-04-25T18:35:05.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.669+0000 7f799be00700 1 -- 172.21.15.112:0/975343565 learned_addr learned my addr 172.21.15.112:0/975343565 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:05.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.669+0000 7f799be00700 1 -- 172.21.15.112:0/975343565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c10fef0 con 0x7f799c082720 2024-04-25T18:35:05.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.669+0000 7f799be00700 1 --2- 172.21.15.112:0/975343565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c082b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f798c0098a0 tx=0x7f798c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12b5f116832327b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:05.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.670+0000 7f799aa00700 1 -- 172.21.15.112:0/975343565 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f798c004160 con 0x7f799c082720 2024-04-25T18:35:05.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.670+0000 7f799aa00700 1 -- 172.21.15.112:0/975343565 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c004300 con 0x7f799c082720 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.670+0000 7f799aa00700 1 -- 172.21.15.112:0/975343565 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f798c011a20 con 0x7f799c082720 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.670+0000 7f79a2a00700 1 -- 172.21.15.112:0/975343565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 msgr2=0x7f799c082b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.670+0000 7f79a2a00700 1 --2- 172.21.15.112:0/975343565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c082b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f798c0098a0 tx=0x7f798c009bb0 comp rx=0 tx=0).stop 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.670+0000 7f79a2a00700 1 -- 172.21.15.112:0/975343565 shutdown_connections 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.670+0000 7f79a2a00700 1 --2- 172.21.15.112:0/975343565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c082b40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.670+0000 7f79a2a00700 1 -- 172.21.15.112:0/975343565 >> 172.21.15.112:0/975343565 conn(0x7f799c07dce0 msgr2=0x7f799c080100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.671+0000 7f79a2a00700 1 -- 172.21.15.112:0/975343565 shutdown_connections 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.671+0000 7f79a2a00700 1 -- 172.21.15.112:0/975343565 wait complete. 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.672+0000 7f79a2a00700 1 Processor -- start 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.672+0000 7f79a2a00700 1 -- start start 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.672+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c1a2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.672+0000 7f79a2a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f799c1132a0 con 0x7f799c082720 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.672+0000 7f799be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c1a2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.672+0000 7f799be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c1a2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54882/0 (socket says 172.21.15.112:54882) 2024-04-25T18:35:05.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.672+0000 7f799be00700 1 -- 172.21.15.112:0/3708301435 learned_addr learned my addr 172.21.15.112:0/3708301435 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.673+0000 7f799be00700 1 -- 172.21.15.112:0/3708301435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c1a3080 con 0x7f799c082720 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.673+0000 7f799be00700 1 --2- 172.21.15.112:0/3708301435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c1a2b40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f798c011770 tx=0x7f798c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.673+0000 7f7998c00700 1 -- 172.21.15.112:0/3708301435 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f798c0039e0 con 0x7f799c082720 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.673+0000 7f7998c00700 1 -- 172.21.15.112:0/3708301435 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c01b8f0 con 0x7f799c082720 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.673+0000 7f79a2a00700 1 -- 172.21.15.112:0/3708301435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c1a3310 con 0x7f799c082720 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.673+0000 7f7998c00700 1 -- 172.21.15.112:0/3708301435 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f798c023760 con 0x7f799c082720 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.673+0000 7f79a2a00700 1 -- 172.21.15.112:0/3708301435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c1a6000 con 0x7f799c082720 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.675+0000 7f7998c00700 1 -- 172.21.15.112:0/3708301435 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f798c01b440 con 0x7f799c082720 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.675+0000 7f79a2a00700 1 -- 172.21.15.112:0/3708301435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c0640a0 con 0x7f799c082720 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.675+0000 7f7998c00700 1 --2- 172.21.15.112:0/3708301435 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7980039650 0x7f798003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.675+0000 7f7998c00700 1 -- 172.21.15.112:0/3708301435 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f798c04e0f0 con 0x7f799c082720 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.675+0000 7f799b400700 1 --2- 172.21.15.112:0/3708301435 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7980039650 0x7f798003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.676+0000 7f799b400700 1 --2- 172.21.15.112:0/3708301435 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7980039650 0x7f798003bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7988006fd0 tx=0x7f7988006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.681+0000 7f7998c00700 1 -- 172.21.15.112:0/3708301435 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f798c01f030 con 0x7f799c082720 2024-04-25T18:35:05.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.903+0000 7f79a2a00700 1 -- 172.21.15.112:0/3708301435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f799c1a6620 con 0x7f799c082720 2024-04-25T18:35:05.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.904+0000 7f7998c00700 1 -- 172.21.15.112:0/3708301435 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f798c02bb90 con 0x7f799c082720 2024-04-25T18:35:05.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.908+0000 7f79a2a00700 1 -- 172.21.15.112:0/3708301435 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7980039650 msgr2=0x7f798003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:05.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.908+0000 7f79a2a00700 1 --2- 172.21.15.112:0/3708301435 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7980039650 0x7f798003bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7988006fd0 tx=0x7f7988006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:05.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.908+0000 7f79a2a00700 1 -- 172.21.15.112:0/3708301435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 msgr2=0x7f799c1a2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:05.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.908+0000 7f79a2a00700 1 --2- 172.21.15.112:0/3708301435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c1a2b40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f798c011770 tx=0x7f798c011aa0 comp rx=0 tx=0).stop 2024-04-25T18:35:05.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.908+0000 7f79a2a00700 1 -- 172.21.15.112:0/3708301435 shutdown_connections 2024-04-25T18:35:05.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.908+0000 7f79a2a00700 1 --2- 172.21.15.112:0/3708301435 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7980039650 0x7f798003bb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:05.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.908+0000 7f79a2a00700 1 --2- 172.21.15.112:0/3708301435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f799c082720 0x7f799c1a2b40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:05.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.908+0000 7f79a2a00700 1 -- 172.21.15.112:0/3708301435 >> 172.21.15.112:0/3708301435 conn(0x7f799c07dce0 msgr2=0x7f799c07e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:05.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.908+0000 7f79a2a00700 1 -- 172.21.15.112:0/3708301435 shutdown_connections 2024-04-25T18:35:05.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:04.908+0000 7f79a2a00700 1 -- 172.21.15.112:0/3708301435 wait complete. 2024-04-25T18:35:07.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.243+0000 7fc468200700 1 Processor -- start 2024-04-25T18:35:07.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.244+0000 7fc468200700 1 -- start start 2024-04-25T18:35:07.375 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.244+0000 7fc468200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc4601079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:07.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.244+0000 7fc468200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc460107f90 con 0x7fc4601075a0 2024-04-25T18:35:07.376 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.245+0000 7fc465800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc4601079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:07.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.245+0000 7fc465800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc4601079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54894/0 (socket says 172.21.15.112:54894) 2024-04-25T18:35:07.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.245+0000 7fc465800700 1 -- 172.21.15.112:0/631368678 learned_addr learned my addr 172.21.15.112:0/631368678 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:07.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.245+0000 7fc465800700 1 -- 172.21.15.112:0/631368678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4601087a0 con 0x7fc4601075a0 2024-04-25T18:35:07.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.245+0000 7fc465800700 1 --2- 172.21.15.112:0/631368678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc4601079c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc45400bb40 tx=0x7fc45400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e24e43cf7ae2d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:07.381 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.246+0000 7fc45fe00700 1 -- 172.21.15.112:0/631368678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc45400dbe0 con 0x7fc4601075a0 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.246+0000 7fc45fe00700 1 -- 172.21.15.112:0/631368678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4540044e0 con 0x7fc4601075a0 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.246+0000 7fc45fe00700 1 -- 172.21.15.112:0/631368678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc454010470 con 0x7fc4601075a0 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.246+0000 7fc468200700 1 -- 172.21.15.112:0/631368678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 msgr2=0x7fc4601079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.246+0000 7fc468200700 1 --2- 172.21.15.112:0/631368678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc4601079c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc45400bb40 tx=0x7fc45400be50 comp rx=0 tx=0).stop 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.247+0000 7fc468200700 1 -- 172.21.15.112:0/631368678 shutdown_connections 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.247+0000 7fc468200700 1 --2- 172.21.15.112:0/631368678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc4601079c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.247+0000 7fc468200700 1 -- 172.21.15.112:0/631368678 >> 172.21.15.112:0/631368678 conn(0x7fc460102ba0 msgr2=0x7fc460104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.247+0000 7fc468200700 1 -- 172.21.15.112:0/631368678 shutdown_connections 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.247+0000 7fc468200700 1 -- 172.21.15.112:0/631368678 wait complete. 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.248+0000 7fc468200700 1 Processor -- start 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.248+0000 7fc468200700 1 -- start start 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.248+0000 7fc468200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc46019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.248+0000 7fc468200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc460062e80 con 0x7fc4601075a0 2024-04-25T18:35:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.249+0000 7fc465800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc46019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:07.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.249+0000 7fc465800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc46019e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54910/0 (socket says 172.21.15.112:54910) 2024-04-25T18:35:07.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.249+0000 7fc465800700 1 -- 172.21.15.112:0/1949683441 learned_addr learned my addr 172.21.15.112:0/1949683441 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:07.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.249+0000 7fc465800700 1 -- 172.21.15.112:0/1949683441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46019ec70 con 0x7fc4601075a0 2024-04-25T18:35:07.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.249+0000 7fc465800700 1 --2- 172.21.15.112:0/1949683441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc46019e730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc454003cc0 tx=0x7fc454003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:07.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.250+0000 7fc45e000700 1 -- 172.21.15.112:0/1949683441 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc454010720 con 0x7fc4601075a0 2024-04-25T18:35:07.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.250+0000 7fc45e000700 1 -- 172.21.15.112:0/1949683441 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc45401d070 con 0x7fc4601075a0 2024-04-25T18:35:07.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.250+0000 7fc468200700 1 -- 172.21.15.112:0/1949683441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46019ef00 con 0x7fc4601075a0 2024-04-25T18:35:07.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.250+0000 7fc45e000700 1 -- 172.21.15.112:0/1949683441 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc454021aa0 con 0x7fc4601075a0 2024-04-25T18:35:07.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.250+0000 7fc468200700 1 -- 172.21.15.112:0/1949683441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4601a1bf0 con 0x7fc4601075a0 2024-04-25T18:35:07.383 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.251+0000 7fc45e000700 1 -- 172.21.15.112:0/1949683441 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fc45401f030 con 0x7fc4601075a0 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.251+0000 7fc468200700 1 -- 172.21.15.112:0/1949683441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4600640a0 con 0x7fc4601075a0 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.251+0000 7fc45e000700 1 --2- 172.21.15.112:0/1949683441 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc4480396a0 0x7fc44803bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.251+0000 7fc45e000700 1 -- 172.21.15.112:0/1949683441 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc45404dc50 con 0x7fc4601075a0 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.252+0000 7fc464e00700 1 --2- 172.21.15.112:0/1949683441 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc4480396a0 0x7fc44803bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.252+0000 7fc464e00700 1 --2- 172.21.15.112:0/1949683441 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc4480396a0 0x7fc44803bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc44c006fd0 tx=0x7fc44c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.257+0000 7fc45e000700 1 -- 172.21.15.112:0/1949683441 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc454053020 con 0x7fc4601075a0 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.596+0000 7fc468200700 1 -- 172.21.15.112:0/1949683441 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc460197c90 con 0x7fc4601075a0 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.600+0000 7fc45e000700 1 -- 172.21.15.112:0/1949683441 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fc460197c90 con 0x7fc4601075a0 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.605+0000 7fc468200700 1 -- 172.21.15.112:0/1949683441 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc4480396a0 msgr2=0x7fc44803bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.605+0000 7fc468200700 1 --2- 172.21.15.112:0/1949683441 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc4480396a0 0x7fc44803bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc44c006fd0 tx=0x7fc44c006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.605+0000 7fc468200700 1 -- 172.21.15.112:0/1949683441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 msgr2=0x7fc46019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:07.384 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.605+0000 7fc468200700 1 --2- 172.21.15.112:0/1949683441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc46019e730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc454003cc0 tx=0x7fc454003ff0 comp rx=0 tx=0).stop 2024-04-25T18:35:07.385 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.605+0000 7fc468200700 1 -- 172.21.15.112:0/1949683441 shutdown_connections 2024-04-25T18:35:07.385 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.605+0000 7fc468200700 1 --2- 172.21.15.112:0/1949683441 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc4480396a0 0x7fc44803bb60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:07.386 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.605+0000 7fc468200700 1 --2- 172.21.15.112:0/1949683441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4601075a0 0x7fc46019e730 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:07.386 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.605+0000 7fc468200700 1 -- 172.21.15.112:0/1949683441 >> 172.21.15.112:0/1949683441 conn(0x7fc460102ba0 msgr2=0x7fc460194d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:07.386 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.605+0000 7fc468200700 1 -- 172.21.15.112:0/1949683441 shutdown_connections 2024-04-25T18:35:07.386 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T18:35:06.605+0000 7fc468200700 1 -- 172.21.15.112:0/1949683441 wait complete. 2024-04-25T18:35:07.386 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T18:35:07.386 INFO:teuthology.orchestra.run.smithi112.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T18:35:07.386 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:07.386 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:35:07.386 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:07.386 INFO:teuthology.orchestra.run.smithi112.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T18:35:07.386 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T18:35:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:07.387 INFO:teuthology.orchestra.run.smithi112.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T18:35:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: ceph telemetry on 2024-04-25T18:35:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:07.387 INFO:teuthology.orchestra.run.smithi112.stdout:For more information see: 2024-04-25T18:35:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T18:35:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:07.387 INFO:teuthology.orchestra.run.smithi112.stdout:Bootstrap complete. 2024-04-25T18:35:07.492 INFO:tasks.cephadm:Fetching config... 2024-04-25T18:35:07.492 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:35:07.493 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T18:35:07.551 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T18:35:07.551 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:35:07.551 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T18:35:07.590 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T18:35:07.591 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:35:07.591 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T18:35:07.699 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:07 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1949683441' entity='client.admin' 2024-04-25T18:35:07.712 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T18:35:07.712 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:35:07.712 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T18:35:07.813 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T18:35:07.813 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2DUIksrtLuNu5/Ru0zw7LxtRazCt70+VyRCF/CW7c84qYHKocH66GkeRtAKsCfii+8WBzkTjxsajwST1wcQgX7rcRvvoqEY8hwmuyNV/lijnIH2bpWcvtve9CKm140lDmuEHfGe/2SXH7e3qhin1fywglYBz3UDNbkBslOjm6C48mIlGUxRZtflOol1vMfqiUeaFD3n0b9tciLdjBzhOEB/monULPGjaF+EGePzY2BXy4FSIIZ6SRoLHxRSn355NeETdOxhl8Yl6i+90ZWREtbugZr2CaIw9TWmTXYqji5TNAB6d9lAMrIXnSKudxJ8VQMyJAlAAiVgodbMdbMn9m7KMh2+U01H4NzpA4HqPuInQkzZj8VmyKqq3+ymXsSA+aWPOPto83BgkpjnyG8Fev+oULcI6gUuCMvoZviZXWBI4Vc045HsirFeNbXx9ZzR/XLtRmooVlEnQDiSmYcZt0UwE5EbujnavqR31CmgfzwA9bXj99Fsg3kKKPsGsbNSs= ceph-2fe18a52-0332-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T18:35:07.940 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2DUIksrtLuNu5/Ru0zw7LxtRazCt70+VyRCF/CW7c84qYHKocH66GkeRtAKsCfii+8WBzkTjxsajwST1wcQgX7rcRvvoqEY8hwmuyNV/lijnIH2bpWcvtve9CKm140lDmuEHfGe/2SXH7e3qhin1fywglYBz3UDNbkBslOjm6C48mIlGUxRZtflOol1vMfqiUeaFD3n0b9tciLdjBzhOEB/monULPGjaF+EGePzY2BXy4FSIIZ6SRoLHxRSn355NeETdOxhl8Yl6i+90ZWREtbugZr2CaIw9TWmTXYqji5TNAB6d9lAMrIXnSKudxJ8VQMyJAlAAiVgodbMdbMn9m7KMh2+U01H4NzpA4HqPuInQkzZj8VmyKqq3+ymXsSA+aWPOPto83BgkpjnyG8Fev+oULcI6gUuCMvoZviZXWBI4Vc045HsirFeNbXx9ZzR/XLtRmooVlEnQDiSmYcZt0UwE5EbujnavqR31CmgfzwA9bXj99Fsg3kKKPsGsbNSs= ceph-2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:35:07.968 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2DUIksrtLuNu5/Ru0zw7LxtRazCt70+VyRCF/CW7c84qYHKocH66GkeRtAKsCfii+8WBzkTjxsajwST1wcQgX7rcRvvoqEY8hwmuyNV/lijnIH2bpWcvtve9CKm140lDmuEHfGe/2SXH7e3qhin1fywglYBz3UDNbkBslOjm6C48mIlGUxRZtflOol1vMfqiUeaFD3n0b9tciLdjBzhOEB/monULPGjaF+EGePzY2BXy4FSIIZ6SRoLHxRSn355NeETdOxhl8Yl6i+90ZWREtbugZr2CaIw9TWmTXYqji5TNAB6d9lAMrIXnSKudxJ8VQMyJAlAAiVgodbMdbMn9m7KMh2+U01H4NzpA4HqPuInQkzZj8VmyKqq3+ymXsSA+aWPOPto83BgkpjnyG8Fev+oULcI6gUuCMvoZviZXWBI4Vc045HsirFeNbXx9ZzR/XLtRmooVlEnQDiSmYcZt0UwE5EbujnavqR31CmgfzwA9bXj99Fsg3kKKPsGsbNSs= ceph-2fe18a52-0332-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T18:35:08.009 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2DUIksrtLuNu5/Ru0zw7LxtRazCt70+VyRCF/CW7c84qYHKocH66GkeRtAKsCfii+8WBzkTjxsajwST1wcQgX7rcRvvoqEY8hwmuyNV/lijnIH2bpWcvtve9CKm140lDmuEHfGe/2SXH7e3qhin1fywglYBz3UDNbkBslOjm6C48mIlGUxRZtflOol1vMfqiUeaFD3n0b9tciLdjBzhOEB/monULPGjaF+EGePzY2BXy4FSIIZ6SRoLHxRSn355NeETdOxhl8Yl6i+90ZWREtbugZr2CaIw9TWmTXYqji5TNAB6d9lAMrIXnSKudxJ8VQMyJAlAAiVgodbMdbMn9m7KMh2+U01H4NzpA4HqPuInQkzZj8VmyKqq3+ymXsSA+aWPOPto83BgkpjnyG8Fev+oULcI6gUuCMvoZviZXWBI4Vc045HsirFeNbXx9ZzR/XLtRmooVlEnQDiSmYcZt0UwE5EbujnavqR31CmgfzwA9bXj99Fsg3kKKPsGsbNSs= ceph-2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:35:08.021 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2DUIksrtLuNu5/Ru0zw7LxtRazCt70+VyRCF/CW7c84qYHKocH66GkeRtAKsCfii+8WBzkTjxsajwST1wcQgX7rcRvvoqEY8hwmuyNV/lijnIH2bpWcvtve9CKm140lDmuEHfGe/2SXH7e3qhin1fywglYBz3UDNbkBslOjm6C48mIlGUxRZtflOol1vMfqiUeaFD3n0b9tciLdjBzhOEB/monULPGjaF+EGePzY2BXy4FSIIZ6SRoLHxRSn355NeETdOxhl8Yl6i+90ZWREtbugZr2CaIw9TWmTXYqji5TNAB6d9lAMrIXnSKudxJ8VQMyJAlAAiVgodbMdbMn9m7KMh2+U01H4NzpA4HqPuInQkzZj8VmyKqq3+ymXsSA+aWPOPto83BgkpjnyG8Fev+oULcI6gUuCMvoZviZXWBI4Vc045HsirFeNbXx9ZzR/XLtRmooVlEnQDiSmYcZt0UwE5EbujnavqR31CmgfzwA9bXj99Fsg3kKKPsGsbNSs= ceph-2fe18a52-0332-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T18:35:08.065 INFO:teuthology.orchestra.run.smithi171.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2DUIksrtLuNu5/Ru0zw7LxtRazCt70+VyRCF/CW7c84qYHKocH66GkeRtAKsCfii+8WBzkTjxsajwST1wcQgX7rcRvvoqEY8hwmuyNV/lijnIH2bpWcvtve9CKm140lDmuEHfGe/2SXH7e3qhin1fywglYBz3UDNbkBslOjm6C48mIlGUxRZtflOol1vMfqiUeaFD3n0b9tciLdjBzhOEB/monULPGjaF+EGePzY2BXy4FSIIZ6SRoLHxRSn355NeETdOxhl8Yl6i+90ZWREtbugZr2CaIw9TWmTXYqji5TNAB6d9lAMrIXnSKudxJ8VQMyJAlAAiVgodbMdbMn9m7KMh2+U01H4NzpA4HqPuInQkzZj8VmyKqq3+ymXsSA+aWPOPto83BgkpjnyG8Fev+oULcI6gUuCMvoZviZXWBI4Vc045HsirFeNbXx9ZzR/XLtRmooVlEnQDiSmYcZt0UwE5EbujnavqR31CmgfzwA9bXj99Fsg3kKKPsGsbNSs= ceph-2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:35:08.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T18:35:08.488 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:35:11.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.287+0000 7f079dc00700 1 -- 172.21.15.112:0/2782656561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07981046c0 msgr2=0x7f0798104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:11.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.287+0000 7f079dc00700 1 --2- 172.21.15.112:0/2782656561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07981046c0 0x7f0798104ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0788009910 tx=0x7f0788009c20 comp rx=0 tx=0).stop 2024-04-25T18:35:11.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.287+0000 7f079dc00700 1 -- 172.21.15.112:0/2782656561 shutdown_connections 2024-04-25T18:35:11.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.287+0000 7f079dc00700 1 --2- 172.21.15.112:0/2782656561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07981046c0 0x7f0798104ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:11.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.287+0000 7f079dc00700 1 -- 172.21.15.112:0/2782656561 >> 172.21.15.112:0/2782656561 conn(0x7f07980ffc60 msgr2=0x7f07981020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:11.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.288+0000 7f079dc00700 1 -- 172.21.15.112:0/2782656561 shutdown_connections 2024-04-25T18:35:11.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.288+0000 7f079dc00700 1 -- 172.21.15.112:0/2782656561 wait complete. 2024-04-25T18:35:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.289+0000 7f079dc00700 1 Processor -- start 2024-04-25T18:35:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.289+0000 7f079dc00700 1 -- start start 2024-04-25T18:35:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.290+0000 7f079dc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07981046c0 0x7f0798199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:11.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.290+0000 7f079dc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f079810a600 con 0x7f07981046c0 2024-04-25T18:35:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.290+0000 7f0796a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07981046c0 0x7f0798199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.290+0000 7f0796a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07981046c0 0x7f0798199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34170/0 (socket says 172.21.15.112:34170) 2024-04-25T18:35:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.290+0000 7f0796a00700 1 -- 172.21.15.112:0/3814950824 learned_addr learned my addr 172.21.15.112:0/3814950824 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:11.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.290+0000 7f0796a00700 1 -- 172.21.15.112:0/3814950824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079819a230 con 0x7f07981046c0 2024-04-25T18:35:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.291+0000 7f0796a00700 1 --2- 172.21.15.112:0/3814950824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07981046c0 0x7f0798199cf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0788005800 tx=0x7f0788009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.291+0000 7f078f400700 1 -- 172.21.15.112:0/3814950824 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0788027070 con 0x7f07981046c0 2024-04-25T18:35:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.291+0000 7f078f400700 1 -- 172.21.15.112:0/3814950824 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f078800be10 con 0x7f07981046c0 2024-04-25T18:35:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.291+0000 7f078f400700 1 -- 172.21.15.112:0/3814950824 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078800f460 con 0x7f07981046c0 2024-04-25T18:35:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.291+0000 7f079dc00700 1 -- 172.21.15.112:0/3814950824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079819a4c0 con 0x7f07981046c0 2024-04-25T18:35:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.291+0000 7f079dc00700 1 -- 172.21.15.112:0/3814950824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f079819d1a0 con 0x7f07981046c0 2024-04-25T18:35:11.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.292+0000 7f079dc00700 1 -- 172.21.15.112:0/3814950824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f079810ad80 con 0x7f07981046c0 2024-04-25T18:35:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.293+0000 7f078f400700 1 -- 172.21.15.112:0/3814950824 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f078802a070 con 0x7f07981046c0 2024-04-25T18:35:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.293+0000 7f078f400700 1 --2- 172.21.15.112:0/3814950824 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f077c0396b0 0x7f077c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:11.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.293+0000 7f078f400700 1 -- 172.21.15.112:0/3814950824 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f078801e030 con 0x7f07981046c0 2024-04-25T18:35:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.293+0000 7f0796000700 1 --2- 172.21.15.112:0/3814950824 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f077c0396b0 0x7f077c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.294+0000 7f0796000700 1 --2- 172.21.15.112:0/3814950824 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f077c0396b0 0x7f077c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0780006fd0 tx=0x7f0780006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.299+0000 7f078f400700 1 -- 172.21.15.112:0/3814950824 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0788013070 con 0x7f07981046c0 2024-04-25T18:35:11.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.522+0000 7f079dc00700 1 -- 172.21.15.112:0/3814950824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f079802e090 con 0x7f07981046c0 2024-04-25T18:35:11.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.526+0000 7f078f400700 1 -- 172.21.15.112:0/3814950824 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f0788031020 con 0x7f07981046c0 2024-04-25T18:35:11.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.529+0000 7f079dc00700 1 -- 172.21.15.112:0/3814950824 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f077c0396b0 msgr2=0x7f077c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:11.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.529+0000 7f079dc00700 1 --2- 172.21.15.112:0/3814950824 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f077c0396b0 0x7f077c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0780006fd0 tx=0x7f0780006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:11.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.529+0000 7f079dc00700 1 -- 172.21.15.112:0/3814950824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07981046c0 msgr2=0x7f0798199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:11.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.529+0000 7f079dc00700 1 --2- 172.21.15.112:0/3814950824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07981046c0 0x7f0798199cf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0788005800 tx=0x7f0788009f60 comp rx=0 tx=0).stop 2024-04-25T18:35:11.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.529+0000 7f079dc00700 1 -- 172.21.15.112:0/3814950824 shutdown_connections 2024-04-25T18:35:11.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.529+0000 7f079dc00700 1 --2- 172.21.15.112:0/3814950824 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f077c0396b0 0x7f077c03bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:11.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.529+0000 7f079dc00700 1 --2- 172.21.15.112:0/3814950824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07981046c0 0x7f0798199cf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:11.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.529+0000 7f079dc00700 1 -- 172.21.15.112:0/3814950824 >> 172.21.15.112:0/3814950824 conn(0x7f07980ffc60 msgr2=0x7f0798100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:11.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.530+0000 7f079dc00700 1 -- 172.21.15.112:0/3814950824 shutdown_connections 2024-04-25T18:35:11.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:11.530+0000 7f079dc00700 1 -- 172.21.15.112:0/3814950824 wait complete. 2024-04-25T18:35:11.876 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:11.876 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:11.877 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:35:11.877 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:11.877 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:35:11.877 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:11.877 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:11 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3814950824' entity='client.admin' 2024-04-25T18:35:11.877 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:35:11.877 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:12.291 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T18:35:12.291 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T18:35:12.717 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:35:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.819+0000 7fdb56000700 1 -- 172.21.15.112:0/594439096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb501046c0 msgr2=0x7fdb50104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.819+0000 7fdb56000700 1 --2- 172.21.15.112:0/594439096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb501046c0 0x7fdb50104ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009910 tx=0x7fdb40009c20 comp rx=0 tx=0).stop 2024-04-25T18:35:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.819+0000 7fdb56000700 1 -- 172.21.15.112:0/594439096 shutdown_connections 2024-04-25T18:35:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.819+0000 7fdb56000700 1 --2- 172.21.15.112:0/594439096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb501046c0 0x7fdb50104ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.819+0000 7fdb56000700 1 -- 172.21.15.112:0/594439096 >> 172.21.15.112:0/594439096 conn(0x7fdb500ffc60 msgr2=0x7fdb501020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.819+0000 7fdb56000700 1 -- 172.21.15.112:0/594439096 shutdown_connections 2024-04-25T18:35:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.819+0000 7fdb56000700 1 -- 172.21.15.112:0/594439096 wait complete. 2024-04-25T18:35:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.820+0000 7fdb56000700 1 Processor -- start 2024-04-25T18:35:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.821+0000 7fdb56000700 1 -- start start 2024-04-25T18:35:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.821+0000 7fdb56000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb501046c0 0x7fdb50199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.821+0000 7fdb56000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5010a600 con 0x7fdb501046c0 2024-04-25T18:35:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.821+0000 7fdb4f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb501046c0 0x7fdb50199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.821+0000 7fdb4f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb501046c0 0x7fdb50199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34198/0 (socket says 172.21.15.112:34198) 2024-04-25T18:35:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.821+0000 7fdb4f400700 1 -- 172.21.15.112:0/3941781611 learned_addr learned my addr 172.21.15.112:0/3941781611 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.822+0000 7fdb4f400700 1 -- 172.21.15.112:0/3941781611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5019a230 con 0x7fdb501046c0 2024-04-25T18:35:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.822+0000 7fdb4f400700 1 --2- 172.21.15.112:0/3941781611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb501046c0 0x7fdb50199cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb40005950 tx=0x7fdb40004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.822+0000 7fdb47e00700 1 -- 172.21.15.112:0/3941781611 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb40026070 con 0x7fdb501046c0 2024-04-25T18:35:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.822+0000 7fdb47e00700 1 -- 172.21.15.112:0/3941781611 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb4000bc00 con 0x7fdb501046c0 2024-04-25T18:35:14.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.822+0000 7fdb47e00700 1 -- 172.21.15.112:0/3941781611 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb40003a10 con 0x7fdb501046c0 2024-04-25T18:35:14.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.822+0000 7fdb56000700 1 -- 172.21.15.112:0/3941781611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5019a4c0 con 0x7fdb501046c0 2024-04-25T18:35:14.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.822+0000 7fdb56000700 1 -- 172.21.15.112:0/3941781611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5019d1a0 con 0x7fdb501046c0 2024-04-25T18:35:14.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.824+0000 7fdb47e00700 1 -- 172.21.15.112:0/3941781611 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fdb40029070 con 0x7fdb501046c0 2024-04-25T18:35:14.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.824+0000 7fdb56000700 1 -- 172.21.15.112:0/3941781611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5010ad80 con 0x7fdb501046c0 2024-04-25T18:35:14.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.824+0000 7fdb47e00700 1 --2- 172.21.15.112:0/3941781611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdb34039650 0x7fdb3403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:14.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.824+0000 7fdb47e00700 1 -- 172.21.15.112:0/3941781611 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdb4001e070 con 0x7fdb501046c0 2024-04-25T18:35:14.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.824+0000 7fdb4ea00700 1 --2- 172.21.15.112:0/3941781611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdb34039650 0x7fdb3403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:14.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.825+0000 7fdb4ea00700 1 --2- 172.21.15.112:0/3941781611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdb34039650 0x7fdb3403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c006fd0 tx=0x7fdb3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:14.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:14.830+0000 7fdb47e00700 1 -- 172.21.15.112:0/3941781611 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb40013070 con 0x7fdb501046c0 2024-04-25T18:35:15.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.062+0000 7fdb56000700 1 -- 172.21.15.112:0/3941781611 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fdb5002e120 con 0x7fdb34039650 2024-04-25T18:35:15.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.069+0000 7fdb47e00700 1 -- 172.21.15.112:0/3941781611 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdb5002e120 con 0x7fdb34039650 2024-04-25T18:35:15.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.072+0000 7fdb56000700 1 -- 172.21.15.112:0/3941781611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdb34039650 msgr2=0x7fdb3403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:15.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.072+0000 7fdb56000700 1 --2- 172.21.15.112:0/3941781611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdb34039650 0x7fdb3403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c006fd0 tx=0x7fdb3c006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:15.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.072+0000 7fdb56000700 1 -- 172.21.15.112:0/3941781611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb501046c0 msgr2=0x7fdb50199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:15.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.072+0000 7fdb56000700 1 --2- 172.21.15.112:0/3941781611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb501046c0 0x7fdb50199cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb40005950 tx=0x7fdb40004fa0 comp rx=0 tx=0).stop 2024-04-25T18:35:15.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.072+0000 7fdb56000700 1 -- 172.21.15.112:0/3941781611 shutdown_connections 2024-04-25T18:35:15.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.072+0000 7fdb56000700 1 --2- 172.21.15.112:0/3941781611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdb34039650 0x7fdb3403bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:15.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.072+0000 7fdb56000700 1 --2- 172.21.15.112:0/3941781611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb501046c0 0x7fdb50199cf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:15.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.072+0000 7fdb56000700 1 -- 172.21.15.112:0/3941781611 >> 172.21.15.112:0/3941781611 conn(0x7fdb500ffc60 msgr2=0x7fdb50100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:15.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.072+0000 7fdb56000700 1 -- 172.21.15.112:0/3941781611 shutdown_connections 2024-04-25T18:35:15.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:15.072+0000 7fdb56000700 1 -- 172.21.15.112:0/3941781611 wait complete. 2024-04-25T18:35:15.912 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-04-25T18:35:15.913 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-25T18:35:15.913 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-04-25T18:35:15.932 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-25T18:35:15.932 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:35:15.989 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-04-25T18:35:15.989 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch host add smithi161 2024-04-25T18:35:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:16 smithi112 ceph-mon[27864]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:16 smithi112 ceph-mon[27864]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:35:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:35:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:35:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:35:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:16 smithi112 ceph-mon[27864]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T18:35:16.389 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:35:17.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:17 smithi112 ceph-mon[27864]: Updating smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:35:18.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:18 smithi112 ceph-mon[27864]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:35:18.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:18 smithi112 ceph-mon[27864]: Updating smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:35:18.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:18 smithi112 ceph-mon[27864]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:18.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:18.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:18.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.673+0000 7f3b15400700 1 -- 172.21.15.112:0/2130170557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b10074d60 msgr2=0x7f3b10075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.673+0000 7f3b15400700 1 --2- 172.21.15.112:0/2130170557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b10074d60 0x7f3b10075140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009910 tx=0x7f3b00009c20 comp rx=0 tx=0).stop 2024-04-25T18:35:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.673+0000 7f3b15400700 1 -- 172.21.15.112:0/2130170557 shutdown_connections 2024-04-25T18:35:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.673+0000 7f3b15400700 1 --2- 172.21.15.112:0/2130170557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b10074d60 0x7f3b10075140 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.673+0000 7f3b15400700 1 -- 172.21.15.112:0/2130170557 >> 172.21.15.112:0/2130170557 conn(0x7f3b100fdf60 msgr2=0x7f3b10100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.673+0000 7f3b15400700 1 -- 172.21.15.112:0/2130170557 shutdown_connections 2024-04-25T18:35:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.673+0000 7f3b15400700 1 -- 172.21.15.112:0/2130170557 wait complete. 2024-04-25T18:35:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.674+0000 7f3b15400700 1 Processor -- start 2024-04-25T18:35:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.675+0000 7f3b15400700 1 -- start start 2024-04-25T18:35:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.675+0000 7f3b15400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b10074d60 0x7f3b1019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.675+0000 7f3b15400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1010eb70 con 0x7f3b10074d60 2024-04-25T18:35:18.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.675+0000 7f3b0e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b10074d60 0x7f3b1019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.675+0000 7f3b0e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b10074d60 0x7f3b1019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34222/0 (socket says 172.21.15.112:34222) 2024-04-25T18:35:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.675+0000 7f3b0e800700 1 -- 172.21.15.112:0/3738124460 learned_addr learned my addr 172.21.15.112:0/3738124460 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.676+0000 7f3b0e800700 1 -- 172.21.15.112:0/3738124460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1019a600 con 0x7f3b10074d60 2024-04-25T18:35:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.676+0000 7f3b0e800700 1 --2- 172.21.15.112:0/3738124460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b10074d60 0x7f3b1019a0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b00006010 tx=0x7f3b00005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.676+0000 7f3b07400700 1 -- 172.21.15.112:0/3738124460 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b00026070 con 0x7f3b10074d60 2024-04-25T18:35:18.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.676+0000 7f3b07400700 1 -- 172.21.15.112:0/3738124460 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b0000bc00 con 0x7f3b10074d60 2024-04-25T18:35:18.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.676+0000 7f3b15400700 1 -- 172.21.15.112:0/3738124460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1019a890 con 0x7f3b10074d60 2024-04-25T18:35:18.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.676+0000 7f3b07400700 1 -- 172.21.15.112:0/3738124460 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b00003910 con 0x7f3b10074d60 2024-04-25T18:35:18.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.676+0000 7f3b15400700 1 -- 172.21.15.112:0/3738124460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1019acf0 con 0x7f3b10074d60 2024-04-25T18:35:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.678+0000 7f3b07400700 1 -- 172.21.15.112:0/3738124460 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f3b00029070 con 0x7f3b10074d60 2024-04-25T18:35:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.678+0000 7f3b15400700 1 -- 172.21.15.112:0/3738124460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b100764a0 con 0x7f3b10074d60 2024-04-25T18:35:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.678+0000 7f3b07400700 1 --2- 172.21.15.112:0/3738124460 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3af40396b0 0x7f3af403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.678+0000 7f3b07400700 1 -- 172.21.15.112:0/3738124460 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3b0001e070 con 0x7f3b10074d60 2024-04-25T18:35:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.678+0000 7f3b0de00700 1 --2- 172.21.15.112:0/3738124460 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3af40396b0 0x7f3af403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.679+0000 7f3b0de00700 1 --2- 172.21.15.112:0/3738124460 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3af40396b0 0x7f3af403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3afc006fd0 tx=0x7f3afc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:18.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.684+0000 7f3b07400700 1 -- 172.21.15.112:0/3738124460 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b00013070 con 0x7f3b10074d60 2024-04-25T18:35:18.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:18.911+0000 7f3b15400700 1 -- 172.21.15.112:0/3738124460 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) -- 0x7f3b1019b420 con 0x7f3af40396b0 2024-04-25T18:35:20.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:20 smithi112 ceph-mon[27864]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:35:20.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:20 smithi112 ceph-mon[27864]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:21.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:21 smithi112 ceph-mon[27864]: Deploying cephadm binary to smithi161 2024-04-25T18:35:22.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:22 smithi112 ceph-mon[27864]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:23.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:23.621+0000 7f3b07400700 1 -- 172.21.15.112:0/3738124460 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f3b1019b420 con 0x7f3af40396b0 2024-04-25T18:35:23.623 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-04-25T18:35:23.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:23.625+0000 7f3b15400700 1 -- 172.21.15.112:0/3738124460 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3af40396b0 msgr2=0x7f3af403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:23.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:23.625+0000 7f3b15400700 1 --2- 172.21.15.112:0/3738124460 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3af40396b0 0x7f3af403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3afc006fd0 tx=0x7f3afc006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:23.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:23.625+0000 7f3b15400700 1 -- 172.21.15.112:0/3738124460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b10074d60 msgr2=0x7f3b1019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:23.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:23.625+0000 7f3b15400700 1 --2- 172.21.15.112:0/3738124460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b10074d60 0x7f3b1019a0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b00006010 tx=0x7f3b00005780 comp rx=0 tx=0).stop 2024-04-25T18:35:23.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:23.625+0000 7f3b15400700 1 -- 172.21.15.112:0/3738124460 shutdown_connections 2024-04-25T18:35:23.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:23.626+0000 7f3b15400700 1 --2- 172.21.15.112:0/3738124460 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3af40396b0 0x7f3af403bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:23.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:23.626+0000 7f3b15400700 1 --2- 172.21.15.112:0/3738124460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b10074d60 0x7f3b1019a0c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:23.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:23.626+0000 7f3b15400700 1 -- 172.21.15.112:0/3738124460 >> 172.21.15.112:0/3738124460 conn(0x7f3b100fdf60 msgr2=0x7f3b10106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:23.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:23.626+0000 7f3b15400700 1 -- 172.21.15.112:0/3738124460 shutdown_connections 2024-04-25T18:35:23.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:23.626+0000 7f3b15400700 1 -- 172.21.15.112:0/3738124460 wait complete. 2024-04-25T18:35:24.360 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:24 smithi112 ceph-mon[27864]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:24.360 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:24 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:24.360 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:24 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:35:24.435 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T18:35:24.861 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:35:25.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:25 smithi112 ceph-mon[27864]: Added host smithi161 2024-04-25T18:35:25.306 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:25.306 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:26.184 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:26 smithi112 ceph-mon[27864]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:27.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.013+0000 7f3d1f800700 1 -- 172.21.15.112:0/1683254341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d18101580 msgr2=0x7f3d18101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:27.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.013+0000 7f3d1f800700 1 --2- 172.21.15.112:0/1683254341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d18101580 0x7f3d18101960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009910 tx=0x7f3d00009c20 comp rx=0 tx=0).stop 2024-04-25T18:35:27.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.013+0000 7f3d1f800700 1 -- 172.21.15.112:0/1683254341 shutdown_connections 2024-04-25T18:35:27.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.013+0000 7f3d1f800700 1 --2- 172.21.15.112:0/1683254341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d18101580 0x7f3d18101960 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:27.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.013+0000 7f3d1f800700 1 -- 172.21.15.112:0/1683254341 >> 172.21.15.112:0/1683254341 conn(0x7f3d18078010 msgr2=0x7f3d180ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:27.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.013+0000 7f3d1f800700 1 -- 172.21.15.112:0/1683254341 shutdown_connections 2024-04-25T18:35:27.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.013+0000 7f3d1f800700 1 -- 172.21.15.112:0/1683254341 wait complete. 2024-04-25T18:35:27.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.014+0000 7f3d1f800700 1 Processor -- start 2024-04-25T18:35:27.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.015+0000 7f3d1f800700 1 -- start start 2024-04-25T18:35:27.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.015+0000 7f3d1f800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d18101580 0x7f3d1819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:27.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.015+0000 7f3d1f800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1810a700 con 0x7f3d18101580 2024-04-25T18:35:27.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.015+0000 7f3d1ce00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d18101580 0x7f3d1819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:27.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.015+0000 7f3d1ce00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d18101580 0x7f3d1819a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38924/0 (socket says 172.21.15.112:38924) 2024-04-25T18:35:27.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.015+0000 7f3d1ce00700 1 -- 172.21.15.112:0/764149387 learned_addr learned my addr 172.21.15.112:0/764149387 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:27.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.016+0000 7f3d1ce00700 1 -- 172.21.15.112:0/764149387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1819a630 con 0x7f3d18101580 2024-04-25T18:35:27.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.016+0000 7f3d1ce00700 1 --2- 172.21.15.112:0/764149387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d18101580 0x7f3d1819a0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3d00006010 tx=0x7f3d00005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:27.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.016+0000 7f3d16000700 1 -- 172.21.15.112:0/764149387 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d00026070 con 0x7f3d18101580 2024-04-25T18:35:27.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.016+0000 7f3d16000700 1 -- 172.21.15.112:0/764149387 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d0000bba0 con 0x7f3d18101580 2024-04-25T18:35:27.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.016+0000 7f3d1f800700 1 -- 172.21.15.112:0/764149387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1819a920 con 0x7f3d18101580 2024-04-25T18:35:27.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.016+0000 7f3d16000700 1 -- 172.21.15.112:0/764149387 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d00003910 con 0x7f3d18101580 2024-04-25T18:35:27.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.016+0000 7f3d1f800700 1 -- 172.21.15.112:0/764149387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1819e9c0 con 0x7f3d18101580 2024-04-25T18:35:27.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.017+0000 7f3d16000700 1 -- 172.21.15.112:0/764149387 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f3d00029070 con 0x7f3d18101580 2024-04-25T18:35:27.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.017+0000 7f3d1f800700 1 -- 172.21.15.112:0/764149387 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d18102a60 con 0x7f3d18101580 2024-04-25T18:35:27.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.018+0000 7f3d16000700 1 --2- 172.21.15.112:0/764149387 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3cf80396b0 0x7f3cf803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:27.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.018+0000 7f3d16000700 1 -- 172.21.15.112:0/764149387 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3d0001e070 con 0x7f3d18101580 2024-04-25T18:35:27.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.018+0000 7f3d0fe00700 1 --2- 172.21.15.112:0/764149387 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3cf80396b0 0x7f3cf803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:27.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.019+0000 7f3d0fe00700 1 --2- 172.21.15.112:0/764149387 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3cf80396b0 0x7f3cf803bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3d08006fd0 tx=0x7f3d08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:27.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.025+0000 7f3d16000700 1 -- 172.21.15.112:0/764149387 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d00013070 con 0x7f3d18101580 2024-04-25T18:35:27.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.239+0000 7f3d1f800700 1 -- 172.21.15.112:0/764149387 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d1819b450 con 0x7f3cf80396b0 2024-04-25T18:35:27.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.241+0000 7f3d16000700 1 -- 172.21.15.112:0/764149387 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f3d1819b450 con 0x7f3cf80396b0 2024-04-25T18:35:27.243 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:27.243 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-04-25T18:35:27.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.243+0000 7f3d1f800700 1 -- 172.21.15.112:0/764149387 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3cf80396b0 msgr2=0x7f3cf803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:27.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.243+0000 7f3d1f800700 1 --2- 172.21.15.112:0/764149387 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3cf80396b0 0x7f3cf803bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3d08006fd0 tx=0x7f3d08006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:27.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.243+0000 7f3d1f800700 1 -- 172.21.15.112:0/764149387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d18101580 msgr2=0x7f3d1819a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:27.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.243+0000 7f3d1f800700 1 --2- 172.21.15.112:0/764149387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d18101580 0x7f3d1819a0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3d00006010 tx=0x7f3d00005780 comp rx=0 tx=0).stop 2024-04-25T18:35:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.243+0000 7f3d1f800700 1 -- 172.21.15.112:0/764149387 shutdown_connections 2024-04-25T18:35:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.243+0000 7f3d1f800700 1 --2- 172.21.15.112:0/764149387 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3cf80396b0 0x7f3cf803bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.243+0000 7f3d1f800700 1 --2- 172.21.15.112:0/764149387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d18101580 0x7f3d1819a0f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.243+0000 7f3d1f800700 1 -- 172.21.15.112:0/764149387 >> 172.21.15.112:0/764149387 conn(0x7f3d18078010 msgr2=0x7f3d180fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.244+0000 7f3d1f800700 1 -- 172.21.15.112:0/764149387 shutdown_connections 2024-04-25T18:35:27.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:27.244+0000 7f3d1f800700 1 -- 172.21.15.112:0/764149387 wait complete. 2024-04-25T18:35:27.676 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:27 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:27.676 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:27 smithi112 ceph-mon[27864]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:28.091 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi171 2024-04-25T18:35:28.091 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T18:35:28.091 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.conf 2024-04-25T18:35:28.111 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T18:35:28.111 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:35:28.168 INFO:tasks.cephadm:Adding host smithi171 to orchestrator... 2024-04-25T18:35:28.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch host add smithi171 2024-04-25T18:35:28.605 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:35:28.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:28 smithi112 ceph-mon[27864]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:35:29.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:29 smithi112 ceph-mon[27864]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:30.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:30.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:30.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:30.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:30.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:35:30.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:35:30.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:35:30.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:30 smithi112 ceph-mon[27864]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-25T18:35:30.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:30 smithi112 ceph-mon[27864]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:35:30.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:30 smithi112 ceph-mon[27864]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:35:30.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.880+0000 7f0231200700 1 -- 172.21.15.112:0/2572325369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f022c1084b0 msgr2=0x7f022c108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:30.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.880+0000 7f0231200700 1 --2- 172.21.15.112:0/2572325369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f022c1084b0 0x7f022c108890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f021c009910 tx=0x7f021c009c20 comp rx=0 tx=0).stop 2024-04-25T18:35:30.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.880+0000 7f0231200700 1 -- 172.21.15.112:0/2572325369 shutdown_connections 2024-04-25T18:35:30.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.880+0000 7f0231200700 1 --2- 172.21.15.112:0/2572325369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f022c1084b0 0x7f022c108890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:30.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.880+0000 7f0231200700 1 -- 172.21.15.112:0/2572325369 >> 172.21.15.112:0/2572325369 conn(0x7f022c0fdf30 msgr2=0x7f022c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:30.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.881+0000 7f0231200700 1 -- 172.21.15.112:0/2572325369 shutdown_connections 2024-04-25T18:35:30.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.881+0000 7f0231200700 1 -- 172.21.15.112:0/2572325369 wait complete. 2024-04-25T18:35:30.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.882+0000 7f0231200700 1 Processor -- start 2024-04-25T18:35:30.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.882+0000 7f0231200700 1 -- start start 2024-04-25T18:35:30.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.883+0000 7f0231200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f022c1084b0 0x7f022c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:30.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.883+0000 7f0231200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f022c10c910 con 0x7f022c1084b0 2024-04-25T18:35:30.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.883+0000 7f0229e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f022c1084b0 0x7f022c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:30.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.883+0000 7f0229e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f022c1084b0 0x7f022c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51620/0 (socket says 172.21.15.112:51620) 2024-04-25T18:35:30.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.883+0000 7f0229e00700 1 -- 172.21.15.112:0/453723459 learned_addr learned my addr 172.21.15.112:0/453723459 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:30.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.883+0000 7f0229e00700 1 -- 172.21.15.112:0/453723459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022c076be0 con 0x7f022c1084b0 2024-04-25T18:35:30.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.884+0000 7f0229e00700 1 --2- 172.21.15.112:0/453723459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f022c1084b0 0x7f022c074d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f021c006010 tx=0x7f021c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:30.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.884+0000 7f0222a00700 1 -- 172.21.15.112:0/453723459 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f021c026070 con 0x7f022c1084b0 2024-04-25T18:35:30.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.884+0000 7f0222a00700 1 -- 172.21.15.112:0/453723459 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f021c00bc00 con 0x7f022c1084b0 2024-04-25T18:35:30.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.884+0000 7f0222a00700 1 -- 172.21.15.112:0/453723459 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f021c003910 con 0x7f022c1084b0 2024-04-25T18:35:30.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.884+0000 7f0231200700 1 -- 172.21.15.112:0/453723459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022c075280 con 0x7f022c1084b0 2024-04-25T18:35:30.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.884+0000 7f0231200700 1 -- 172.21.15.112:0/453723459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022c075740 con 0x7f022c1084b0 2024-04-25T18:35:30.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.885+0000 7f0222a00700 1 -- 172.21.15.112:0/453723459 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f021c029070 con 0x7f022c1084b0 2024-04-25T18:35:30.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.885+0000 7f0231200700 1 -- 172.21.15.112:0/453723459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f022c103b60 con 0x7f022c1084b0 2024-04-25T18:35:30.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.886+0000 7f0222a00700 1 --2- 172.21.15.112:0/453723459 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02100396b0 0x7f021003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:30.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.886+0000 7f0222a00700 1 -- 172.21.15.112:0/453723459 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f021c01e070 con 0x7f022c1084b0 2024-04-25T18:35:30.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.886+0000 7f0229400700 1 --2- 172.21.15.112:0/453723459 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02100396b0 0x7f021003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:30.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.887+0000 7f0229400700 1 --2- 172.21.15.112:0/453723459 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02100396b0 0x7f021003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0218006fd0 tx=0x7f0218006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:30.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:30.891+0000 7f0222a00700 1 -- 172.21.15.112:0/453723459 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f021c013070 con 0x7f022c1084b0 2024-04-25T18:35:31.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:31.120+0000 7f0231200700 1 -- 172.21.15.112:0/453723459 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}) -- 0x7f022c076270 con 0x7f02100396b0 2024-04-25T18:35:31.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:31 smithi112 ceph-mon[27864]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:35:31.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:31.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:31.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:31.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:31 smithi112 ceph-mon[27864]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:32.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:32 smithi112 ceph-mon[27864]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:35:32.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:32 smithi112 ceph-mon[27864]: Deploying cephadm binary to smithi171 2024-04-25T18:35:33.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:33 smithi112 ceph-mon[27864]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:35.723+0000 7f0222a00700 1 -- 172.21.15.112:0/453723459 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f022c076270 con 0x7f02100396b0 2024-04-25T18:35:35.725 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi171' with addr '172.21.15.171' 2024-04-25T18:35:35.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:35.727+0000 7f0231200700 1 -- 172.21.15.112:0/453723459 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02100396b0 msgr2=0x7f021003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:35.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:35.727+0000 7f0231200700 1 --2- 172.21.15.112:0/453723459 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02100396b0 0x7f021003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0218006fd0 tx=0x7f0218006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:35.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:35.727+0000 7f0231200700 1 -- 172.21.15.112:0/453723459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f022c1084b0 msgr2=0x7f022c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:35.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:35.727+0000 7f0231200700 1 --2- 172.21.15.112:0/453723459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f022c1084b0 0x7f022c074d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f021c006010 tx=0x7f021c005780 comp rx=0 tx=0).stop 2024-04-25T18:35:35.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:35.728+0000 7f0231200700 1 -- 172.21.15.112:0/453723459 shutdown_connections 2024-04-25T18:35:35.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:35.728+0000 7f0231200700 1 --2- 172.21.15.112:0/453723459 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02100396b0 0x7f021003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:35.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:35.728+0000 7f0231200700 1 --2- 172.21.15.112:0/453723459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f022c1084b0 0x7f022c074d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:35.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:35.728+0000 7f0231200700 1 -- 172.21.15.112:0/453723459 >> 172.21.15.112:0/453723459 conn(0x7f022c0fdf30 msgr2=0x7f022c10a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:35.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:35.728+0000 7f0231200700 1 -- 172.21.15.112:0/453723459 shutdown_connections 2024-04-25T18:35:35.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:35.728+0000 7f0231200700 1 -- 172.21.15.112:0/453723459 wait complete. 2024-04-25T18:35:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:36 smithi112 ceph-mon[27864]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:35:36.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T18:35:37.043 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:35:37.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:37 smithi112 ceph-mon[27864]: Added host smithi171 2024-04-25T18:35:37.192 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:37.192 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:38.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:38 smithi112 ceph-mon[27864]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.098+0000 7fbd6ae00700 1 -- 172.21.15.112:0/587482688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6410a630 msgr2=0x7fbd6410aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.098+0000 7fbd6ae00700 1 --2- 172.21.15.112:0/587482688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6410a630 0x7fbd6410aa10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009910 tx=0x7fbd54009c20 comp rx=0 tx=0).stop 2024-04-25T18:35:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.098+0000 7fbd6ae00700 1 -- 172.21.15.112:0/587482688 shutdown_connections 2024-04-25T18:35:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.098+0000 7fbd6ae00700 1 --2- 172.21.15.112:0/587482688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6410a630 0x7fbd6410aa10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.098+0000 7fbd6ae00700 1 -- 172.21.15.112:0/587482688 >> 172.21.15.112:0/587482688 conn(0x7fbd64100160 msgr2=0x7fbd64102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.098+0000 7fbd6ae00700 1 -- 172.21.15.112:0/587482688 shutdown_connections 2024-04-25T18:35:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.098+0000 7fbd6ae00700 1 -- 172.21.15.112:0/587482688 wait complete. 2024-04-25T18:35:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.099+0000 7fbd6ae00700 1 Processor -- start 2024-04-25T18:35:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.099+0000 7fbd6ae00700 1 -- start start 2024-04-25T18:35:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.099+0000 7fbd6ae00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6410a630 0x7fbd6419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.099+0000 7fbd6ae00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6410eb20 con 0x7fbd6410a630 2024-04-25T18:35:39.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.100+0000 7fbd63e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6410a630 0x7fbd6419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:39.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.100+0000 7fbd63e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6410a630 0x7fbd6419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51642/0 (socket says 172.21.15.112:51642) 2024-04-25T18:35:39.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.100+0000 7fbd63e00700 1 -- 172.21.15.112:0/2612654917 learned_addr learned my addr 172.21.15.112:0/2612654917 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:39.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.100+0000 7fbd63e00700 1 -- 172.21.15.112:0/2612654917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6419a5b0 con 0x7fbd6410a630 2024-04-25T18:35:39.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.100+0000 7fbd63e00700 1 --2- 172.21.15.112:0/2612654917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6410a630 0x7fbd6419a070 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbd54006010 tx=0x7fbd54005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:39.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.101+0000 7fbd61600700 1 -- 172.21.15.112:0/2612654917 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd54026070 con 0x7fbd6410a630 2024-04-25T18:35:39.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.101+0000 7fbd61600700 1 -- 172.21.15.112:0/2612654917 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd5400bba0 con 0x7fbd6410a630 2024-04-25T18:35:39.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.101+0000 7fbd6ae00700 1 -- 172.21.15.112:0/2612654917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6419a8a0 con 0x7fbd6410a630 2024-04-25T18:35:39.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.101+0000 7fbd61600700 1 -- 172.21.15.112:0/2612654917 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd54003910 con 0x7fbd6410a630 2024-04-25T18:35:39.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.101+0000 7fbd6ae00700 1 -- 172.21.15.112:0/2612654917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd6419e940 con 0x7fbd6410a630 2024-04-25T18:35:39.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.102+0000 7fbd61600700 1 -- 172.21.15.112:0/2612654917 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fbd54029070 con 0x7fbd6410a630 2024-04-25T18:35:39.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.102+0000 7fbd6ae00700 1 -- 172.21.15.112:0/2612654917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd64105d70 con 0x7fbd6410a630 2024-04-25T18:35:39.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.102+0000 7fbd61600700 1 --2- 172.21.15.112:0/2612654917 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbd48039650 0x7fbd4803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:39.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.102+0000 7fbd61600700 1 -- 172.21.15.112:0/2612654917 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbd5401e070 con 0x7fbd6410a630 2024-04-25T18:35:39.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.103+0000 7fbd5be00700 1 --2- 172.21.15.112:0/2612654917 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbd48039650 0x7fbd4803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:39.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.103+0000 7fbd5be00700 1 --2- 172.21.15.112:0/2612654917 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbd48039650 0x7fbd4803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c006fd0 tx=0x7fbd4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:39.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.108+0000 7fbd61600700 1 -- 172.21.15.112:0/2612654917 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd54013070 con 0x7fbd6410a630 2024-04-25T18:35:39.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.342+0000 7fbd6ae00700 1 -- 172.21.15.112:0/2612654917 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd6419b3d0 con 0x7fbd48039650 2024-04-25T18:35:39.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.345+0000 7fbd61600700 1 -- 172.21.15.112:0/2612654917 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fbd6419b3d0 con 0x7fbd48039650 2024-04-25T18:35:39.346 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:35:39.346 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}] 2024-04-25T18:35:39.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.348+0000 7fbd6ae00700 1 -- 172.21.15.112:0/2612654917 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbd48039650 msgr2=0x7fbd4803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:39.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.348+0000 7fbd6ae00700 1 --2- 172.21.15.112:0/2612654917 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbd48039650 0x7fbd4803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c006fd0 tx=0x7fbd4c006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:39.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.348+0000 7fbd6ae00700 1 -- 172.21.15.112:0/2612654917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6410a630 msgr2=0x7fbd6419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:39.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.348+0000 7fbd6ae00700 1 --2- 172.21.15.112:0/2612654917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6410a630 0x7fbd6419a070 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbd54006010 tx=0x7fbd54005780 comp rx=0 tx=0).stop 2024-04-25T18:35:39.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.348+0000 7fbd6ae00700 1 -- 172.21.15.112:0/2612654917 shutdown_connections 2024-04-25T18:35:39.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.348+0000 7fbd6ae00700 1 --2- 172.21.15.112:0/2612654917 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbd48039650 0x7fbd4803bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:39.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.348+0000 7fbd6ae00700 1 --2- 172.21.15.112:0/2612654917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd6410a630 0x7fbd6419a070 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:39.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.348+0000 7fbd6ae00700 1 -- 172.21.15.112:0/2612654917 >> 172.21.15.112:0/2612654917 conn(0x7fbd64100160 msgr2=0x7fbd64100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:39.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.348+0000 7fbd6ae00700 1 -- 172.21.15.112:0/2612654917 shutdown_connections 2024-04-25T18:35:39.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:39.348+0000 7fbd6ae00700 1 -- 172.21.15.112:0/2612654917 wait complete. 2024-04-25T18:35:39.696 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:39.697 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:39 smithi112 ceph-mon[27864]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:40.126 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T18:35:40.126 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T18:35:40.570 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:35:40.714 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:40 smithi112 ceph-mon[27864]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:35:41.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:41 smithi112 ceph-mon[27864]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:42.860 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:42.860 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:42.860 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:42.860 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:42.860 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:35:42.860 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:35:42.860 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:35:42.861 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:42 smithi112 ceph-mon[27864]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T18:35:42.861 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:42 smithi112 ceph-mon[27864]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:35:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.899+0000 7f3f4f200700 1 -- 172.21.15.112:0/3867765244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f50074e10 msgr2=0x7f3f500751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.899+0000 7f3f4f200700 1 --2- 172.21.15.112:0/3867765244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f50074e10 0x7f3f500751f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009910 tx=0x7f3f40009c20 comp rx=0 tx=0).stop 2024-04-25T18:35:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.899+0000 7f3f4f200700 1 -- 172.21.15.112:0/3867765244 shutdown_connections 2024-04-25T18:35:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.899+0000 7f3f4f200700 1 --2- 172.21.15.112:0/3867765244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f50074e10 0x7f3f500751f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.899+0000 7f3f4f200700 1 -- 172.21.15.112:0/3867765244 >> 172.21.15.112:0/3867765244 conn(0x7f3f500fdd90 msgr2=0x7f3f501001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.899+0000 7f3f4f200700 1 -- 172.21.15.112:0/3867765244 shutdown_connections 2024-04-25T18:35:42.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.899+0000 7f3f4f200700 1 -- 172.21.15.112:0/3867765244 wait complete. 2024-04-25T18:35:42.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.900+0000 7f3f4f200700 1 Processor -- start 2024-04-25T18:35:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.900+0000 7f3f4f200700 1 -- start start 2024-04-25T18:35:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.900+0000 7f3f4f200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f50074e10 0x7f3f50199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.900+0000 7f3f4f200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5010e9b0 con 0x7f3f50074e10 2024-04-25T18:35:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.901+0000 7f3f4de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f50074e10 0x7f3f50199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.901+0000 7f3f4de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f50074e10 0x7f3f50199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59004/0 (socket says 172.21.15.112:59004) 2024-04-25T18:35:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.901+0000 7f3f4de00700 1 -- 172.21.15.112:0/2267125902 learned_addr learned my addr 172.21.15.112:0/2267125902 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:35:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.901+0000 7f3f4de00700 1 -- 172.21.15.112:0/2267125902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5019a480 con 0x7f3f50074e10 2024-04-25T18:35:42.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.901+0000 7f3f4de00700 1 --2- 172.21.15.112:0/2267125902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f50074e10 0x7f3f50199f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f400056d0 tx=0x7f3f400057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:42.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.902+0000 7f3f47400700 1 -- 172.21.15.112:0/2267125902 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f40026070 con 0x7f3f50074e10 2024-04-25T18:35:42.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.902+0000 7f3f47400700 1 -- 172.21.15.112:0/2267125902 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f40004370 con 0x7f3f50074e10 2024-04-25T18:35:42.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.902+0000 7f3f47400700 1 -- 172.21.15.112:0/2267125902 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f400039b0 con 0x7f3f50074e10 2024-04-25T18:35:42.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.902+0000 7f3f4f200700 1 -- 172.21.15.112:0/2267125902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5019a710 con 0x7f3f50074e10 2024-04-25T18:35:42.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.902+0000 7f3f4f200700 1 -- 172.21.15.112:0/2267125902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5019ab70 con 0x7f3f50074e10 2024-04-25T18:35:42.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.903+0000 7f3f47400700 1 -- 172.21.15.112:0/2267125902 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f3f40029070 con 0x7f3f50074e10 2024-04-25T18:35:42.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.903+0000 7f3f4f200700 1 -- 172.21.15.112:0/2267125902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f500764c0 con 0x7f3f50074e10 2024-04-25T18:35:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.903+0000 7f3f47400700 1 --2- 172.21.15.112:0/2267125902 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3f340396b0 0x7f3f3403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.903+0000 7f3f47400700 1 -- 172.21.15.112:0/2267125902 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f4001e070 con 0x7f3f50074e10 2024-04-25T18:35:42.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.904+0000 7f3f45400700 1 --2- 172.21.15.112:0/2267125902 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3f340396b0 0x7f3f3403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:42.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.904+0000 7f3f45400700 1 --2- 172.21.15.112:0/2267125902 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3f340396b0 0x7f3f3403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:42.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:42.909+0000 7f3f47400700 1 -- 172.21.15.112:0/2267125902 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f40013070 con 0x7f3f50074e10 2024-04-25T18:35:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.147+0000 7f3f4f200700 1 -- 172.21.15.112:0/2267125902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3f5019b2a0 con 0x7f3f50074e10 2024-04-25T18:35:43.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.989+0000 7f3f47400700 1 -- 172.21.15.112:0/2267125902 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3f40013c90 con 0x7f3f50074e10 2024-04-25T18:35:43.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.993+0000 7f3f4f200700 1 -- 172.21.15.112:0/2267125902 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3f340396b0 msgr2=0x7f3f3403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:43.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.993+0000 7f3f4f200700 1 --2- 172.21.15.112:0/2267125902 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3f340396b0 0x7f3f3403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:43.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.993+0000 7f3f4f200700 1 -- 172.21.15.112:0/2267125902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f50074e10 msgr2=0x7f3f50199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:43.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.993+0000 7f3f4f200700 1 --2- 172.21.15.112:0/2267125902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f50074e10 0x7f3f50199f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f400056d0 tx=0x7f3f400057b0 comp rx=0 tx=0).stop 2024-04-25T18:35:43.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.993+0000 7f3f4f200700 1 -- 172.21.15.112:0/2267125902 shutdown_connections 2024-04-25T18:35:43.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.993+0000 7f3f4f200700 1 --2- 172.21.15.112:0/2267125902 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3f340396b0 0x7f3f3403bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:43.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.993+0000 7f3f4f200700 1 --2- 172.21.15.112:0/2267125902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f50074e10 0x7f3f50199f40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:43.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.993+0000 7f3f4f200700 1 -- 172.21.15.112:0/2267125902 >> 172.21.15.112:0/2267125902 conn(0x7f3f500fdd90 msgr2=0x7f3f50108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:43.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.994+0000 7f3f4f200700 1 -- 172.21.15.112:0/2267125902 shutdown_connections 2024-04-25T18:35:43.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:35:43.994+0000 7f3f4f200700 1 -- 172.21.15.112:0/2267125902 wait complete. 2024-04-25T18:35:43.997 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2024-04-25T18:35:44.437 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:43 smithi112 ceph-mon[27864]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:35:44.437 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:43 smithi112 ceph-mon[27864]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:35:44.437 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:43 smithi112 ceph-mon[27864]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:44.437 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:44.437 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:44.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:44.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:43 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2267125902' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T18:35:44.853 INFO:tasks.cephadm:Adding mon.a on smithi112 2024-04-25T18:35:44.853 INFO:tasks.cephadm:Adding mon.b on smithi161 2024-04-25T18:35:44.853 INFO:tasks.cephadm:Adding mon.c on smithi171 2024-04-25T18:35:44.853 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi112:172.21.15.112=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c' 2024-04-25T18:35:45.090 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:35:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:44 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2267125902' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T18:35:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:44 smithi112 ceph-mon[27864]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:35:46.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:45 smithi112 ceph-mon[27864]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:46.512 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.510+0000 7f79117c6700 1 -- 172.21.15.171:0/311274379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c108490 msgr2=0x7f790c108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:46.512 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.510+0000 7f79117c6700 1 --2- 172.21.15.171:0/311274379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c108490 0x7f790c108870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f790400b0a0 tx=0x7f790400b3b0 comp rx=0 tx=0).stop 2024-04-25T18:35:46.512 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.510+0000 7f79117c6700 1 -- 172.21.15.171:0/311274379 shutdown_connections 2024-04-25T18:35:46.512 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.510+0000 7f79117c6700 1 --2- 172.21.15.171:0/311274379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c108490 0x7f790c108870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:46.512 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.510+0000 7f79117c6700 1 -- 172.21.15.171:0/311274379 >> 172.21.15.171:0/311274379 conn(0x7f790c077a30 msgr2=0x7f790c077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:46.512 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.510+0000 7f79117c6700 1 -- 172.21.15.171:0/311274379 shutdown_connections 2024-04-25T18:35:46.512 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.510+0000 7f79117c6700 1 -- 172.21.15.171:0/311274379 wait complete. 2024-04-25T18:35:46.512 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.511+0000 7f79117c6700 1 Processor -- start 2024-04-25T18:35:46.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.511+0000 7f79117c6700 1 -- start start 2024-04-25T18:35:46.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.511+0000 7f79117c6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c108490 0x7f790c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:46.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.511+0000 7f79117c6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f790c10c880 con 0x7f790c108490 2024-04-25T18:35:46.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.512+0000 7f790affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c108490 0x7f790c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:46.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.512+0000 7f790affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c108490 0x7f790c197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.171:57418/0 (socket says 172.21.15.171:57418) 2024-04-25T18:35:46.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.512+0000 7f790affd700 1 -- 172.21.15.171:0/3510933882 learned_addr learned my addr 172.21.15.171:0/3510933882 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T18:35:46.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.512+0000 7f790affd700 1 -- 172.21.15.171:0/3510933882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c198330 con 0x7f790c108490 2024-04-25T18:35:46.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.513+0000 7f790affd700 1 --2- 172.21.15.171:0/3510933882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c108490 0x7f790c197df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f790400d010 tx=0x7f7904003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:46.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.513+0000 7f78f3fff700 1 -- 172.21.15.171:0/3510933882 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7904021020 con 0x7f790c108490 2024-04-25T18:35:46.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.513+0000 7f79117c6700 1 -- 172.21.15.171:0/3510933882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c198620 con 0x7f790c108490 2024-04-25T18:35:46.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.513+0000 7f79117c6700 1 -- 172.21.15.171:0/3510933882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c19c6c0 con 0x7f790c108490 2024-04-25T18:35:46.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.513+0000 7f78f3fff700 1 -- 172.21.15.171:0/3510933882 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79040044b0 con 0x7f790c108490 2024-04-25T18:35:46.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.513+0000 7f78f3fff700 1 -- 172.21.15.171:0/3510933882 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7904017040 con 0x7f790c108490 2024-04-25T18:35:46.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.514+0000 7f79117c6700 1 -- 172.21.15.171:0/3510933882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f8005330 con 0x7f790c108490 2024-04-25T18:35:46.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.515+0000 7f78f3fff700 1 -- 172.21.15.171:0/3510933882 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f7904024020 con 0x7f790c108490 2024-04-25T18:35:46.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.515+0000 7f78f3fff700 1 --2- 172.21.15.171:0/3510933882 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78f40396a0 0x7f78f403bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:46.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.515+0000 7f78f3fff700 1 -- 172.21.15.171:0/3510933882 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f790400f050 con 0x7f790c108490 2024-04-25T18:35:46.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.515+0000 7f790a7fc700 1 --2- 172.21.15.171:0/3510933882 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78f40396a0 0x7f78f403bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:46.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.516+0000 7f790a7fc700 1 --2- 172.21.15.171:0/3510933882 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78f40396a0 0x7f78f403bb60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7900006fd0 tx=0x7f7900006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:46.531 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.530+0000 7f78f3fff700 1 -- 172.21.15.171:0/3510933882 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f790402b030 con 0x7f790c108490 2024-04-25T18:35:46.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.739+0000 7f79117c6700 1 -- 172.21.15.171:0/3510933882 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}) -- 0x7f78f8000c80 con 0x7f78f40396a0 2024-04-25T18:35:46.754 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.753+0000 7f78f3fff700 1 -- 172.21.15.171:0/3510933882 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f78f8000c80 con 0x7f78f40396a0 2024-04-25T18:35:46.754 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mon update... 2024-04-25T18:35:46.756 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.755+0000 7f79117c6700 1 -- 172.21.15.171:0/3510933882 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78f40396a0 msgr2=0x7f78f403bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:46.756 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.755+0000 7f79117c6700 1 --2- 172.21.15.171:0/3510933882 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78f40396a0 0x7f78f403bb60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7900006fd0 tx=0x7f7900006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:46.757 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.755+0000 7f79117c6700 1 -- 172.21.15.171:0/3510933882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c108490 msgr2=0x7f790c197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:46.757 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.755+0000 7f79117c6700 1 --2- 172.21.15.171:0/3510933882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c108490 0x7f790c197df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f790400d010 tx=0x7f7904003ce0 comp rx=0 tx=0).stop 2024-04-25T18:35:46.757 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.755+0000 7f79117c6700 1 -- 172.21.15.171:0/3510933882 shutdown_connections 2024-04-25T18:35:46.757 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.755+0000 7f79117c6700 1 --2- 172.21.15.171:0/3510933882 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78f40396a0 0x7f78f403bb60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:46.757 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.755+0000 7f79117c6700 1 --2- 172.21.15.171:0/3510933882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c108490 0x7f790c197df0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:46.757 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.755+0000 7f79117c6700 1 -- 172.21.15.171:0/3510933882 >> 172.21.15.171:0/3510933882 conn(0x7f790c077a30 msgr2=0x7f790c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:46.757 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.755+0000 7f79117c6700 1 -- 172.21.15.171:0/3510933882 shutdown_connections 2024-04-25T18:35:46.757 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:46.755+0000 7f79117c6700 1 -- 172.21.15.171:0/3510933882 wait complete. 2024-04-25T18:35:47.333 DEBUG:teuthology.orchestra.run.smithi161:mon.b> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T18:35:47.336 DEBUG:teuthology.orchestra.run.smithi171:mon.c> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T18:35:47.338 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T18:35:47.339 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T18:35:48.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:47 smithi112 ceph-mon[27864]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:35:48.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:47 smithi112 ceph-mon[27864]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c;count:3 2024-04-25T18:35:48.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:48.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:35:48.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:35:48.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:35:48.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:48.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:35:48.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:35:48.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:47 smithi112 ceph-mon[27864]: Deploying daemon mon.c on smithi171 2024-04-25T18:35:48.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:47 smithi112 ceph-mon[27864]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:49.004 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:35:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:50 smithi112 ceph-mon[27864]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.104 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 podman[33221]: 2024-04-25 18:35:51.019253521 +0000 UTC m=+0.329381685 container create c5d70d8d788b976837fd4bf53d574e3ab2f2792ac8c6a01dfa19b7797beac6ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:35:51.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.205+0000 7f8e399de700 1 -- 172.21.15.171:0/2627111042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e34101580 msgr2=0x7f8e34101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:51.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.205+0000 7f8e399de700 1 --2- 172.21.15.171:0/2627111042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e34101580 0x7f8e34101960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009910 tx=0x7f8e24009c20 comp rx=0 tx=0).stop 2024-04-25T18:35:51.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.205+0000 7f8e399de700 1 -- 172.21.15.171:0/2627111042 shutdown_connections 2024-04-25T18:35:51.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.205+0000 7f8e399de700 1 --2- 172.21.15.171:0/2627111042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e34101580 0x7f8e34101960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:51.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.205+0000 7f8e399de700 1 -- 172.21.15.171:0/2627111042 >> 172.21.15.171:0/2627111042 conn(0x7f8e34078010 msgr2=0x7f8e340ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:51.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.205+0000 7f8e399de700 1 -- 172.21.15.171:0/2627111042 shutdown_connections 2024-04-25T18:35:51.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.205+0000 7f8e399de700 1 -- 172.21.15.171:0/2627111042 wait complete. 2024-04-25T18:35:51.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.206+0000 7f8e399de700 1 Processor -- start 2024-04-25T18:35:51.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.206+0000 7f8e399de700 1 -- start start 2024-04-25T18:35:51.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.206+0000 7f8e399de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e34101580 0x7f8e34196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:51.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.206+0000 7f8e399de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3410b800 con 0x7f8e34101580 2024-04-25T18:35:51.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.207+0000 7f8e32ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e34101580 0x7f8e34196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:51.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.207+0000 7f8e32ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e34101580 0x7f8e34196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.171:57434/0 (socket says 172.21.15.171:57434) 2024-04-25T18:35:51.208 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.207+0000 7f8e32ffd700 1 -- 172.21.15.171:0/4207365789 learned_addr learned my addr 172.21.15.171:0/4207365789 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T18:35:51.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.207+0000 7f8e32ffd700 1 -- 172.21.15.171:0/4207365789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e34197310 con 0x7f8e34101580 2024-04-25T18:35:51.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.208+0000 7f8e32ffd700 1 --2- 172.21.15.171:0/4207365789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e34101580 0x7f8e34196dd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e24006010 tx=0x7f8e24005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:51.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.208+0000 7f8e389dc700 1 -- 172.21.15.171:0/4207365789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e24026070 con 0x7f8e34101580 2024-04-25T18:35:51.209 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.208+0000 7f8e399de700 1 -- 172.21.15.171:0/4207365789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e34197600 con 0x7f8e34101580 2024-04-25T18:35:51.210 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.208+0000 7f8e399de700 1 -- 172.21.15.171:0/4207365789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3419b6a0 con 0x7f8e34101580 2024-04-25T18:35:51.210 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.208+0000 7f8e389dc700 1 -- 172.21.15.171:0/4207365789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e2400bba0 con 0x7f8e34101580 2024-04-25T18:35:51.210 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.208+0000 7f8e389dc700 1 -- 172.21.15.171:0/4207365789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e24030060 con 0x7f8e34101580 2024-04-25T18:35:51.210 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.209+0000 7f8e399de700 1 -- 172.21.15.171:0/4207365789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e34102a60 con 0x7f8e34101580 2024-04-25T18:35:51.210 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.209+0000 7f8e389dc700 1 -- 172.21.15.171:0/4207365789 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f8e24029070 con 0x7f8e34101580 2024-04-25T18:35:51.211 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.209+0000 7f8e389dc700 1 --2- 172.21.15.171:0/4207365789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e1c039650 0x7f8e1c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:51.211 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.210+0000 7f8e389dc700 1 -- 172.21.15.171:0/4207365789 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8e2401e070 con 0x7f8e34101580 2024-04-25T18:35:51.211 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.210+0000 7f8e327fc700 1 --2- 172.21.15.171:0/4207365789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e1c039650 0x7f8e1c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:51.212 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.211+0000 7f8e327fc700 1 --2- 172.21.15.171:0/4207365789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e1c039650 0x7f8e1c03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8e28006fd0 tx=0x7f8e28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:51.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.225+0000 7f8e389dc700 1 -- 172.21.15.171:0/4207365789 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e24013070 con 0x7f8e34101580 2024-04-25T18:35:51.377 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 podman[33221]: 2024-04-25 18:35:51.376097616 +0000 UTC m=+0.686225741 container init c5d70d8d788b976837fd4bf53d574e3ab2f2792ac8c6a01dfa19b7797beac6ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-25T18:35:51.446 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.445+0000 7f8e389dc700 1 -- 172.21.15.171:0/4207365789 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8e2400e3e0 con 0x7f8e34101580 2024-04-25T18:35:51.528 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:51.527+0000 7f8e399de700 1 -- 172.21.15.171:0/4207365789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8e34198130 con 0x7f8e34101580 2024-04-25T18:35:51.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 podman[33221]: 2024-04-25 18:35:51.38003977 +0000 UTC m=+0.690167893 container start c5d70d8d788b976837fd4bf53d574e3ab2f2792ac8c6a01dfa19b7797beac6ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T18:35:51.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T18:35:51.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T18:35:51.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pidfile_write: ignore empty --pid-file 2024-04-25T18:35:51.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: load: jerasure load: lrc 2024-04-25T18:35:51.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: RocksDB version: 7.9.2 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Git sha 0 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: DB SUMMARY 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: DB Session ID: OKLAEZ5Z0YFPHICWWZ43 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: CURRENT file: CURRENT 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.error_if_exists: 0 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.create_if_missing: 0 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.paranoid_checks: 1 2024-04-25T18:35:51.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.env: 0x558061005cc0 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.info_log: 0x558063511be0 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.statistics: (nil) 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.use_fsync: 0 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_log_file_size: 0 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T18:35:51.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.allow_fallocate: 1 2024-04-25T18:35:51.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T18:35:51.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T18:35:51.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.use_direct_reads: 0 2024-04-25T18:35:51.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T18:35:51.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T18:35:51.686 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.db_log_dir: 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.wal_dir: 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.write_buffer_manager: 0x5580634a50e0 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T18:35:51.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.unordered_write: 0 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.row_cache: None 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.wal_filter: None 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.two_write_queues: 0 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T18:35:51.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.wal_compression: 0 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.atomic_flush: 0 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.log_readahead_size: 0 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_background_jobs: 2 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_background_compactions: -1 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_subcompactions: 1 2024-04-25T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_open_files: -1 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_background_flushes: -1 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Compression algorithms supported: 2024-04-25T18:35:51.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: kZSTD supported: 0 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: kXpressCompression supported: 0 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: kZlibCompression supported: 1 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: kSnappyCompression supported: 1 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: kLZ4Compression supported: 1 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: kBZip2Compression supported: 0 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T18:35:51.691 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.merge_operator: 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_filter: None 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558063511dc0) 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks: 1 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_top_level_index_and_filter: 1 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_type: 0 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_index_type: 0 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_shortening: 1 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T18:35:51.692 INFO:journalctl@ceph.mon.c.smithi171.stdout: checksum: 4 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: no_block_cache: 0 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache: 0x5580634d71f0 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_name: BinnedLRUCache 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_options: 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: capacity : 536870912 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_shard_bits : 4 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: strict_capacity_limit : 0 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: high_pri_pool_ratio: 0.000 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_compressed: (nil) 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: persistent_cache: (nil) 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size: 4096 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size_deviation: 10 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_restart_interval: 16 2024-04-25T18:35:51.693 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_block_restart_interval: 1 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: metadata_block_size: 4096 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: partition_filters: 0 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: use_delta_encoding: 1 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: filter_policy: bloomfilter 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: whole_key_filtering: 1 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: verify_compression: 0 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: read_amp_bytes_per_bit: 0 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: format_version: 5 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_index_compression: 1 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_align: 0 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: max_auto_readahead_size: 262144 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: prepopulate_block_cache: 0 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: initial_auto_readahead_size: 8192 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T18:35:51.694 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compression: NoCompression 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.num_levels: 7 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T18:35:51.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T18:35:51.696 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T18:35:51.697 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.inplace_update_support: 0 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T18:35:51.698 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.bloom_locality: 0 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.max_successive_merges: 0 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.ttl: 2592000 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.enable_blob_files: false 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.min_blob_size: 0 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T18:35:51.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c97fda0a-53bd-4e59-bf3d-f724efa3768d 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070151410021, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070151410439, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714070151, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c97fda0a-53bd-4e59-bf3d-f724efa3768d", "db_session_id": "OKLAEZ5Z0YFPHICWWZ43", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070151410520, "job": 1, "event": "recovery_finished"} 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5580635ec000 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: DB pointer 0x5580635d6000 2024-04-25T18:35:51.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** DB Stats ** 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T18:35:51.701 INFO:journalctl@ceph.mon.c.smithi171.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T18:35:51.702 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache BinnedLRUCache@0x5580634d71f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: using public_addr v2:172.21.15.171:0/0 -> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: starting mon.c rank -1 at public addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] at bind addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(???) e0 preinit fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout: e1 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T18:35:51.703 INFO:journalctl@ceph.mon.c.smithi171.stdout: legacy client fscid: -1 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout: No filesystems configured 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:35:51.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Added host smithi161 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:35:51.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Deploying cephadm binary to smithi171 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Added host smithi171 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:35:51.707 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2267125902' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2267125902' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c;count:3 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:51.708 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:35:51.709 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:35:51.709 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: Deploying daemon mon.c on smithi171 2024-04-25T18:35:51.709 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.709 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:51.709 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 ceph-mon[33266]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T18:35:51.709 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 bash[33221]: c5d70d8d788b976837fd4bf53d574e3ab2f2792ac8c6a01dfa19b7797beac6ee 2024-04-25T18:35:51.709 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:51 smithi171 systemd[1]: Started Ceph mon.c for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:35:56.474 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:56.472+0000 7f8e389dc700 1 -- 172.21.15.171:0/4207365789 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f8e2402a3b0 con 0x7f8e34101580 2024-04-25T18:35:56.474 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:35:56.474 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","modified":"2024-04-25T18:35:51.442246Z","created":"2024-04-25T18:33:33.098667Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T18:35:56.477 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:56.475+0000 7f8e399de700 1 -- 172.21.15.171:0/4207365789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e1c039650 msgr2=0x7f8e1c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:56.477 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:56.475+0000 7f8e399de700 1 --2- 172.21.15.171:0/4207365789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e1c039650 0x7f8e1c03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8e28006fd0 tx=0x7f8e28006e40 comp rx=0 tx=0).stop 2024-04-25T18:35:56.477 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:56.475+0000 7f8e399de700 1 -- 172.21.15.171:0/4207365789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e34101580 msgr2=0x7f8e34196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:56.477 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:56.475+0000 7f8e399de700 1 --2- 172.21.15.171:0/4207365789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e34101580 0x7f8e34196dd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e24006010 tx=0x7f8e24005780 comp rx=0 tx=0).stop 2024-04-25T18:35:56.477 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:56.476+0000 7f8e399de700 1 -- 172.21.15.171:0/4207365789 shutdown_connections 2024-04-25T18:35:56.477 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:56.476+0000 7f8e399de700 1 --2- 172.21.15.171:0/4207365789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e1c039650 0x7f8e1c03bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:56.477 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:56.476+0000 7f8e399de700 1 --2- 172.21.15.171:0/4207365789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e34101580 0x7f8e34196dd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:56.477 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:56.476+0000 7f8e399de700 1 -- 172.21.15.171:0/4207365789 >> 172.21.15.171:0/4207365789 conn(0x7f8e34078010 msgr2=0x7f8e34102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:56.477 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:56.476+0000 7f8e399de700 1 -- 172.21.15.171:0/4207365789 shutdown_connections 2024-04-25T18:35:56.478 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:56.476+0000 7f8e399de700 1 -- 172.21.15.171:0/4207365789 wait complete. 2024-04-25T18:35:56.478 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2024-04-25T18:35:56.782 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:35:56.782 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: mon.a calling monitor election 2024-04-25T18:35:56.782 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: from='client.? 172.21.15.171:0/4207365789' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: mon.c calling monitor election 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: monmap epoch 2 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: last_changed 2024-04-25T18:35:51.442246+0000 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: created 2024-04-25T18:33:33.098667+0000 2024-04-25T18:35:56.783 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: min_mon_release 19 (squid) 2024-04-25T18:35:56.784 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: election_strategy: 1 2024-04-25T18:35:56.784 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T18:35:56.784 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T18:35:56.784 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: fsmap 2024-04-25T18:35:56.784 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:35:56.784 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: mgrmap e13: x(active, since 61s) 2024-04-25T18:35:56.784 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: overall HEALTH_OK 2024-04-25T18:35:56.784 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:56.784 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:56.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: mon.a calling monitor election 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: from='client.? 172.21.15.171:0/4207365789' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: mon.c calling monitor election 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: monmap epoch 2 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:35:56.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: last_changed 2024-04-25T18:35:51.442246+0000 2024-04-25T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: created 2024-04-25T18:33:33.098667+0000 2024-04-25T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: min_mon_release 19 (squid) 2024-04-25T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: election_strategy: 1 2024-04-25T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: fsmap 2024-04-25T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: mgrmap e13: x(active, since 61s) 2024-04-25T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: overall HEALTH_OK 2024-04-25T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:56.790 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:57.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:57.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:35:57.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:35:57.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:57 smithi171 ceph-mon[33266]: Deploying daemon mon.b on smithi161 2024-04-25T18:35:57.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:57 smithi171 ceph-mon[33266]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:57.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:35:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:57.819 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:35:57.819 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:35:57.820 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:35:57.820 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:57 smithi112 ceph-mon[27864]: Deploying daemon mon.b on smithi161 2024-04-25T18:35:57.820 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:57 smithi112 ceph-mon[27864]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:35:57.820 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:35:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:35:58.094 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T18:35:58.094 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T18:35:58.329 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:35:58.723 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:35:58 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:35:58.441+0000 7f6e39400700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T18:35:59.666 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.664+0000 7ff238fc8700 1 -- 172.21.15.171:0/110638551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff234108620 msgr2=0x7ff234108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:59.666 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.664+0000 7ff238fc8700 1 --2- 172.21.15.171:0/110638551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff234108620 0x7ff234108a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff228009910 tx=0x7ff228009c20 comp rx=0 tx=0).stop 2024-04-25T18:35:59.666 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.664+0000 7ff238fc8700 1 -- 172.21.15.171:0/110638551 shutdown_connections 2024-04-25T18:35:59.666 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.664+0000 7ff238fc8700 1 --2- 172.21.15.171:0/110638551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff234108620 0x7ff234108a00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:59.666 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.664+0000 7ff238fc8700 1 -- 172.21.15.171:0/110638551 >> 172.21.15.171:0/110638551 conn(0x7ff2340fe060 msgr2=0x7ff234100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:59.666 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.664+0000 7ff238fc8700 1 -- 172.21.15.171:0/110638551 shutdown_connections 2024-04-25T18:35:59.666 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.664+0000 7ff238fc8700 1 -- 172.21.15.171:0/110638551 wait complete. 2024-04-25T18:35:59.668 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.665+0000 7ff238fc8700 1 Processor -- start 2024-04-25T18:35:59.668 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.665+0000 7ff238fc8700 1 -- start start 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.665+0000 7ff238fc8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff234108620 0x7ff234198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.665+0000 7ff238fc8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2341986c0 0x7ff23419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.665+0000 7ff238fc8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff23410ca90 con 0x7ff234108620 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.665+0000 7ff238fc8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff23410c910 con 0x7ff2341986c0 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.666+0000 7ff231d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2341986c0 0x7ff23419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.666+0000 7ff231d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2341986c0 0x7ff23419ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:48646/0 (socket says 172.21.15.171:48646) 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.666+0000 7ff231d9b700 1 -- 172.21.15.171:0/424170993 learned_addr learned my addr 172.21.15.171:0/424170993 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.666+0000 7ff231d9b700 1 -- 172.21.15.171:0/424170993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2341986c0 msgr2=0x7ff23419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.666+0000 7ff231d9b700 1 -- 172.21.15.171:0/424170993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2341986c0 msgr2=0x7ff23419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.666+0000 7ff231d9b700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2341986c0 0x7ff23419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.666+0000 7ff231d9b700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2341986c0 0x7ff23419ca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.666+0000 7ff23259c700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff234108620 0x7ff234198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.666+0000 7ff23259c700 1 -- 172.21.15.171:0/424170993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2341986c0 msgr2=0x7ff23419ca50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:35:59.669 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.666+0000 7ff23259c700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2341986c0 0x7ff23419ca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:59.670 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.666+0000 7ff23259c700 1 -- 172.21.15.171:0/424170993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff234198d00 con 0x7ff234108620 2024-04-25T18:35:59.670 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.667+0000 7ff23259c700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff234108620 0x7ff234198180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff228014000 tx=0x7ff228005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:59.670 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.667+0000 7ff2237fe700 1 -- 172.21.15.171:0/424170993 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff228005dc0 con 0x7ff234108620 2024-04-25T18:35:59.670 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.667+0000 7ff238fc8700 1 -- 172.21.15.171:0/424170993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23419cf90 con 0x7ff234108620 2024-04-25T18:35:59.670 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.667+0000 7ff238fc8700 1 -- 172.21.15.171:0/424170993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23419d490 con 0x7ff234108620 2024-04-25T18:35:59.670 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.667+0000 7ff2237fe700 1 -- 172.21.15.171:0/424170993 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff228004710 con 0x7ff234108620 2024-04-25T18:35:59.670 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.667+0000 7ff2237fe700 1 -- 172.21.15.171:0/424170993 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff22800bd80 con 0x7ff234108620 2024-04-25T18:35:59.671 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.668+0000 7ff238fc8700 1 -- 172.21.15.171:0/424170993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff214005330 con 0x7ff234108620 2024-04-25T18:35:59.671 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.669+0000 7ff2237fe700 1 -- 172.21.15.171:0/424170993 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7ff228029070 con 0x7ff234108620 2024-04-25T18:35:59.671 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.669+0000 7ff2237fe700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff21c0393f0 0x7ff21c03b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:35:59.671 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.669+0000 7ff2237fe700 1 -- 172.21.15.171:0/424170993 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff22801e070 con 0x7ff234108620 2024-04-25T18:35:59.671 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.669+0000 7ff231d9b700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff21c0393f0 0x7ff21c03b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:35:59.671 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.670+0000 7ff231d9b700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff21c0393f0 0x7ff21c03b8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff224009180 tx=0x7ff224008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:35:59.687 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.687+0000 7ff2237fe700 1 -- 172.21.15.171:0/424170993 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff228013070 con 0x7ff234108620 2024-04-25T18:35:59.969 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.968+0000 7ff238fc8700 1 -- 172.21.15.171:0/424170993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff214005620 con 0x7ff234108620 2024-04-25T18:35:59.970 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.969+0000 7ff2237fe700 1 -- 172.21.15.171:0/424170993 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7ff228017600 con 0x7ff234108620 2024-04-25T18:35:59.971 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:35:59.971 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","modified":"2024-04-25T18:35:51.442246Z","created":"2024-04-25T18:33:33.098667Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T18:35:59.973 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.972+0000 7ff238fc8700 1 -- 172.21.15.171:0/424170993 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff21c0393f0 msgr2=0x7ff21c03b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:59.973 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.972+0000 7ff238fc8700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff21c0393f0 0x7ff21c03b8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff224009180 tx=0x7ff224008040 comp rx=0 tx=0).stop 2024-04-25T18:35:59.973 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.972+0000 7ff238fc8700 1 -- 172.21.15.171:0/424170993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff234108620 msgr2=0x7ff234198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:35:59.973 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.972+0000 7ff238fc8700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff234108620 0x7ff234198180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff228014000 tx=0x7ff228005030 comp rx=0 tx=0).stop 2024-04-25T18:35:59.973 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.972+0000 7ff238fc8700 1 -- 172.21.15.171:0/424170993 shutdown_connections 2024-04-25T18:35:59.973 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.972+0000 7ff238fc8700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff21c0393f0 0x7ff21c03b8b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:59.973 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.972+0000 7ff238fc8700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff234108620 0x7ff234198180 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:59.973 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.972+0000 7ff238fc8700 1 --2- 172.21.15.171:0/424170993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2341986c0 0x7ff23419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:35:59.973 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.972+0000 7ff238fc8700 1 -- 172.21.15.171:0/424170993 >> 172.21.15.171:0/424170993 conn(0x7ff2340fe060 msgr2=0x7ff2340fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:35:59.974 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.972+0000 7ff238fc8700 1 -- 172.21.15.171:0/424170993 shutdown_connections 2024-04-25T18:35:59.974 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:35:59.972+0000 7ff238fc8700 1 -- 172.21.15.171:0/424170993 wait complete. 2024-04-25T18:35:59.974 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2024-04-25T18:36:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:00 smithi171 ceph-mon[33266]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:00 smithi171 ceph-mon[33266]: from='client.? 172.21.15.171:0/424170993' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:36:00.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:00 smithi112 ceph-mon[27864]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:00.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:00 smithi112 ceph-mon[27864]: from='client.? 172.21.15.171:0/424170993' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:36:01.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:00 smithi161 systemd[1]: Started Ceph mon.b for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:36:01.548 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T18:36:01.548 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T18:36:01.793 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:36:06.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.071+0000 7fbaa6712700 1 -- 172.21.15.171:0/3982286924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba880035c0 msgr2=0x7fba88005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:06.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.071+0000 7fbaa6712700 1 --2- 172.21.15.171:0/3982286924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba880035c0 0x7fba88005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbaa006ad30 tx=0x7fba9000feb0 comp rx=0 tx=0).stop 2024-04-25T18:36:06.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.071+0000 7fbaa6712700 1 -- 172.21.15.171:0/3982286924 shutdown_connections 2024-04-25T18:36:06.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.071+0000 7fbaa6712700 1 --2- 172.21.15.171:0/3982286924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba880035c0 0x7fba88005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:06.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.071+0000 7fbaa6712700 1 -- 172.21.15.171:0/3982286924 >> 172.21.15.171:0/3982286924 conn(0x7fbaa00fcf40 msgr2=0x7fbaa00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:36:06.074 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.071+0000 7fbaa6712700 1 -- 172.21.15.171:0/3982286924 shutdown_connections 2024-04-25T18:36:06.074 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.071+0000 7fbaa6712700 1 -- 172.21.15.171:0/3982286924 wait complete. 2024-04-25T18:36:06.075 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.072+0000 7fbaa6712700 1 Processor -- start 2024-04-25T18:36:06.075 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.072+0000 7fbaa6712700 1 -- start start 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa6712700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba880035c0 0x7fbaa01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa6712700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbaa0198730 0x7fbaa019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa6712700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa019d000 0x7fbaa019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa6712700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa010a450 con 0x7fba880035c0 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa6712700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa010a2d0 con 0x7fbaa0198730 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa6712700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa010a5d0 con 0x7fbaa019d000 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa4caf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa019d000 0x7fbaa019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa4caf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa019d000 0x7fbaa019d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:48648/0 (socket says 172.21.15.171:48648) 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa4caf700 1 -- 172.21.15.171:0/2656963893 learned_addr learned my addr 172.21.15.171:0/2656963893 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fba9f7fe700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbaa0198730 0x7fbaa019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa4caf700 1 -- 172.21.15.171:0/2656963893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa019d000 msgr2=0x7fbaa019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa4caf700 1 -- 172.21.15.171:0/2656963893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa019d000 msgr2=0x7fbaa019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa4caf700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa019d000 0x7fbaa019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T18:36:06.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fbaa4caf700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa019d000 0x7fbaa019d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:36:06.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fba9ffff700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba880035c0 0x7fbaa01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:06.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fba9f7fe700 1 -- 172.21.15.171:0/2656963893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbaa0198730 msgr2=0x7fbaa019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T18:36:06.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fba9f7fe700 1 -- 172.21.15.171:0/2656963893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbaa0198730 msgr2=0x7fbaa019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T18:36:06.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fba9f7fe700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbaa0198730 0x7fbaa019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T18:36:06.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.073+0000 7fba9f7fe700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbaa0198730 0x7fbaa019cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:36:06.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.074+0000 7fba9ffff700 1 -- 172.21.15.171:0/2656963893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa019d000 msgr2=0x7fbaa019d4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:36:06.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.074+0000 7fba9ffff700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa019d000 0x7fbaa019d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:06.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.074+0000 7fba9ffff700 1 -- 172.21.15.171:0/2656963893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbaa0198730 msgr2=0x7fbaa019cac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:36:06.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.074+0000 7fba9ffff700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbaa0198730 0x7fbaa019cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:06.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.074+0000 7fba9ffff700 1 -- 172.21.15.171:0/2656963893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa01a4d70 con 0x7fba880035c0 2024-04-25T18:36:06.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.074+0000 7fba9ffff700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba880035c0 0x7fbaa01981f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba90005ed0 tx=0x7fba90013960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:36:06.078 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.075+0000 7fba9d7fa700 1 -- 172.21.15.171:0/2656963893 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90029030 con 0x7fba880035c0 2024-04-25T18:36:06.078 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.075+0000 7fbaa6712700 1 -- 172.21.15.171:0/2656963893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa01a4f40 con 0x7fba880035c0 2024-04-25T18:36:06.078 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.075+0000 7fbaa6712700 1 -- 172.21.15.171:0/2656963893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa01a53f0 con 0x7fba880035c0 2024-04-25T18:36:06.079 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.075+0000 7fba9d7fa700 1 -- 172.21.15.171:0/2656963893 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba900044e0 con 0x7fba880035c0 2024-04-25T18:36:06.079 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.075+0000 7fba9d7fa700 1 -- 172.21.15.171:0/2656963893 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90003880 con 0x7fba880035c0 2024-04-25T18:36:06.079 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.076+0000 7fbaa6712700 1 -- 172.21.15.171:0/2656963893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa00640a0 con 0x7fba880035c0 2024-04-25T18:36:06.079 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.076+0000 7fba9d7fa700 1 -- 172.21.15.171:0/2656963893 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fba9002c070 con 0x7fba880035c0 2024-04-25T18:36:06.079 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.076+0000 7fba9d7fa700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fba8c039ae0 0x7fba8c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:06.079 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.076+0000 7fba9d7fa700 1 -- 172.21.15.171:0/2656963893 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fba9004eb40 con 0x7fba880035c0 2024-04-25T18:36:06.079 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.077+0000 7fba9f7fe700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fba8c039ae0 0x7fba8c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:06.079 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.078+0000 7fba9f7fe700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fba8c039ae0 0x7fba8c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fba94009850 tx=0x7fba94008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:36:06.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.092+0000 7fba9d7fa700 1 -- 172.21.15.171:0/2656963893 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba90033070 con 0x7fba880035c0 2024-04-25T18:36:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: mon.c calling monitor election 2024-04-25T18:36:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:36:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: mon.a calling monitor election 2024-04-25T18:36:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: monmap epoch 3 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: last_changed 2024-04-25T18:36:00.851988+0000 2024-04-25T18:36:06.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: created 2024-04-25T18:33:33.098667+0000 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: min_mon_release 19 (squid) 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: election_strategy: 1 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: fsmap 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: mgrmap e13: x(active, since 70s) 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: overall HEALTH_OK 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:06.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:06.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: mon.c calling monitor election 2024-04-25T18:36:06.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:36:06.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: mon.a calling monitor election 2024-04-25T18:36:06.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: monmap epoch 3 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: last_changed 2024-04-25T18:36:00.851988+0000 2024-04-25T18:36:06.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: created 2024-04-25T18:33:33.098667+0000 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: min_mon_release 19 (squid) 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: election_strategy: 1 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: fsmap 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: mgrmap e13: x(active, since 70s) 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: overall HEALTH_OK 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:06.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:06.390 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.389+0000 7fbaa6712700 1 -- 172.21.15.171:0/2656963893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbaa002df80 con 0x7fba880035c0 2024-04-25T18:36:06.391 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.389+0000 7fba9d7fa700 1 -- 172.21.15.171:0/2656963893 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7fba90023c20 con 0x7fba880035c0 2024-04-25T18:36:06.391 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T18:36:06.391 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":3,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","modified":"2024-04-25T18:36:00.851988Z","created":"2024-04-25T18:33:33.098667Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 -- 172.21.15.171:0/2656963893 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fba8c039ae0 msgr2=0x7fba8c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fba8c039ae0 0x7fba8c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fba94009850 tx=0x7fba94008040 comp rx=0 tx=0).stop 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 -- 172.21.15.171:0/2656963893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba880035c0 msgr2=0x7fbaa01981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba880035c0 0x7fbaa01981f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba90005ed0 tx=0x7fba90013960 comp rx=0 tx=0).stop 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 -- 172.21.15.171:0/2656963893 shutdown_connections 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fba8c039ae0 0x7fba8c03bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa019d000 0x7fbaa019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba880035c0 0x7fbaa01981f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 --2- 172.21.15.171:0/2656963893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbaa0198730 0x7fbaa019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 -- 172.21.15.171:0/2656963893 >> 172.21.15.171:0/2656963893 conn(0x7fbaa00fcf40 msgr2=0x7fbaa0107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 -- 172.21.15.171:0/2656963893 shutdown_connections 2024-04-25T18:36:06.394 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:06.392+0000 7fbaa6712700 1 -- 172.21.15.171:0/2656963893 wait complete. 2024-04-25T18:36:06.395 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 3 2024-04-25T18:36:06.951 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T18:36:06.951 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T18:36:07.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:07.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:06 smithi171 ceph-mon[33266]: from='client.? 172.21.15.171:0/2656963893' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:36:07.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:07.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:07.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:06 smithi112 ceph-mon[27864]: from='client.? 172.21.15.171:0/2656963893' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:36:07.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:07.370 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:36:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: mon.c calling monitor election 2024-04-25T18:36:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:36:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: mon.a calling monitor election 2024-04-25T18:36:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: monmap epoch 3 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:36:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: last_changed 2024-04-25T18:36:00.851988+0000 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: created 2024-04-25T18:33:33.098667+0000 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: min_mon_release 19 (squid) 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: election_strategy: 1 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: fsmap 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: mgrmap e13: x(active, since 70s) 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: overall HEALTH_OK 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:08.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='client.? 172.21.15.171:0/2656963893' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:36:08.224 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:09.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: mon.b calling monitor election 2024-04-25T18:36:09.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: mon.b calling monitor election 2024-04-25T18:36:09.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: mon.c calling monitor election 2024-04-25T18:36:09.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: mon.a calling monitor election 2024-04-25T18:36:09.000 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: monmap epoch 3 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: last_changed 2024-04-25T18:36:00.851988+0000 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: created 2024-04-25T18:33:33.098667+0000 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: min_mon_release 19 (squid) 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: election_strategy: 1 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: fsmap 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: mgrmap e13: x(active, since 72s) 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: overall HEALTH_OK 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T18:36:09.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-25T18:36:09.002 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T18:36:09.002 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.002 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.002 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.002 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: mon.b calling monitor election 2024-04-25T18:36:09.198 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: mon.b calling monitor election 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: mon.c calling monitor election 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: mon.a calling monitor election 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: monmap epoch 3 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: last_changed 2024-04-25T18:36:00.851988+0000 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: created 2024-04-25T18:33:33.098667+0000 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: min_mon_release 19 (squid) 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: election_strategy: 1 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: fsmap 2024-04-25T18:36:09.199 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:36:09.200 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: mgrmap e13: x(active, since 72s) 2024-04-25T18:36:09.200 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: overall HEALTH_OK 2024-04-25T18:36:09.200 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T18:36:09.200 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-25T18:36:09.200 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T18:36:09.200 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.200 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.200 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.200 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: mon.b calling monitor election 2024-04-25T18:36:09.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: mon.b calling monitor election 2024-04-25T18:36:09.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: mon.c calling monitor election 2024-04-25T18:36:09.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: mon.a calling monitor election 2024-04-25T18:36:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T18:36:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: monmap epoch 3 2024-04-25T18:36:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:36:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: last_changed 2024-04-25T18:36:00.851988+0000 2024-04-25T18:36:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: created 2024-04-25T18:33:33.098667+0000 2024-04-25T18:36:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: min_mon_release 19 (squid) 2024-04-25T18:36:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: election_strategy: 1 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: 2: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: fsmap 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: mgrmap e13: x(active, since 72s) 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: overall HEALTH_OK 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.224 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.224 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.514+0000 7ff21e000700 1 -- 172.21.15.112:0/2621381098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2181035e0 msgr2=0x7ff2181039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.514+0000 7ff21e000700 1 --2- 172.21.15.112:0/2621381098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2181035e0 0x7ff2181039c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff208009910 tx=0x7ff208009c20 comp rx=0 tx=0).stop 2024-04-25T18:36:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.515+0000 7ff21e000700 1 -- 172.21.15.112:0/2621381098 shutdown_connections 2024-04-25T18:36:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.515+0000 7ff21e000700 1 --2- 172.21.15.112:0/2621381098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2181035e0 0x7ff2181039c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.515+0000 7ff21e000700 1 -- 172.21.15.112:0/2621381098 >> 172.21.15.112:0/2621381098 conn(0x7ff218078010 msgr2=0x7ff2180ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:36:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.515+0000 7ff21e000700 1 -- 172.21.15.112:0/2621381098 shutdown_connections 2024-04-25T18:36:09.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.515+0000 7ff21e000700 1 -- 172.21.15.112:0/2621381098 wait complete. 2024-04-25T18:36:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.516+0000 7ff21e000700 1 Processor -- start 2024-04-25T18:36:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.516+0000 7ff21e000700 1 -- start start 2024-04-25T18:36:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.516+0000 7ff21e000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2181035e0 0x7ff218198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.516+0000 7ff21e000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff218198690 0x7ff21819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:09.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.516+0000 7ff21e000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff21819cf60 0x7ff21819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.516+0000 7ff21e000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff21810c980 con 0x7ff2181035e0 2024-04-25T18:36:09.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.516+0000 7ff21e000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff21810c800 con 0x7ff218198690 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.516+0000 7ff21e000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff21810cb00 con 0x7ff21819cf60 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.517+0000 7ff217400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2181035e0 0x7ff218198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.517+0000 7ff217400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2181035e0 0x7ff218198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54028/0 (socket says 172.21.15.112:54028) 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.517+0000 7ff217400700 1 -- 172.21.15.112:0/3553806020 learned_addr learned my addr 172.21.15.112:0/3553806020 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.517+0000 7ff216a00700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff218198690 0x7ff21819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.517+0000 7ff217e00700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff21819cf60 0x7ff21819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.517+0000 7ff217400700 1 -- 172.21.15.112:0/3553806020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff21819cf60 msgr2=0x7ff21819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.517+0000 7ff217400700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff21819cf60 0x7ff21819d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.517+0000 7ff217400700 1 -- 172.21.15.112:0/3553806020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff218198690 msgr2=0x7ff21819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.517+0000 7ff217400700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff218198690 0x7ff21819ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.517+0000 7ff217400700 1 -- 172.21.15.112:0/3553806020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff218101170 con 0x7ff2181035e0 2024-04-25T18:36:09.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.518+0000 7ff217400700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2181035e0 0x7ff218198150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff208005b40 tx=0x7ff208004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:36:09.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.518+0000 7ff20fe00700 1 -- 172.21.15.112:0/3553806020 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208027070 con 0x7ff2181035e0 2024-04-25T18:36:09.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.518+0000 7ff21e000700 1 -- 172.21.15.112:0/3553806020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff218101400 con 0x7ff2181035e0 2024-04-25T18:36:09.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.518+0000 7ff21e000700 1 -- 172.21.15.112:0/3553806020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff218101930 con 0x7ff2181035e0 2024-04-25T18:36:09.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.518+0000 7ff20fe00700 1 -- 172.21.15.112:0/3553806020 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2080047d0 con 0x7ff2181035e0 2024-04-25T18:36:09.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.518+0000 7ff20fe00700 1 -- 172.21.15.112:0/3553806020 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208022a70 con 0x7ff2181035e0 2024-04-25T18:36:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.520+0000 7ff21e000700 1 -- 172.21.15.112:0/3553806020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f8005330 con 0x7ff2181035e0 2024-04-25T18:36:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.520+0000 7ff20fe00700 1 -- 172.21.15.112:0/3553806020 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7ff20802a070 con 0x7ff2181035e0 2024-04-25T18:36:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.520+0000 7ff20fe00700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff1fc039ae0 0x7ff1fc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:09.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.520+0000 7ff20fe00700 1 -- 172.21.15.112:0/3553806020 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff20804de40 con 0x7ff2181035e0 2024-04-25T18:36:09.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.521+0000 7ff216a00700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff1fc039ae0 0x7ff1fc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:09.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.522+0000 7ff216a00700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff1fc039ae0 0x7ff1fc03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff204009850 tx=0x7ff204008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:36:09.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.527+0000 7ff20fe00700 1 -- 172.21.15.112:0/3553806020 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff208052070 con 0x7ff2181035e0 2024-04-25T18:36:09.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.764+0000 7ff21e000700 1 -- 172.21.15.112:0/3553806020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff1f8005620 con 0x7ff2181035e0 2024-04-25T18:36:09.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.765+0000 7ff20fe00700 1 -- 172.21.15.112:0/3553806020 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7ff208031020 con 0x7ff2181035e0 2024-04-25T18:36:09.766 INFO:teuthology.orchestra.run.smithi112.stdout:# minimal ceph.conf for 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:36:09.767 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-04-25T18:36:09.767 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:36:09.767 INFO:teuthology.orchestra.run.smithi112.stdout: mon_host = [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-04-25T18:36:09.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.768+0000 7ff21e000700 1 -- 172.21.15.112:0/3553806020 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff1fc039ae0 msgr2=0x7ff1fc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:09.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.768+0000 7ff21e000700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff1fc039ae0 0x7ff1fc03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff204009850 tx=0x7ff204008040 comp rx=0 tx=0).stop 2024-04-25T18:36:09.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.768+0000 7ff21e000700 1 -- 172.21.15.112:0/3553806020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2181035e0 msgr2=0x7ff218198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:09.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.768+0000 7ff21e000700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2181035e0 0x7ff218198150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff208005b40 tx=0x7ff208004f80 comp rx=0 tx=0).stop 2024-04-25T18:36:09.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.768+0000 7ff21e000700 1 -- 172.21.15.112:0/3553806020 shutdown_connections 2024-04-25T18:36:09.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.768+0000 7ff21e000700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff1fc039ae0 0x7ff1fc03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.768+0000 7ff21e000700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff21819cf60 0x7ff21819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.768+0000 7ff21e000700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2181035e0 0x7ff218198150 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.768+0000 7ff21e000700 1 --2- 172.21.15.112:0/3553806020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff218198690 0x7ff21819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.768+0000 7ff21e000700 1 -- 172.21.15.112:0/3553806020 >> 172.21.15.112:0/3553806020 conn(0x7ff218078010 msgr2=0x7ff2181063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:36:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.768+0000 7ff21e000700 1 -- 172.21.15.112:0/3553806020 shutdown_connections 2024-04-25T18:36:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:09.769+0000 7ff21e000700 1 -- 172.21.15.112:0/3553806020 wait complete. 2024-04-25T18:36:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:36:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:36:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: Updating smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:36:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:36:10.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:36:10.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:10.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:09 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3553806020' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:36:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:36:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: Updating smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:36:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:36:10.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:36:10.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:10.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:09 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3553806020' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:36:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:36:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: Updating smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:36:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:36:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:36:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:36:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:09 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3553806020' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:10.637 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T18:36:10.637 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:36:10.637 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T18:36:10.723 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:36:10.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:36:10.849 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-25T18:36:10.850 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T18:36:10.878 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-25T18:36:10.878 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:36:10.943 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T18:36:10.943 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T18:36:10.971 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T18:36:10.972 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:36:11.036 INFO:tasks.cephadm:Adding mgr.x on smithi112 2024-04-25T18:36:11.036 INFO:tasks.cephadm:Adding mgr.y on smithi161 2024-04-25T18:36:11.036 INFO:tasks.cephadm:Adding mgr.z on smithi171 2024-04-25T18:36:11.037 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi112=x;smithi161=y;smithi171=z' 2024-04-25T18:36:11.214 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:11 smithi112 ceph-mon[27864]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T18:36:11.214 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:11 smithi112 ceph-mon[27864]: Reconfiguring daemon mon.a on smithi112 2024-04-25T18:36:11.214 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:11.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:11 smithi161 ceph-mon[32609]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T18:36:11.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:11 smithi161 ceph-mon[32609]: Reconfiguring daemon mon.a on smithi112 2024-04-25T18:36:11.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:11 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:11.305 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:36:11.425 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:11 smithi171 ceph-mon[33266]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T18:36:11.425 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:11 smithi171 ceph-mon[33266]: Reconfiguring daemon mon.a on smithi112 2024-04-25T18:36:11.425 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:11 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:12.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:12 smithi112 ceph-mon[27864]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:12.398 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:12 smithi171 ceph-mon[33266]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:12 smithi161 ceph-mon[32609]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:12.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.855+0000 7f8e5f51d700 1 -- 172.21.15.171:0/2029068917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e581014c0 msgr2=0x7f8e581018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:12.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.855+0000 7f8e5f51d700 1 --2- 172.21.15.171:0/2029068917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e581014c0 0x7f8e581018e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009910 tx=0x7f8e48009c20 comp rx=0 tx=0).stop 2024-04-25T18:36:12.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.855+0000 7f8e5f51d700 1 -- 172.21.15.171:0/2029068917 shutdown_connections 2024-04-25T18:36:12.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.855+0000 7f8e5f51d700 1 --2- 172.21.15.171:0/2029068917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e581014c0 0x7f8e581018e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:12.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.855+0000 7f8e5f51d700 1 -- 172.21.15.171:0/2029068917 >> 172.21.15.171:0/2029068917 conn(0x7f8e580fca80 msgr2=0x7f8e580feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:36:12.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.855+0000 7f8e5f51d700 1 -- 172.21.15.171:0/2029068917 shutdown_connections 2024-04-25T18:36:12.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.855+0000 7f8e5f51d700 1 -- 172.21.15.171:0/2029068917 wait complete. 2024-04-25T18:36:12.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.855+0000 7f8e5f51d700 1 Processor -- start 2024-04-25T18:36:12.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.855+0000 7f8e5f51d700 1 -- start start 2024-04-25T18:36:12.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5f51d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e581014c0 0x7f8e580fe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5f51d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e580fd020 0x7f8e580fd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5f51d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e580fd9e0 0x7f8e581a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5f51d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58107400 con 0x7f8e580fd020 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5f51d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58107280 con 0x7f8e580fd9e0 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5f51d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58107580 con 0x7f8e581014c0 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5d2b9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e581014c0 0x7f8e580fe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5d2b9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e581014c0 0x7f8e580fe9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:45390/0 (socket says 172.21.15.171:45390) 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5d2b9700 1 -- 172.21.15.171:0/499050857 learned_addr learned my addr 172.21.15.171:0/499050857 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5daba700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e580fd9e0 0x7f8e581a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5d2b9700 1 -- 172.21.15.171:0/499050857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e580fd9e0 msgr2=0x7f8e581a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5d2b9700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e580fd9e0 0x7f8e581a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5d2b9700 1 -- 172.21.15.171:0/499050857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e580fd020 msgr2=0x7f8e580fd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5d2b9700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e580fd020 0x7f8e580fd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5d2b9700 1 -- 172.21.15.171:0/499050857 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e581a2310 con 0x7f8e581014c0 2024-04-25T18:36:12.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5daba700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e580fd9e0 0x7f8e581a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:36:12.860 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5d2b9700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e581014c0 0x7f8e580fe9d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8e48005e00 tx=0x7f8e48009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:36:12.860 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e4e7fc700 1 -- 172.21.15.171:0/499050857 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48027070 con 0x7f8e581014c0 2024-04-25T18:36:12.860 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e4e7fc700 1 -- 172.21.15.171:0/499050857 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e4800be10 con 0x7f8e581014c0 2024-04-25T18:36:12.860 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5f51d700 1 -- 172.21.15.171:0/499050857 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e581a25a0 con 0x7f8e581014c0 2024-04-25T18:36:12.860 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e4e7fc700 1 -- 172.21.15.171:0/499050857 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4800f460 con 0x7f8e581014c0 2024-04-25T18:36:12.860 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.856+0000 7f8e5f51d700 1 -- 172.21.15.171:0/499050857 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e581a2b00 con 0x7f8e581014c0 2024-04-25T18:36:12.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.857+0000 7f8e4e7fc700 1 -- 172.21.15.171:0/499050857 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f8e4802a070 con 0x7f8e581014c0 2024-04-25T18:36:12.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.858+0000 7f8e4e7fc700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e44039ae0 0x7f8e4403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:12.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.858+0000 7f8e4e7fc700 1 -- 172.21.15.171:0/499050857 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8e4804db60 con 0x7f8e581014c0 2024-04-25T18:36:12.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.858+0000 7f8e5f51d700 1 -- 172.21.15.171:0/499050857 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e580640a0 con 0x7f8e581014c0 2024-04-25T18:36:12.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.858+0000 7f8e5cab8700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e44039ae0 0x7f8e4403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:12.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.859+0000 7f8e5cab8700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e44039ae0 0x7f8e4403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8e540098e0 tx=0x7f8e54006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:36:12.864 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:12.863+0000 7f8e4e7fc700 1 -- 172.21.15.171:0/499050857 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e48031070 con 0x7f8e581014c0 2024-04-25T18:36:13.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.072+0000 7f8e5f51d700 1 -- 172.21.15.171:0/499050857 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi161=y;smithi171=z", "target": ["mon-mgr", ""]}) -- 0x7f8e58062e80 con 0x7f8e44039ae0 2024-04-25T18:36:13.086 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.085+0000 7f8e4e7fc700 1 -- 172.21.15.171:0/499050857 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8e58062e80 con 0x7f8e44039ae0 2024-04-25T18:36:13.087 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mgr update... 2024-04-25T18:36:13.089 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.087+0000 7f8e5f51d700 1 -- 172.21.15.171:0/499050857 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e44039ae0 msgr2=0x7f8e4403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:13.089 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.087+0000 7f8e5f51d700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e44039ae0 0x7f8e4403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8e540098e0 tx=0x7f8e54006dd0 comp rx=0 tx=0).stop 2024-04-25T18:36:13.089 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.087+0000 7f8e5f51d700 1 -- 172.21.15.171:0/499050857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e581014c0 msgr2=0x7f8e580fe9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:13.089 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.087+0000 7f8e5f51d700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e581014c0 0x7f8e580fe9d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8e48005e00 tx=0x7f8e48009e60 comp rx=0 tx=0).stop 2024-04-25T18:36:13.089 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.087+0000 7f8e5f51d700 1 -- 172.21.15.171:0/499050857 shutdown_connections 2024-04-25T18:36:13.089 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.087+0000 7f8e5f51d700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8e44039ae0 0x7f8e4403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:13.089 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.087+0000 7f8e5f51d700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e580fd9e0 0x7f8e581a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:13.089 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.087+0000 7f8e5f51d700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e581014c0 0x7f8e580fe9d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:13.089 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.087+0000 7f8e5f51d700 1 --2- 172.21.15.171:0/499050857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e580fd020 0x7f8e580fd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:13.090 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.087+0000 7f8e5f51d700 1 -- 172.21.15.171:0/499050857 >> 172.21.15.171:0/499050857 conn(0x7f8e580fca80 msgr2=0x7f8e5806b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:36:13.090 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.087+0000 7f8e5f51d700 1 -- 172.21.15.171:0/499050857 shutdown_connections 2024-04-25T18:36:13.090 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:36:13.088+0000 7f8e5f51d700 1 -- 172.21.15.171:0/499050857 wait complete. 2024-04-25T18:36:13.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:13 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:13.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:13 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:13.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:13 smithi112 ceph-mon[27864]: Reconfiguring mon.b (monmap changed)... 2024-04-25T18:36:13.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:13 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:36:13.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:13 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:36:13.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:13 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:13.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:13 smithi112 ceph-mon[27864]: Reconfiguring daemon mon.b on smithi161 2024-04-25T18:36:13.317 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:13 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:13.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:13 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:13.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:13 smithi161 ceph-mon[32609]: Reconfiguring mon.b (monmap changed)... 2024-04-25T18:36:13.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:13 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:36:13.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:13 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:36:13.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:13 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:13.318 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:13 smithi161 ceph-mon[32609]: Reconfiguring daemon mon.b on smithi161 2024-04-25T18:36:13.428 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:13 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:13.428 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:13 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:13.428 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:13 smithi171 ceph-mon[33266]: Reconfiguring mon.b (monmap changed)... 2024-04-25T18:36:13.428 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:13 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:36:13.428 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:13 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:36:13.428 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:13 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:13.428 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:13 smithi171 ceph-mon[33266]: Reconfiguring daemon mon.b on smithi161 2024-04-25T18:36:13.678 DEBUG:teuthology.orchestra.run.smithi161:mgr.y> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T18:36:13.681 DEBUG:teuthology.orchestra.run.smithi171:mgr.z> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T18:36:13.683 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T18:36:13.684 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:36:13.684 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T18:36:13.743 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T18:36:13.743 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-04-25T18:36:13.788 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T18:36:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:13.789 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-25T18:36:13.789 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:13.789 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:13.789 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 18:35:09.110830494 +0000 2024-04-25T18:36:13.789 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 18:30:09.286903620 +0000 2024-04-25T18:36:13.789 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 18:30:09.286903620 +0000 2024-04-25T18:36:13.789 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 18:30:09.286903620 +0000 2024-04-25T18:36:13.789 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T18:36:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T18:36:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T18:36:13.913 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000386182 s, 1.3 MB/s 2024-04-25T18:36:13.917 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T18:36:13.970 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-04-25T18:36:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T18:36:14.021 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:14.021 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-25T18:36:14.021 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:14.022 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:14.022 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 18:35:09.175828801 +0000 2024-04-25T18:36:14.022 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 18:30:09.861888524 +0000 2024-04-25T18:36:14.022 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 18:30:09.861888524 +0000 2024-04-25T18:36:14.022 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 18:30:09.861888524 +0000 2024-04-25T18:36:14.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T18:36:14.105 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T18:36:14.105 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T18:36:14.105 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000398996 s, 1.3 MB/s 2024-04-25T18:36:14.108 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T18:36:14.160 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-04-25T18:36:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T18:36:14.205 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:14.205 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-25T18:36:14.205 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:14.206 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:14.206 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 18:35:09.243827030 +0000 2024-04-25T18:36:14.206 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 18:30:10.469872562 +0000 2024-04-25T18:36:14.206 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 18:30:10.469872562 +0000 2024-04-25T18:36:14.206 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 18:30:10.469872562 +0000 2024-04-25T18:36:14.206 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T18:36:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:14 smithi112 ceph-mon[27864]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:14 smithi112 ceph-mon[27864]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi161=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:36:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:14 smithi112 ceph-mon[27864]: Saving service mgr spec with placement smithi112=x;smithi161=y;smithi171=z;count:3 2024-04-25T18:36:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:36:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:36:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:14.277 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T18:36:14.277 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T18:36:14.277 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000298739 s, 1.7 MB/s 2024-04-25T18:36:14.281 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T18:36:14.379 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-04-25T18:36:14.402 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:14 smithi171 ceph-mon[33266]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:14.402 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:14 smithi171 ceph-mon[33266]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi161=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:36:14.402 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:14 smithi171 ceph-mon[33266]: Saving service mgr spec with placement smithi112=x;smithi161=y;smithi171=z;count:3 2024-04-25T18:36:14.403 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:14.403 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:14.403 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:14.403 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:36:14.403 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:36:14.403 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T18:36:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:14.431 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-25T18:36:14.431 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:14.431 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:14.431 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 18:35:09.314825180 +0000 2024-04-25T18:36:14.431 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 18:30:11.087856336 +0000 2024-04-25T18:36:14.431 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 18:30:11.087856336 +0000 2024-04-25T18:36:14.431 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 18:30:11.087856336 +0000 2024-04-25T18:36:14.432 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T18:36:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:14 smithi161 ceph-mon[32609]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:14 smithi161 ceph-mon[32609]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi161=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:36:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:14 smithi161 ceph-mon[32609]: Saving service mgr spec with placement smithi112=x;smithi161=y;smithi171=z;count:3 2024-04-25T18:36:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:36:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:36:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:14.504 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T18:36:14.504 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T18:36:14.504 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.00038305 s, 1.3 MB/s 2024-04-25T18:36:14.507 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T18:36:14.561 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-04-25T18:36:14.562 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T18:36:14.581 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T18:36:14.581 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-04-25T18:36:14.639 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T18:36:14.639 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:14.639 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T18:36:14.639 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:14.639 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:14.639 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-04-25 18:36:00.225809099 +0000 2024-04-25T18:36:14.639 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-04-25 18:29:05.966841694 +0000 2024-04-25T18:36:14.639 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-04-25 18:29:05.966841694 +0000 2024-04-25T18:36:14.639 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-04-25 18:29:05.966841694 +0000 2024-04-25T18:36:14.639 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T18:36:14.705 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-04-25T18:36:14.706 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-04-25T18:36:14.706 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000240209 s, 2.1 MB/s 2024-04-25T18:36:14.707 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T18:36:14.767 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-04-25T18:36:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T18:36:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:14.823 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T18:36:14.824 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:14.824 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:14.824 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-04-25 18:36:00.225809099 +0000 2024-04-25T18:36:14.824 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-04-25 18:29:06.280834601 +0000 2024-04-25T18:36:14.824 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-04-25 18:29:06.280834601 +0000 2024-04-25T18:36:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-04-25 18:29:06.280834601 +0000 2024-04-25T18:36:14.824 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T18:36:14.888 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-04-25T18:36:14.889 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-04-25T18:36:14.889 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000188354 s, 2.7 MB/s 2024-04-25T18:36:14.890 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T18:36:14.947 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-04-25T18:36:15.002 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T18:36:15.003 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:15.003 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T18:36:15.003 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:15.003 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:15.003 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-04-25 18:36:00.225809099 +0000 2024-04-25T18:36:15.003 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-04-25 18:29:06.610827146 +0000 2024-04-25T18:36:15.003 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-04-25 18:29:06.610827146 +0000 2024-04-25T18:36:15.003 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-04-25 18:29:06.610827146 +0000 2024-04-25T18:36:15.003 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T18:36:15.066 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-04-25T18:36:15.066 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-04-25T18:36:15.066 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000228366 s, 2.2 MB/s 2024-04-25T18:36:15.068 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T18:36:15.124 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-04-25T18:36:15.179 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T18:36:15.179 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:15.179 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T18:36:15.179 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:15.179 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:15.179 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-04-25 18:36:00.226809077 +0000 2024-04-25T18:36:15.179 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-04-25 18:29:06.956819329 +0000 2024-04-25T18:36:15.179 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-04-25 18:29:06.956819329 +0000 2024-04-25T18:36:15.179 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-04-25 18:29:06.956819329 +0000 2024-04-25T18:36:15.180 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T18:36:15.242 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-04-25T18:36:15.242 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-04-25T18:36:15.242 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000198591 s, 2.6 MB/s 2024-04-25T18:36:15.243 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T18:36:15.300 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T18:36:15.300 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T18:36:15.315 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T18:36:15.315 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-04-25T18:36:15.371 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T18:36:15.371 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:15.371 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T18:36:15.371 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:15.371 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:15.371 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 18:35:50.374782002 +0000 2024-04-25T18:36:15.371 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 18:29:05.957966961 +0000 2024-04-25T18:36:15.371 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 18:29:05.957966961 +0000 2024-04-25T18:36:15.371 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 18:29:05.957966961 +0000 2024-04-25T18:36:15.371 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T18:36:15.428 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:15 smithi171 ceph-mon[33266]: Reconfiguring mon.c (monmap changed)... 2024-04-25T18:36:15.428 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:15 smithi171 ceph-mon[33266]: Reconfiguring daemon mon.c on smithi171 2024-04-25T18:36:15.435 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T18:36:15.435 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T18:36:15.435 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000188747 s, 2.7 MB/s 2024-04-25T18:36:15.436 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T18:36:15.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:15 smithi161 ceph-mon[32609]: Reconfiguring mon.c (monmap changed)... 2024-04-25T18:36:15.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:15 smithi161 ceph-mon[32609]: Reconfiguring daemon mon.c on smithi171 2024-04-25T18:36:15.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:15 smithi112 ceph-mon[27864]: Reconfiguring mon.c (monmap changed)... 2024-04-25T18:36:15.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:15 smithi112 ceph-mon[27864]: Reconfiguring daemon mon.c on smithi171 2024-04-25T18:36:15.491 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-04-25T18:36:15.546 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T18:36:15.546 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:15.546 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T18:36:15.546 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:15.546 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:15.546 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 18:35:50.374782002 +0000 2024-04-25T18:36:15.546 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 18:29:06.299960020 +0000 2024-04-25T18:36:15.546 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 18:29:06.299960020 +0000 2024-04-25T18:36:15.546 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 18:29:06.299960020 +0000 2024-04-25T18:36:15.546 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T18:36:15.608 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T18:36:15.608 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T18:36:15.608 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.00025641 s, 2.0 MB/s 2024-04-25T18:36:15.610 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T18:36:15.665 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-04-25T18:36:15.719 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T18:36:15.719 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:15.720 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T18:36:15.720 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:15.720 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:15.720 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 18:35:50.374782002 +0000 2024-04-25T18:36:15.720 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 18:29:06.620953505 +0000 2024-04-25T18:36:15.720 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 18:29:06.620953505 +0000 2024-04-25T18:36:15.720 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 18:29:06.620953505 +0000 2024-04-25T18:36:15.720 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T18:36:15.783 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T18:36:15.783 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T18:36:15.783 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000214551 s, 2.4 MB/s 2024-04-25T18:36:15.784 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T18:36:15.839 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-04-25T18:36:15.894 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T18:36:15.894 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:36:15.894 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T18:36:15.894 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:36:15.895 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:36:15.895 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 18:35:50.375781982 +0000 2024-04-25T18:36:15.895 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 18:29:06.961946585 +0000 2024-04-25T18:36:15.895 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 18:29:06.961946585 +0000 2024-04-25T18:36:15.895 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 18:29:06.961946585 +0000 2024-04-25T18:36:15.895 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T18:36:15.957 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T18:36:15.957 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T18:36:15.957 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000230719 s, 2.2 MB/s 2024-04-25T18:36:15.958 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T18:36:16.014 INFO:tasks.cephadm:Deploying osd.0 on smithi112 with /dev/vg_nvme/lv_4... 2024-04-25T18:36:16.014 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T18:36:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:16 smithi171 ceph-mon[33266]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:36:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:36:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:36:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:16.444 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:36:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:16 smithi161 ceph-mon[32609]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:36:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:36:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:36:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:16 smithi112 ceph-mon[27864]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:36:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:36:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:36:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:17.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:17 smithi112 ceph-mon[27864]: Deploying daemon mgr.z on smithi171 2024-04-25T18:36:17.372 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:17 smithi161 ceph-mon[32609]: Deploying daemon mgr.z on smithi171 2024-04-25T18:36:17.416 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:17 smithi171 ceph-mon[33266]: Deploying daemon mgr.z on smithi171 2024-04-25T18:36:17.417 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:17 smithi171 systemd[1]: Starting Ceph mgr.z for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:36:17.682 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:17 smithi171 podman[35128]: 2024-04-25 18:36:17.318782295 +0000 UTC m=+0.017755585 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:36:17.682 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:17 smithi171 podman[35128]: 2024-04-25 18:36:17.417640206 +0000 UTC m=+0.116613525 container create 1dd5024bb2eba322c3e22796ac4c36e119692a2a8eb4a662bbbc8946c0335b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-25T18:36:17.682 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:17 smithi171 podman[35128]: 2024-04-25 18:36:17.628841081 +0000 UTC m=+0.327814358 container init 1dd5024bb2eba322c3e22796ac4c36e119692a2a8eb4a662bbbc8946c0335b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:36:17.683 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:17 smithi171 podman[35128]: 2024-04-25 18:36:17.631348203 +0000 UTC m=+0.330321487 container start 1dd5024bb2eba322c3e22796ac4c36e119692a2a8eb4a662bbbc8946c0335b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T18:36:17.683 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:17 smithi171 ceph-mgr[35147]: -- 172.21.15.171:0/1135193998 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c1f4305a0 con 0x558c1f404c00 2024-04-25T18:36:17.980 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:17 smithi171 bash[35128]: 1dd5024bb2eba322c3e22796ac4c36e119692a2a8eb4a662bbbc8946c0335b3b 2024-04-25T18:36:17.980 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:17 smithi171 systemd[1]: Started Ceph mgr.z for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:36:17.981 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:17 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:17.794+0000 7fcc771ee200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:36:17.981 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:17 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:17.979+0000 7fcc771ee200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:36:18.392 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:18.081+0000 7fcc771ee200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:36:18.392 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:18.391+0000 7fcc771ee200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:36:18.393 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:18 smithi171 ceph-mon[33266]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:18.393 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:18 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:18.393 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:18 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:18.393 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:18 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:18.393 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:18 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:36:18.393 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:18 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:36:18.393 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:18 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:36:18.393 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:18 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:18.468 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:18 smithi161 ceph-mon[32609]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:18.468 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:18 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:18.468 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:18 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:18.469 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:18 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:18.469 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:18 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:36:18.469 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:18 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:36:18.469 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:18 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:36:18.469 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:18 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:18 smithi112 ceph-mon[27864]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:18.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:18.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:18.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:18.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:36:18.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:36:18.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:36:18.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:18.682 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:18.475+0000 7fcc771ee200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:36:19.407 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:19 smithi112 ceph-mon[27864]: Deploying daemon mgr.y on smithi161 2024-04-25T18:36:19.408 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T18:36:19.408 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T18:36:19.408 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T18:36:19.408 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T18:36:19.408 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0357676 s, 293 MB/s 2024-04-25T18:36:19.408 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T18:36:19.422 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:19 smithi161 ceph-mon[32609]: Deploying daemon mgr.y on smithi161 2024-04-25T18:36:19.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:19 smithi171 ceph-mon[33266]: Deploying daemon mgr.y on smithi161 2024-04-25T18:36:19.932 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:19.660+0000 7fcc771ee200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:36:19.932 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:19.828+0000 7fcc771ee200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:36:20.287 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:20 smithi171 ceph-mon[33266]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:20.287 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:19.999+0000 7fcc771ee200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:36:20.287 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:20.285+0000 7fcc771ee200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:36:20.295 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-04-25T18:36:20.370 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:20 smithi112 ceph-mon[27864]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:20.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:20 smithi161 ceph-mon[32609]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:20.471 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:20.352+0000 7fc258bd1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:36:20.586 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:20.379+0000 7fcc771ee200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:36:20.586 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:20.585+0000 7fcc771ee200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:36:20.720 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:36:20.932 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:20.787+0000 7fcc771ee200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:36:20.971 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:20.538+0000 7fc258bd1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:36:20.971 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:20.643+0000 7fc258bd1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:36:21.326 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:20.972+0000 7fc258bd1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:36:21.326 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:21.069+0000 7fc258bd1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:36:21.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:21.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:21 smithi112 ceph-mon[27864]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:21.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:21.641 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:21 smithi161 ceph-mon[32609]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:21.649 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:21.419+0000 7fcc771ee200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:36:21.649 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.649 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.649 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.649 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:21.649 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:21.649 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:21 smithi171 ceph-mon[33266]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:21.932 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:21.861+0000 7fcc771ee200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:36:22.432 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:21.960+0000 7fcc771ee200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:36:22.432 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:22.161+0000 7fcc771ee200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:36:22.695 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:22.354+0000 7fc258bd1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:36:22.696 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:22.522+0000 7fc258bd1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:36:22.696 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:22.694+0000 7fc258bd1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:36:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.820+0000 7f177d600700 1 -- 172.21.15.112:0/1554973852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1778102aa0 msgr2=0x7f1778102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.820+0000 7f177d600700 1 --2- 172.21.15.112:0/1554973852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1778102aa0 0x7f1778102ec0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f17680099c0 tx=0x7f1768009cd0 comp rx=0 tx=0).stop 2024-04-25T18:36:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.820+0000 7f177d600700 1 -- 172.21.15.112:0/1554973852 shutdown_connections 2024-04-25T18:36:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.820+0000 7f177d600700 1 --2- 172.21.15.112:0/1554973852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1778104640 0x7f177810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.820+0000 7f177d600700 1 --2- 172.21.15.112:0/1554973852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1778102aa0 0x7f1778102ec0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.820+0000 7f177d600700 1 --2- 172.21.15.112:0/1554973852 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1778103ca0 0x7f1778104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:22.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.820+0000 7f177d600700 1 -- 172.21.15.112:0/1554973852 >> 172.21.15.112:0/1554973852 conn(0x7f17780fe040 msgr2=0x7f1778100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:36:22.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.820+0000 7f177d600700 1 -- 172.21.15.112:0/1554973852 shutdown_connections 2024-04-25T18:36:22.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.820+0000 7f177d600700 1 -- 172.21.15.112:0/1554973852 wait complete. 2024-04-25T18:36:22.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.821+0000 7f177d600700 1 Processor -- start 2024-04-25T18:36:22.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.821+0000 7f177d600700 1 -- start start 2024-04-25T18:36:22.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f177d600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1778102aa0 0x7f177819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:22.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f177d600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1778103ca0 0x7f177819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:22.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f177d600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1778104640 0x7f17781a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f177d600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f177810e560 con 0x7f1778104640 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f177d600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f177810e3e0 con 0x7f1778103ca0 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f177d600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f177810e6e0 con 0x7f1778102aa0 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f1776a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1778102aa0 0x7f177819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f1776a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1778102aa0 0x7f177819ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:36196/0 (socket says 172.21.15.112:36196) 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f1776a00700 1 -- 172.21.15.112:0/3532318576 learned_addr learned my addr 172.21.15.112:0/3532318576 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f1776000700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1778103ca0 0x7f177819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f1777400700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1778104640 0x7f17781a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f1776000700 1 -- 172.21.15.112:0/3532318576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1778102aa0 msgr2=0x7f177819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f1776000700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1778102aa0 0x7f177819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f1776000700 1 -- 172.21.15.112:0/3532318576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1778104640 msgr2=0x7f17781a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:22.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f1776000700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1778104640 0x7f17781a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:22.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.822+0000 7f1776000700 1 -- 172.21.15.112:0/3532318576 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17781a2ab0 con 0x7f1778103ca0 2024-04-25T18:36:22.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.823+0000 7f1776000700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1778103ca0 0x7f177819b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f176400e9f0 tx=0x7f176400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:36:22.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.823+0000 7f176f400700 1 -- 172.21.15.112:0/3532318576 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176400cb70 con 0x7f1778103ca0 2024-04-25T18:36:22.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.823+0000 7f176f400700 1 -- 172.21.15.112:0/3532318576 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176400ee80 con 0x7f1778103ca0 2024-04-25T18:36:22.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.823+0000 7f176f400700 1 -- 172.21.15.112:0/3532318576 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764003d40 con 0x7f1778103ca0 2024-04-25T18:36:22.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.823+0000 7f177d600700 1 -- 172.21.15.112:0/3532318576 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17781a2da0 con 0x7f1778103ca0 2024-04-25T18:36:22.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.823+0000 7f177d600700 1 -- 172.21.15.112:0/3532318576 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17781a3260 con 0x7f1778103ca0 2024-04-25T18:36:22.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.824+0000 7f176f400700 1 -- 172.21.15.112:0/3532318576 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f1764018430 con 0x7f1778103ca0 2024-04-25T18:36:22.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.824+0000 7f177d600700 1 -- 172.21.15.112:0/3532318576 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1778069760 con 0x7f1778103ca0 2024-04-25T18:36:22.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.825+0000 7f176f400700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f175c039890 0x7f175c03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:22.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.825+0000 7f176f400700 1 -- 172.21.15.112:0/3532318576 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f176404cd30 con 0x7f1778103ca0 2024-04-25T18:36:22.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.825+0000 7f1776a00700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f175c039890 0x7f175c03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:22.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.826+0000 7f1776a00700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f175c039890 0x7f175c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1768003820 tx=0x7f1768005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:36:22.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:22.831+0000 7f176f400700 1 -- 172.21.15.112:0/3532318576 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1778069760 con 0x7f1778103ca0 2024-04-25T18:36:22.939 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:22.687+0000 7fcc771ee200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:36:22.939 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:22.779+0000 7fcc771ee200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:36:22.939 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:22.939 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:22.939 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:22.939 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:22.939 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:22.939 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:22.940 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:22.940 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:36:22.940 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:36:22.940 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:22.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:22.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:22.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:22.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:22.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:22.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:22.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:22.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:36:22.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:36:22.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:23.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:23.090+0000 7f177d600700 1 -- 172.21.15.112:0/3532318576 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1778076390 con 0x7f175c039890 2024-04-25T18:36:23.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:23.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:23.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:23.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:23.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:23.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:23.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:23.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:36:23.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:36:23.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:23.249 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:23.022+0000 7fcc771ee200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:36:23.249 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:23.154+0000 7fcc771ee200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:36:23.249 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:23.248+0000 7fcc771ee200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:36:23.280 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:22.983+0000 7fc258bd1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:36:23.280 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:23.074+0000 7fc258bd1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:36:23.280 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:23.278+0000 7fc258bd1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:36:23.668 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:23.364+0000 7fcc771ee200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:36:23.669 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:23.667+0000 7fcc771ee200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:36:23.721 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:23.469+0000 7fc258bd1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:36:24.221 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:24 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:24.073+0000 7fc258bd1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:36:24.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:23 smithi161 ceph-mon[32609]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T18:36:24.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:23 smithi161 ceph-mon[32609]: Reconfiguring daemon mgr.x on smithi112 2024-04-25T18:36:24.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:23 smithi161 ceph-mon[32609]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:24.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:23 smithi161 ceph-mon[32609]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:36:24.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:23 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:36:24.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:23 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:36:24.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:23 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:24.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:23 smithi112 ceph-mon[27864]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T18:36:24.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:23 smithi112 ceph-mon[27864]: Reconfiguring daemon mgr.x on smithi112 2024-04-25T18:36:24.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:23 smithi112 ceph-mon[27864]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:24.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:23 smithi112 ceph-mon[27864]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:36:24.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:23 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:36:24.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:23 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:36:24.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:23 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:24.268 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-mon[33266]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T18:36:24.268 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-mon[33266]: Reconfiguring daemon mgr.x on smithi112 2024-04-25T18:36:24.268 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-mon[33266]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:24.268 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-mon[33266]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:36:24.268 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:36:24.268 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:36:24.268 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:23 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:24.682 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:24 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:24.267+0000 7fcc771ee200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:36:24.682 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:36:24 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:36:24.481+0000 7fcc771ee200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:36:24.795 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:24 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:24.496+0000 7fc258bd1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:36:24.795 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:24 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:24.591+0000 7fc258bd1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:36:24.795 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:24 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:24.793+0000 7fc258bd1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:36:25.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:25.466+0000 7f176f400700 1 -- 172.21.15.112:0/3532318576 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 14) ==== 92603+0+0 (secure 0 0 0) 0x7f1764021430 con 0x7f1778103ca0 2024-04-25T18:36:25.654 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:25.318+0000 7fc258bd1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:36:25.654 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:25.409+0000 7fc258bd1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:36:25.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:25.654 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: Standby manager daemon z started 2024-04-25T18:36:25.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:36:25.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:25.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:36:25.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:36:25.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:36:25.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:25.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:25.655 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:25.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: Standby manager daemon z started 2024-04-25T18:36:25.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:36:25.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:25.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:36:25.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:36:25.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:36:25.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:25.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:25.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: Standby manager daemon z started 2024-04-25T18:36:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:36:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:36:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:36:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.171:0/2220891933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:36:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:25.971 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:25.652+0000 7fc258bd1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:36:25.971 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:25.785+0000 7fc258bd1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:36:25.971 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:25.879+0000 7fc258bd1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:36:26.297 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:25 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:25.993+0000 7fc258bd1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:36:26.298 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:26 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:26.296+0000 7fc258bd1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:36:26.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:26 smithi161 ceph-mon[32609]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-25T18:36:26.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:26 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:36:26.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:26 smithi112 ceph-mon[27864]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-25T18:36:26.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:26 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:36:26.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:26 smithi171 ceph-mon[33266]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-25T18:36:26.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:26 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:36:27.221 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:26 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:26.890+0000 7fc258bd1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:36:27.221 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:36:27 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:36:27.102+0000 7fc258bd1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:36:27.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:27.483+0000 7f176f400700 1 -- 172.21.15.112:0/3532318576 <== mon.2 v2:172.21.15.161:3300/0 8 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f1764020390 con 0x7f1778103ca0 2024-04-25T18:36:27.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:27 smithi161 ceph-mon[32609]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:27.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:27 smithi161 ceph-mon[32609]: Standby manager daemon y started 2024-04-25T18:36:27.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:27 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:36:27.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:27 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:36:27.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:27 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:36:27.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:27 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:36:27.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:27 smithi112 ceph-mon[27864]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:27.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:27 smithi112 ceph-mon[27864]: Standby manager daemon y started 2024-04-25T18:36:27.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:27 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:36:27.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:27 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:36:27.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:27 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:36:27.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:27 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:36:27.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:27 smithi171 ceph-mon[33266]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:27.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:27 smithi171 ceph-mon[33266]: Standby manager daemon y started 2024-04-25T18:36:27.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:27 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:36:27.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:27 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:36:27.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:27 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:36:27.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:27 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.161:0/861591112' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:36:28.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:28 smithi171 ceph-mon[33266]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-25T18:36:28.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:28 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:36:28.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:28 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/4093613780' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb4bc01-e29b-4402-8a7a-3387d1731ec4"}]: dispatch 2024-04-25T18:36:28.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:28 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/4093613780' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efb4bc01-e29b-4402-8a7a-3387d1731ec4"}]': finished 2024-04-25T18:36:28.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:28 smithi171 ceph-mon[33266]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T18:36:28.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:28 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:28.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:28 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1825134351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:36:28.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:28 smithi161 ceph-mon[32609]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-25T18:36:28.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:28 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:36:28.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:28 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/4093613780' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb4bc01-e29b-4402-8a7a-3387d1731ec4"}]: dispatch 2024-04-25T18:36:28.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:28 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/4093613780' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efb4bc01-e29b-4402-8a7a-3387d1731ec4"}]': finished 2024-04-25T18:36:28.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:28 smithi161 ceph-mon[32609]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T18:36:28.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:28 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:28.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:28 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1825134351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:36:28.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:28 smithi112 ceph-mon[27864]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-25T18:36:28.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:28 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:36:28.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:28 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/4093613780' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb4bc01-e29b-4402-8a7a-3387d1731ec4"}]: dispatch 2024-04-25T18:36:28.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:28 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/4093613780' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efb4bc01-e29b-4402-8a7a-3387d1731ec4"}]': finished 2024-04-25T18:36:28.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:28 smithi112 ceph-mon[27864]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T18:36:28.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:28 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:28.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:28 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1825134351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:36:29.818 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:29 smithi171 ceph-mon[33266]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:29.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:29 smithi161 ceph-mon[32609]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:29.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:29 smithi112 ceph-mon[27864]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:32 smithi171 ceph-mon[33266]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:32 smithi161 ceph-mon[32609]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:32 smithi112 ceph-mon[27864]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:34.263 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:34 smithi112 ceph-mon[27864]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:34.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:34 smithi171 ceph-mon[33266]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:34 smithi161 ceph-mon[32609]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:36.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:36 smithi112 ceph-mon[27864]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:36 smithi171 ceph-mon[33266]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:36 smithi161 ceph-mon[32609]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:38.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:38 smithi112 ceph-mon[27864]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:38.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:38 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T18:36:38.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:38 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:38 smithi171 ceph-mon[33266]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:38 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T18:36:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:38 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:38 smithi161 ceph-mon[32609]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:38 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T18:36:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:38 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:39.333 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:39 smithi171 ceph-mon[33266]: Deploying daemon osd.0 on smithi112 2024-04-25T18:36:39.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:39 smithi161 ceph-mon[32609]: Deploying daemon osd.0 on smithi112 2024-04-25T18:36:39.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:39 smithi112 ceph-mon[27864]: Deploying daemon osd.0 on smithi112 2024-04-25T18:36:40.169 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:40 smithi112 ceph-mon[27864]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:40.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:40 smithi171 ceph-mon[33266]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:40.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:40 smithi161 ceph-mon[32609]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:42.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:42 smithi171 ceph-mon[33266]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:42.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:42 smithi161 ceph-mon[32609]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:42.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:42 smithi112 ceph-mon[27864]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:44.380 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:44 smithi112 ceph-mon[27864]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:44 smithi171 ceph-mon[33266]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:44 smithi161 ceph-mon[32609]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:46.411 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:46 smithi112 ceph-mon[27864]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:46 smithi171 ceph-mon[33266]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:46 smithi161 ceph-mon[32609]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:47.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:47.301 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:47.302 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:47.372 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:47.372 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:47.372 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:47.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:47.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:47.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:48.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:48 smithi112 ceph-mon[27864]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:48 smithi171 ceph-mon[33266]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:48 smithi161 ceph-mon[32609]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:50.308 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:50 smithi112 ceph-mon[27864]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:50.308 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:50.308 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:50.308 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:50.308 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:50.309 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:50.309 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:50 smithi112 ceph-mon[27864]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:36:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:50 smithi171 ceph-mon[33266]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:50 smithi171 ceph-mon[33266]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:36:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:50 smithi161 ceph-mon[32609]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:50 smithi161 ceph-mon[32609]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:36:51.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.190+0000 7f176f400700 1 -- 172.21.15.112:0/3532318576 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1778076390 con 0x7f175c039890 2024-04-25T18:36:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 0 on host 'smithi112' 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.194+0000 7f177d600700 1 -- 172.21.15.112:0/3532318576 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f175c039890 msgr2=0x7f175c03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.194+0000 7f177d600700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f175c039890 0x7f175c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1768003820 tx=0x7f1768005fb0 comp rx=0 tx=0).stop 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.194+0000 7f177d600700 1 -- 172.21.15.112:0/3532318576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1778103ca0 msgr2=0x7f177819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.195+0000 7f177d600700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1778103ca0 0x7f177819b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f176400e9f0 tx=0x7f176400ed00 comp rx=0 tx=0).stop 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.195+0000 7f177d600700 1 -- 172.21.15.112:0/3532318576 shutdown_connections 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.195+0000 7f177d600700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f175c039890 0x7f175c03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.195+0000 7f177d600700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1778104640 0x7f17781a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.195+0000 7f177d600700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1778102aa0 0x7f177819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.195+0000 7f177d600700 1 --2- 172.21.15.112:0/3532318576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1778103ca0 0x7f177819b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.195+0000 7f177d600700 1 -- 172.21.15.112:0/3532318576 >> 172.21.15.112:0/3532318576 conn(0x7f17780fe040 msgr2=0x7f1778074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.195+0000 7f177d600700 1 -- 172.21.15.112:0/3532318576 shutdown_connections 2024-04-25T18:36:51.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:36:51.195+0000 7f177d600700 1 -- 172.21.15.112:0/3532318576 wait complete. 2024-04-25T18:36:51.890 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:51 smithi112 ceph-mon[27864]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T18:36:51.890 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:51 smithi112 ceph-mon[27864]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T18:36:51.890 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:51 smithi112 ceph-mon[27864]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:36:51.890 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:51 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:51.890 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:51 smithi112 ceph-mon[27864]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:51.890 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:51 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:51.890 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:51 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:51.890 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:51 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:52.025 DEBUG:teuthology.orchestra.run.smithi112:osd.0> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T18:36:52.029 INFO:tasks.cephadm:Deploying osd.1 on smithi161 with /dev/vg_nvme/lv_4... 2024-04-25T18:36:52.029 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T18:36:52.057 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:51 smithi161 ceph-mon[32609]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T18:36:52.057 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:51 smithi161 ceph-mon[32609]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T18:36:52.057 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:51 smithi161 ceph-mon[32609]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:36:52.057 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:51 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:52.057 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:51 smithi161 ceph-mon[32609]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:52.057 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:51 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:52.057 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:51 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:52.057 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:51 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:52.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:51 smithi171 ceph-mon[33266]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T18:36:52.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:51 smithi171 ceph-mon[33266]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T18:36:52.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:51 smithi171 ceph-mon[33266]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:36:52.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:52.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:51 smithi171 ceph-mon[33266]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:36:52.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:52.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:52.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:51 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:36:52.273 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:36:52.797 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:52 smithi112 ceph-mon[27864]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:36:52.798 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:52 smithi112 ceph-mon[27864]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T18:36:52.798 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:52 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:52.798 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:52 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:52.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:52 smithi161 ceph-mon[32609]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:36:52.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:52 smithi161 ceph-mon[32609]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T18:36:52.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:52 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:52.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:52 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:53.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:52 smithi171 ceph-mon[33266]: from='osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:36:53.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:52 smithi171 ceph-mon[33266]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T18:36:53.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:52 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:53.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:52 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:53.933 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:53 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:36:53.934 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:53 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:36:53.934 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:53 smithi112 ceph-mon[27864]: osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535] boot 2024-04-25T18:36:53.934 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:53 smithi112 ceph-mon[27864]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T18:36:53.934 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:53 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:53.934 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:53 smithi112 ceph-mon[27864]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:36:53.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:53 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:36:53.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:53 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:36:53.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:53 smithi161 ceph-mon[32609]: osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535] boot 2024-04-25T18:36:53.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:53 smithi161 ceph-mon[32609]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T18:36:53.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:53 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:53.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:53 smithi161 ceph-mon[32609]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:36:54.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:53 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:36:54.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:53 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:36:54.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:53 smithi171 ceph-mon[33266]: osd.0 [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535] boot 2024-04-25T18:36:54.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:53 smithi171 ceph-mon[33266]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T18:36:54.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:53 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:36:54.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:53 smithi171 ceph-mon[33266]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:36:54.216 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T18:36:54.216 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T18:36:54.217 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-04-25T18:36:54.217 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-04-25T18:36:54.217 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114369 s, 917 MB/s 2024-04-25T18:36:54.217 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-04-25T18:36:54.779 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:54 smithi161 ceph-mon[32609]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T18:36:54.805 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-04-25T18:36:55.079 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:36:55.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:54 smithi171 ceph-mon[33266]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T18:36:55.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:54 smithi112 ceph-mon[27864]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T18:36:56.026 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:55 smithi161 ceph-mon[32609]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:36:56.124 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:55 smithi112 ceph-mon[27864]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:36:56.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:55 smithi171 ceph-mon[33266]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:36:56.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.531+0000 7f0519f75700 1 -- 172.21.15.161:0/1081354456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0514103d90 msgr2=0x7f05141041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:56.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.531+0000 7f0519f75700 1 --2- 172.21.15.161:0/1081354456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0514103d90 0x7f05141041f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0508009990 tx=0x7f0508009ca0 comp rx=0 tx=0).stop 2024-04-25T18:36:56.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.531+0000 7f0519f75700 1 -- 172.21.15.161:0/1081354456 shutdown_connections 2024-04-25T18:36:56.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.531+0000 7f0519f75700 1 --2- 172.21.15.161:0/1081354456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0514104730 0x7f051410afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:56.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.531+0000 7f0519f75700 1 --2- 172.21.15.161:0/1081354456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0514100db0 0x7f05141011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:56.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.531+0000 7f0519f75700 1 --2- 172.21.15.161:0/1081354456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0514103d90 0x7f05141041f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:56.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.531+0000 7f0519f75700 1 -- 172.21.15.161:0/1081354456 >> 172.21.15.161:0/1081354456 conn(0x7f05140fc710 msgr2=0x7f05140feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:36:56.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.531+0000 7f0519f75700 1 -- 172.21.15.161:0/1081354456 shutdown_connections 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.531+0000 7f0519f75700 1 -- 172.21.15.161:0/1081354456 wait complete. 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.531+0000 7f0519f75700 1 Processor -- start 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.531+0000 7f0519f75700 1 -- start start 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0519f75700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0514100db0 0x7f0514198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0519f75700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0514103d90 0x7f05141991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0519f75700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0514104730 0x7f05141a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0519f75700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f051410e5f0 con 0x7f0514103d90 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0519f75700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f051410e470 con 0x7f0514100db0 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0519f75700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f051410e770 con 0x7f0514104730 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f05137fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0514100db0 0x7f0514198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f05137fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0514100db0 0x7f0514198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54662/0 (socket says 172.21.15.161:54662) 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f05137fe700 1 -- 172.21.15.161:0/1811783915 learned_addr learned my addr 172.21.15.161:0/1811783915 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-04-25T18:36:56.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0513fff700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0514104730 0x7f05141a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:56.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f05137fe700 1 -- 172.21.15.161:0/1811783915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0514104730 msgr2=0x7f05141a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:56.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f05137fe700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0514104730 0x7f05141a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:56.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f05137fe700 1 -- 172.21.15.161:0/1811783915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0514103d90 msgr2=0x7f05141991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:36:56.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f05137fe700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0514103d90 0x7f05141991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:36:56.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f05137fe700 1 -- 172.21.15.161:0/1811783915 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05141a08b0 con 0x7f0514100db0 2024-04-25T18:36:56.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f05137fe700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0514100db0 0x7f0514198c60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f050400b500 tx=0x7f050400b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:36:56.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0510ff9700 1 -- 172.21.15.161:0/1811783915 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504015020 con 0x7f0514100db0 2024-04-25T18:36:56.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0519f75700 1 -- 172.21.15.161:0/1811783915 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05141a0ba0 con 0x7f0514100db0 2024-04-25T18:36:56.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0510ff9700 1 -- 172.21.15.161:0/1811783915 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0504011ce0 con 0x7f0514100db0 2024-04-25T18:36:56.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0510ff9700 1 -- 172.21.15.161:0/1811783915 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504010d20 con 0x7f0514100db0 2024-04-25T18:36:56.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.532+0000 7f0519f75700 1 -- 172.21.15.161:0/1811783915 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05141a10e0 con 0x7f0514100db0 2024-04-25T18:36:56.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.534+0000 7f0519f75700 1 -- 172.21.15.161:0/1811783915 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0514069730 con 0x7f0514100db0 2024-04-25T18:36:56.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.534+0000 7f0510ff9700 1 -- 172.21.15.161:0/1811783915 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f0504018020 con 0x7f0514100db0 2024-04-25T18:36:56.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.535+0000 7f0510ff9700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04fc0a3bb0 0x7f04fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:36:56.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.535+0000 7f0510ff9700 1 -- 172.21.15.161:0/1811783915 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f05040ce0a0 con 0x7f0514100db0 2024-04-25T18:36:56.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.535+0000 7f0512ffd700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04fc0a3bb0 0x7f04fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:36:56.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.536+0000 7f0512ffd700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04fc0a3bb0 0x7f04fc0a6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f05080098e0 tx=0x7f0508003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:36:56.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.540+0000 7f0510ff9700 1 -- 172.21.15.161:0/1811783915 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f050401f020 con 0x7f0514100db0 2024-04-25T18:36:56.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:36:56.745+0000 7f0519f75700 1 -- 172.21.15.161:0/1811783915 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0514062e80 con 0x7f04fc0a3bb0 2024-04-25T18:36:57.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:36:57.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:36:57.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:36:57.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:36:57.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:36:57.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:36:57.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:36:57.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:57.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:36:57.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi112 2024-04-25T18:36:57.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:57.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:57.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:36:57.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi112 to 17089M 2024-04-25T18:36:57.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:57.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:57.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:57.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:36:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:36:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:36:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi112 2024-04-25T18:36:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:36:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi112 to 17089M 2024-04-25T18:36:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:58.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:58.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:58.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:36:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:58.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:36:58.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:58.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:36:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi112 2024-04-25T18:36:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:36:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi112 to 17089M 2024-04-25T18:36:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:36:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:36:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:36:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:36:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:00 smithi171 ceph-mon[33266]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:00 smithi171 ceph-mon[33266]: from='client.? 172.21.15.161:0/4291454354' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a83fa2-96a6-4743-8c17-21531f5e885e"}]: dispatch 2024-04-25T18:37:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:00 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a83fa2-96a6-4743-8c17-21531f5e885e"}]: dispatch 2024-04-25T18:37:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:00 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65a83fa2-96a6-4743-8c17-21531f5e885e"}]': finished 2024-04-25T18:37:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:00 smithi171 ceph-mon[33266]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T18:37:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:00 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:00 smithi171 ceph-mon[33266]: from='client.? 172.21.15.161:0/3076752321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:37:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:00 smithi161 ceph-mon[32609]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:00 smithi161 ceph-mon[32609]: from='client.? 172.21.15.161:0/4291454354' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a83fa2-96a6-4743-8c17-21531f5e885e"}]: dispatch 2024-04-25T18:37:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:00 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a83fa2-96a6-4743-8c17-21531f5e885e"}]: dispatch 2024-04-25T18:37:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:00 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65a83fa2-96a6-4743-8c17-21531f5e885e"}]': finished 2024-04-25T18:37:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:00 smithi161 ceph-mon[32609]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T18:37:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:00 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:00 smithi161 ceph-mon[32609]: from='client.? 172.21.15.161:0/3076752321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:37:00.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:00 smithi112 ceph-mon[27864]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:00.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:00 smithi112 ceph-mon[27864]: from='client.? 172.21.15.161:0/4291454354' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a83fa2-96a6-4743-8c17-21531f5e885e"}]: dispatch 2024-04-25T18:37:00.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:00 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a83fa2-96a6-4743-8c17-21531f5e885e"}]: dispatch 2024-04-25T18:37:00.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:00 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65a83fa2-96a6-4743-8c17-21531f5e885e"}]': finished 2024-04-25T18:37:00.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:00 smithi112 ceph-mon[27864]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T18:37:00.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:00 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:00.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:00 smithi112 ceph-mon[27864]: from='client.? 172.21.15.161:0/3076752321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:37:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:02 smithi171 ceph-mon[33266]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:02 smithi161 ceph-mon[32609]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:02 smithi112 ceph-mon[27864]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:04.344 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:04 smithi161 ceph-mon[32609]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:04.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:04 smithi171 ceph-mon[33266]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:04.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:04 smithi112 ceph-mon[27864]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:06 smithi171 ceph-mon[33266]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:06 smithi161 ceph-mon[32609]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:06.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:06 smithi112 ceph-mon[27864]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:07.421 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T18:37:07.422 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:07.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:07 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T18:37:07.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:07 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:07.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:07 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T18:37:07.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:07 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:08.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:08 smithi112 ceph-mon[27864]: Deploying daemon osd.1 on smithi161 2024-04-25T18:37:08.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:08 smithi112 ceph-mon[27864]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:08.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:08 smithi171 ceph-mon[33266]: Deploying daemon osd.1 on smithi161 2024-04-25T18:37:08.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:08 smithi171 ceph-mon[33266]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:08 smithi161 ceph-mon[32609]: Deploying daemon osd.1 on smithi161 2024-04-25T18:37:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:08 smithi161 ceph-mon[32609]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:10 smithi161 ceph-mon[32609]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:10.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:10 smithi171 ceph-mon[33266]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:10.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:10 smithi112 ceph-mon[27864]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:12.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:12 smithi171 ceph-mon[33266]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:12 smithi161 ceph-mon[32609]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:12.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:12 smithi112 ceph-mon[27864]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:13.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:13 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:13.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:13 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:13.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:13 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:13.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:13 smithi171 ceph-mon[33266]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:13.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:13 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:13.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:13 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:13.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:13 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:13.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:13 smithi161 ceph-mon[32609]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:13.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:13 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:13.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:13 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:13.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:13 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:13.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:13 smithi112 ceph-mon[27864]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:15.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.435+0000 7f0510ff9700 1 -- 172.21.15.161:0/1811783915 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0514062e80 con 0x7f04fc0a3bb0 2024-04-25T18:37:15.437 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 1 on host 'smithi161' 2024-04-25T18:37:15.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 -- 172.21.15.161:0/1811783915 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04fc0a3bb0 msgr2=0x7f04fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:37:15.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04fc0a3bb0 0x7f04fc0a6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f05080098e0 tx=0x7f0508003860 comp rx=0 tx=0).stop 2024-04-25T18:37:15.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 -- 172.21.15.161:0/1811783915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0514100db0 msgr2=0x7f0514198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:37:15.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0514100db0 0x7f0514198c60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f050400b500 tx=0x7f050400b810 comp rx=0 tx=0).stop 2024-04-25T18:37:15.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 -- 172.21.15.161:0/1811783915 shutdown_connections 2024-04-25T18:37:15.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04fc0a3bb0 0x7f04fc0a6070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:15.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0514104730 0x7f05141a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:15.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0514100db0 0x7f0514198c60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:15.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 --2- 172.21.15.161:0/1811783915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0514103d90 0x7f05141991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:15.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 -- 172.21.15.161:0/1811783915 >> 172.21.15.161:0/1811783915 conn(0x7f05140fc710 msgr2=0x7f0514107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:37:15.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 -- 172.21.15.161:0/1811783915 shutdown_connections 2024-04-25T18:37:15.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:37:15.437+0000 7f0519f75700 1 -- 172.21.15.161:0/1811783915 wait complete. 2024-04-25T18:37:16.037 DEBUG:teuthology.orchestra.run.smithi161:osd.1> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T18:37:16.039 INFO:tasks.cephadm:Deploying osd.2 on smithi171 with /dev/vg_nvme/lv_4... 2024-04-25T18:37:16.040 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T18:37:16.284 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:37:16.405 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:16 smithi171 ceph-mon[33266]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:16.406 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:16.406 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:16.406 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:16 smithi171 ceph-mon[33266]: from='osd.1 [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:37:16.406 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:16 smithi171 ceph-mon[33266]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:37:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:16 smithi161 ceph-mon[32609]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:16 smithi161 ceph-mon[32609]: from='osd.1 [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:37:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:16 smithi161 ceph-mon[32609]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:37:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:16 smithi112 ceph-mon[27864]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:16 smithi112 ceph-mon[27864]: from='osd.1 [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:37:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:16 smithi112 ceph-mon[27864]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:37:17.409 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:17 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:17.409 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:17 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:17.409 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:17 smithi171 ceph-mon[33266]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T18:37:17.409 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:17 smithi171 ceph-mon[33266]: from='osd.1 [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:37:17.409 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:17 smithi171 ceph-mon[33266]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T18:37:17.409 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:17 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:17.409 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:17 smithi171 ceph-mon[33266]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:37:17.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:17 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:17.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:17 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:17.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:17 smithi161 ceph-mon[32609]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T18:37:17.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:17 smithi161 ceph-mon[32609]: from='osd.1 [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:37:17.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:17 smithi161 ceph-mon[32609]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T18:37:17.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:17 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:17.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:17 smithi161 ceph-mon[32609]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:37:17.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:17 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:17.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:17 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:17.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:17 smithi112 ceph-mon[27864]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T18:37:17.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:17 smithi112 ceph-mon[27864]: from='osd.1 [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:37:17.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:17 smithi112 ceph-mon[27864]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T18:37:17.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:17 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:17.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:17 smithi112 ceph-mon[27864]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:37:18.127 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T18:37:18.127 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T18:37:18.127 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-25T18:37:18.127 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-25T18:37:18.127 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113557 s, 923 MB/s 2024-04-25T18:37:18.127 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-25T18:37:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:18 smithi161 ceph-mon[32609]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:18 smithi161 ceph-mon[32609]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:37:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:18 smithi161 ceph-mon[32609]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T18:37:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:18 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:18.428 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:18 smithi171 ceph-mon[33266]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:18.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:18 smithi171 ceph-mon[33266]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:37:18.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:18 smithi171 ceph-mon[33266]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T18:37:18.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:18 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:18 smithi112 ceph-mon[27864]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:18 smithi112 ceph-mon[27864]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:37:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:18 smithi112 ceph-mon[27864]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T18:37:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:18.721 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:37:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1[37195]: 2024-04-25T18:37:18.382+0000 7f504662d700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T18:37:18.721 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:37:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1[37195]: 2024-04-25T18:37:18.545+0000 7f503e418700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:37:18.758 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_4 2024-04-25T18:37:19.005 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:37:19.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:19 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:37:19.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:19 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:37:19.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:19 smithi171 ceph-mon[33266]: from='osd.1 ' entity='osd.1' 2024-04-25T18:37:19.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:19 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:19.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:19 smithi171 ceph-mon[33266]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:19.720 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:19 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:37:19.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:19 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:37:19.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:19 smithi161 ceph-mon[32609]: from='osd.1 ' entity='osd.1' 2024-04-25T18:37:19.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:19 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:19.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:19 smithi161 ceph-mon[32609]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:19.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:19 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:37:19.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:19 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:37:19.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:19 smithi112 ceph-mon[27864]: from='osd.1 ' entity='osd.1' 2024-04-25T18:37:19.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:19 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:19.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:19 smithi112 ceph-mon[27864]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:37:20.497 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.494+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3765432547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3eb8103540 msgr2=0x7f3eb8109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:37:20.497 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.494+0000 7f3ebeadc700 1 --2- 172.21.15.171:0/3765432547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3eb8103540 0x7f3eb8109df0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000b0a0 tx=0x7f3eb000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:37:20.497 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.494+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3765432547 shutdown_connections 2024-04-25T18:37:20.497 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.494+0000 7f3ebeadc700 1 --2- 172.21.15.171:0/3765432547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3eb8103540 0x7f3eb8109df0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:20.497 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.494+0000 7f3ebeadc700 1 --2- 172.21.15.171:0/3765432547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eb81019a0 0x7f3eb8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:20.497 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.494+0000 7f3ebeadc700 1 --2- 172.21.15.171:0/3765432547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb8102ba0 0x7f3eb8103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:20.497 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.494+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3765432547 >> 172.21.15.171:0/3765432547 conn(0x7f3eb80fcf40 msgr2=0x7f3eb80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:37:20.497 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.494+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3765432547 shutdown_connections 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.494+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3765432547 wait complete. 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebeadc700 1 Processor -- start 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebeadc700 1 -- start start 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebeadc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3eb81019a0 0x7f3eb8198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebeadc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb8102ba0 0x7f3eb8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebeadc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eb8103540 0x7f3eb81a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebeadc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb810d530 con 0x7f3eb8102ba0 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebeadc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb810d3b0 con 0x7f3eb8103540 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebeadc700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb810d6b0 con 0x7f3eb81019a0 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebc878700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3eb81019a0 0x7f3eb8198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebc878700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3eb81019a0 0x7f3eb8198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:48312/0 (socket says 172.21.15.171:48312) 2024-04-25T18:37:20.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebc878700 1 -- 172.21.15.171:0/3463426484 learned_addr learned my addr 172.21.15.171:0/3463426484 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T18:37:20.499 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebc878700 1 -- 172.21.15.171:0/3463426484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eb8103540 msgr2=0x7f3eb81a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:37:20.499 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebc878700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eb8103540 0x7f3eb81a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:20.499 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebc878700 1 -- 172.21.15.171:0/3463426484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb8102ba0 msgr2=0x7f3eb8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:37:20.499 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebc878700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb8102ba0 0x7f3eb8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:20.499 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebc878700 1 -- 172.21.15.171:0/3463426484 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb81a09a0 con 0x7f3eb81019a0 2024-04-25T18:37:20.499 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.495+0000 7f3ebc878700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3eb81019a0 0x7f3eb8198d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00b530 tx=0x7f3eac00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:37:20.499 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.496+0000 7f3eb5ffb700 1 -- 172.21.15.171:0/3463426484 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac015020 con 0x7f3eb81019a0 2024-04-25T18:37:20.499 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.496+0000 7f3eb5ffb700 1 -- 172.21.15.171:0/3463426484 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eac011ce0 con 0x7f3eb81019a0 2024-04-25T18:37:20.499 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.496+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3463426484 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb81a0c90 con 0x7f3eb81019a0 2024-04-25T18:37:20.499 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.496+0000 7f3eb5ffb700 1 -- 172.21.15.171:0/3463426484 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac010d60 con 0x7f3eb81019a0 2024-04-25T18:37:20.499 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.496+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3463426484 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb81a1200 con 0x7f3eb81019a0 2024-04-25T18:37:20.500 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.497+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3463426484 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb8069730 con 0x7f3eb81019a0 2024-04-25T18:37:20.500 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.498+0000 7f3eb5ffb700 1 -- 172.21.15.171:0/3463426484 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f3eac018020 con 0x7f3eb81019a0 2024-04-25T18:37:20.500 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.498+0000 7f3eb5ffb700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3ea80a3bb0 0x7f3ea80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:37:20.500 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.498+0000 7f3eb5ffb700 1 -- 172.21.15.171:0/3463426484 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f3eac0ce3b0 con 0x7f3eb81019a0 2024-04-25T18:37:20.500 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.499+0000 7f3eb7fff700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3ea80a3bb0 0x7f3ea80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:37:20.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.500+0000 7f3eb7fff700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3ea80a3bb0 0x7f3ea80a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40097b0 tx=0x7f3ea4006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:37:20.504 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.503+0000 7f3eb5ffb700 1 -- 172.21.15.171:0/3463426484 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3eac01f020 con 0x7f3eb81019a0 2024-04-25T18:37:20.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: osd.1 [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744] boot 2024-04-25T18:37:20.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T18:37:20.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:20.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi161 2024-04-25T18:37:20.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:37:20.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi161 to 17088M 2024-04-25T18:37:20.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:20.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:20.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:20.685 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:20.685 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:20.685 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:20 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.709 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:20.708+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3463426484 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3eb806ae60 con 0x7f3ea80a3bb0 2024-04-25T18:37:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: osd.1 [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744] boot 2024-04-25T18:37:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T18:37:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi161 2024-04-25T18:37:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:37:20.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi161 to 17088M 2024-04-25T18:37:20.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:20.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:20.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:20.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:20.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:20.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:20 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: osd.1 [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744] boot 2024-04-25T18:37:20.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T18:37:20.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:37:20.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi161 2024-04-25T18:37:20.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:37:20.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi161 to 17088M 2024-04-25T18:37:20.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:20.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:20.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:20.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:20.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:20.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:20.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:20 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:21.655 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:21 smithi171 ceph-mon[33266]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:37:21.655 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:37:21.655 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:37:21.655 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:21.655 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:21 smithi171 ceph-mon[33266]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T18:37:21.655 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:21 smithi171 ceph-mon[33266]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:21.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:21 smithi161 ceph-mon[32609]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:37:21.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:37:21.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:37:21.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:21.691 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:21 smithi161 ceph-mon[32609]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T18:37:21.692 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:21 smithi161 ceph-mon[32609]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:21.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:21 smithi112 ceph-mon[27864]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:37:21.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:37:21.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:37:21.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:21.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:21 smithi112 ceph-mon[27864]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T18:37:21.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:21 smithi112 ceph-mon[27864]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:24 smithi171 ceph-mon[33266]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:24 smithi171 ceph-mon[33266]: from='client.? 172.21.15.171:0/751650429' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7562ec4c-c126-48ec-a51c-56fa461a2e05"}]: dispatch 2024-04-25T18:37:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:24 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7562ec4c-c126-48ec-a51c-56fa461a2e05"}]: dispatch 2024-04-25T18:37:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:24 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7562ec4c-c126-48ec-a51c-56fa461a2e05"}]': finished 2024-04-25T18:37:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:24 smithi171 ceph-mon[33266]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T18:37:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:24 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:24 smithi171 ceph-mon[33266]: from='client.? 172.21.15.171:0/114638777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:37:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:24 smithi161 ceph-mon[32609]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:24 smithi161 ceph-mon[32609]: from='client.? 172.21.15.171:0/751650429' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7562ec4c-c126-48ec-a51c-56fa461a2e05"}]: dispatch 2024-04-25T18:37:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:24 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7562ec4c-c126-48ec-a51c-56fa461a2e05"}]: dispatch 2024-04-25T18:37:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:24 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7562ec4c-c126-48ec-a51c-56fa461a2e05"}]': finished 2024-04-25T18:37:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:24 smithi161 ceph-mon[32609]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T18:37:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:24 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:24 smithi161 ceph-mon[32609]: from='client.? 172.21.15.171:0/114638777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:37:24.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:24 smithi112 ceph-mon[27864]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:24.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:24 smithi112 ceph-mon[27864]: from='client.? 172.21.15.171:0/751650429' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7562ec4c-c126-48ec-a51c-56fa461a2e05"}]: dispatch 2024-04-25T18:37:24.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:24 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7562ec4c-c126-48ec-a51c-56fa461a2e05"}]: dispatch 2024-04-25T18:37:24.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:24 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7562ec4c-c126-48ec-a51c-56fa461a2e05"}]': finished 2024-04-25T18:37:24.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:24 smithi112 ceph-mon[27864]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T18:37:24.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:24 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:24.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:24 smithi112 ceph-mon[27864]: from='client.? 172.21.15.171:0/114638777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:37:26.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:26 smithi171 ceph-mon[33266]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:26.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:26 smithi161 ceph-mon[32609]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:26.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:26 smithi112 ceph-mon[27864]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:28 smithi171 ceph-mon[33266]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:28.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:28 smithi112 ceph-mon[27864]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:28 smithi161 ceph-mon[32609]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:30.426 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:30 smithi171 ceph-mon[33266]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:30 smithi161 ceph-mon[32609]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:30.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:30 smithi112 ceph-mon[27864]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:31.391 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T18:37:31.391 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T18:37:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:31.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T18:37:31.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:32.398 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:32 smithi171 ceph-mon[33266]: Deploying daemon osd.2 on smithi171 2024-04-25T18:37:32.398 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:32 smithi171 ceph-mon[33266]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:32 smithi161 ceph-mon[32609]: Deploying daemon osd.2 on smithi171 2024-04-25T18:37:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:32 smithi161 ceph-mon[32609]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:32 smithi112 ceph-mon[27864]: Deploying daemon osd.2 on smithi171 2024-04-25T18:37:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:32 smithi112 ceph-mon[27864]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:34.387 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:34 smithi171 ceph-mon[33266]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:34 smithi161 ceph-mon[32609]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:34.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:34 smithi112 ceph-mon[27864]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:36 smithi171 ceph-mon[33266]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:36 smithi161 ceph-mon[32609]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:36 smithi112 ceph-mon[27864]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:37.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:37.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:37.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:37.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:37 smithi171 ceph-mon[33266]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:37.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:37.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:37.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:37.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:37 smithi161 ceph-mon[32609]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:37.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:37.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:37.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:37.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:37 smithi112 ceph-mon[27864]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:39.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:39.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:39.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:39.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:39.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:39.883 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:39 smithi171 ceph-mon[33266]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:39.942 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 2 on host 'smithi171' 2024-04-25T18:37:39.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.940+0000 7f3eb5ffb700 1 -- 172.21.15.171:0/3463426484 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3eb806ae60 con 0x7f3ea80a3bb0 2024-04-25T18:37:39.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.942+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3463426484 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3ea80a3bb0 msgr2=0x7f3ea80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:37:39.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.942+0000 7f3ebeadc700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3ea80a3bb0 0x7f3ea80a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40097b0 tx=0x7f3ea4006d20 comp rx=0 tx=0).stop 2024-04-25T18:37:39.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.943+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3463426484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3eb81019a0 msgr2=0x7f3eb8198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:37:39.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.943+0000 7f3ebeadc700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3eb81019a0 0x7f3eb8198d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00b530 tx=0x7f3eac00b8f0 comp rx=0 tx=0).stop 2024-04-25T18:37:39.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.943+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3463426484 shutdown_connections 2024-04-25T18:37:39.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.943+0000 7f3ebeadc700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3ea80a3bb0 0x7f3ea80a6070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:39.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.943+0000 7f3ebeadc700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eb8103540 0x7f3eb81a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:39.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.943+0000 7f3ebeadc700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3eb81019a0 0x7f3eb8198d40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:39.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.943+0000 7f3ebeadc700 1 --2- 172.21.15.171:0/3463426484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb8102ba0 0x7f3eb8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:39.945 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.943+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3463426484 >> 172.21.15.171:0/3463426484 conn(0x7f3eb80fcf40 msgr2=0x7f3eb806a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:37:39.945 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.943+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3463426484 shutdown_connections 2024-04-25T18:37:39.945 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:37:39.943+0000 7f3ebeadc700 1 -- 172.21.15.171:0/3463426484 wait complete. 2024-04-25T18:37:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:39 smithi161 ceph-mon[32609]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:39.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:39 smithi112 ceph-mon[27864]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:40.499 DEBUG:teuthology.orchestra.run.smithi171:osd.2> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T18:37:40.501 INFO:tasks.cephadm:Deploying osd.3 on smithi112 with /dev/vg_nvme/lv_3... 2024-04-25T18:37:40.502 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T18:37:40.938 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:37:41.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:40 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:41.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:40 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:41.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:40 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:41.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:40 smithi171 ceph-mon[33266]: from='osd.2 [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:37:41.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:40 smithi171 ceph-mon[33266]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:37:41.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:40 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:41.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:40 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:41.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:40 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:41.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:40 smithi161 ceph-mon[32609]: from='osd.2 [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:37:41.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:40 smithi161 ceph-mon[32609]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:37:41.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:40 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:41.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:40 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:37:41.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:40 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:41.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:40 smithi112 ceph-mon[27864]: from='osd.2 [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:37:41.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:40 smithi112 ceph-mon[27864]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:37:42.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:41 smithi171 ceph-mon[33266]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T18:37:42.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:41 smithi171 ceph-mon[33266]: from='osd.2 [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:37:42.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:41 smithi171 ceph-mon[33266]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T18:37:42.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:41 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:42.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:41 smithi171 ceph-mon[33266]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:37:42.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:41 smithi171 ceph-mon[33266]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:41 smithi161 ceph-mon[32609]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T18:37:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:41 smithi161 ceph-mon[32609]: from='osd.2 [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:37:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:41 smithi161 ceph-mon[32609]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T18:37:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:41 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:41 smithi161 ceph-mon[32609]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:37:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:41 smithi161 ceph-mon[32609]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:41 smithi112 ceph-mon[27864]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T18:37:42.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:41 smithi112 ceph-mon[27864]: from='osd.2 [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:37:42.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:41 smithi112 ceph-mon[27864]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T18:37:42.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:41 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:42.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:41 smithi112 ceph-mon[27864]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:37:42.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:41 smithi112 ceph-mon[27864]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:43.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:42 smithi171 ceph-mon[33266]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:37:43.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:42 smithi171 ceph-mon[33266]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T18:37:43.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:43.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:43.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:42 smithi171 ceph-mon[33266]: from='osd.2 ' entity='osd.2' 2024-04-25T18:37:43.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:43.183 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:37:42 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2[38410]: 2024-04-25T18:37:42.907+0000 7f6504a7e700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T18:37:43.184 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2[38410]: 2024-04-25T18:37:43.043+0000 7f64fc869700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:37:43.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:42 smithi112 ceph-mon[27864]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:37:43.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:42 smithi112 ceph-mon[27864]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T18:37:43.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:43.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:43.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:42 smithi112 ceph-mon[27864]: from='osd.2 ' entity='osd.2' 2024-04-25T18:37:43.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:43.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:42 smithi161 ceph-mon[32609]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:37:43.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:42 smithi161 ceph-mon[32609]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T18:37:43.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:43.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:43.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:42 smithi161 ceph-mon[32609]: from='osd.2 ' entity='osd.2' 2024-04-25T18:37:43.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T18:37:43.941 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T18:37:43.941 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T18:37:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T18:37:43.942 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.03139 s, 334 MB/s 2024-04-25T18:37:43.942 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T18:37:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:43 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:37:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:43 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:37:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:43 smithi161 ceph-mon[32609]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:37:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:44.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:44.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:43 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:37:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:43 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:37:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:43 smithi112 ceph-mon[27864]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:37:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:37:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:37:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-mon[33266]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:37:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:37:44.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:44.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:37:44.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:44.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:44.813 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-04-25T18:37:45.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:44 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi171 2024-04-25T18:37:45.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:44 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-25T18:37:45.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:44 smithi161 ceph-mon[32609]: osd.2 [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073] boot 2024-04-25T18:37:45.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:44 smithi161 ceph-mon[32609]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T18:37:45.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:44 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:44 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi171 2024-04-25T18:37:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:44 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-25T18:37:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:44 smithi112 ceph-mon[27864]: osd.2 [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073] boot 2024-04-25T18:37:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:44 smithi112 ceph-mon[27864]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T18:37:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:44 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:45.238 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:37:45.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:44 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi171 2024-04-25T18:37:45.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:44 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-25T18:37:45.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:44 smithi171 ceph-mon[33266]: osd.2 [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073] boot 2024-04-25T18:37:45.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:44 smithi171 ceph-mon[33266]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T18:37:45.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:44 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:37:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:45 smithi112 ceph-mon[27864]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T18:37:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:45 smithi112 ceph-mon[27864]: pgmap v94: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:45 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:37:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:45 smithi171 ceph-mon[33266]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T18:37:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:45 smithi171 ceph-mon[33266]: pgmap v94: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:45 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:37:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:45 smithi161 ceph-mon[32609]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T18:37:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:45 smithi161 ceph-mon[32609]: pgmap v94: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:45 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:37:47.206 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:46 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T18:37:47.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:46 smithi112 ceph-mon[27864]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T18:37:47.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:46 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:37:47.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:46 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T18:37:47.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:46 smithi161 ceph-mon[32609]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T18:37:47.329 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:46 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:37:47.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:46 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T18:37:47.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:46 smithi171 ceph-mon[33266]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T18:37:47.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:46 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:37:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.447+0000 7f26e8400700 1 -- 172.21.15.112:0/3358690910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0103530 msgr2=0x7f26e0105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:37:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.447+0000 7f26e8400700 1 --2- 172.21.15.112:0/3358690910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0103530 0x7f26e0105920 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009990 tx=0x7f26d4009ca0 comp rx=0 tx=0).stop 2024-04-25T18:37:47.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.447+0000 7f26e8400700 1 -- 172.21.15.112:0/3358690910 shutdown_connections 2024-04-25T18:37:47.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.447+0000 7f26e8400700 1 --2- 172.21.15.112:0/3358690910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e0108790 0x7f26e010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:47.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.447+0000 7f26e8400700 1 --2- 172.21.15.112:0/3358690910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0103530 0x7f26e0105920 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:47.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.447+0000 7f26e8400700 1 --2- 172.21.15.112:0/3358690910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26e0105e60 0x7f26e0108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:47.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.447+0000 7f26e8400700 1 -- 172.21.15.112:0/3358690910 >> 172.21.15.112:0/3358690910 conn(0x7f26e00fcf40 msgr2=0x7f26e00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:37:47.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.447+0000 7f26e8400700 1 -- 172.21.15.112:0/3358690910 shutdown_connections 2024-04-25T18:37:47.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.447+0000 7f26e8400700 1 -- 172.21.15.112:0/3358690910 wait complete. 2024-04-25T18:37:47.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.449+0000 7f26e8400700 1 Processor -- start 2024-04-25T18:37:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.449+0000 7f26e8400700 1 -- start start 2024-04-25T18:37:47.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.450+0000 7f26e8400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26e0103530 0x7f26e019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:37:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.450+0000 7f26e8400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0105e60 0x7f26e019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:37:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.450+0000 7f26e8400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e0108790 0x7f26e01a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:37:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.450+0000 7f26e8400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26e010e520 con 0x7f26e0105e60 2024-04-25T18:37:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.450+0000 7f26e8400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f26e010e3a0 con 0x7f26e0108790 2024-04-25T18:37:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.450+0000 7f26e8400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f26e010e6a0 con 0x7f26e0103530 2024-04-25T18:37:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.450+0000 7f26e5a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26e0103530 0x7f26e019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:37:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.450+0000 7f26e5a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26e0103530 0x7f26e019aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:41402/0 (socket says 172.21.15.112:41402) 2024-04-25T18:37:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.450+0000 7f26e5a00700 1 -- 172.21.15.112:0/3539109169 learned_addr learned my addr 172.21.15.112:0/3539109169 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:37:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.450+0000 7f26e6400700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e0108790 0x7f26e01a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:37:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.451+0000 7f26e5000700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0105e60 0x7f26e019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.451+0000 7f26e5a00700 1 -- 172.21.15.112:0/3539109169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e0108790 msgr2=0x7f26e01a2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.451+0000 7f26e5a00700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e0108790 0x7f26e01a2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.451+0000 7f26e5a00700 1 -- 172.21.15.112:0/3539109169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0105e60 msgr2=0x7f26e019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.451+0000 7f26e5a00700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0105e60 0x7f26e019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.451+0000 7f26e5a00700 1 -- 172.21.15.112:0/3539109169 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e01a2b30 con 0x7f26e0103530 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.451+0000 7f26e5000700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0105e60 0x7f26e019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.451+0000 7f26e5a00700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26e0103530 0x7f26e019aed0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f26d4000c00 tx=0x7f26d4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.451+0000 7f26de000700 1 -- 172.21.15.112:0/3539109169 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4026070 con 0x7f26e0103530 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.452+0000 7f26de000700 1 -- 172.21.15.112:0/3539109169 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26d4021410 con 0x7f26e0103530 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.452+0000 7f26e8400700 1 -- 172.21.15.112:0/3539109169 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e01a2dc0 con 0x7f26e0103530 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.452+0000 7f26de000700 1 -- 172.21.15.112:0/3539109169 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4017650 con 0x7f26e0103530 2024-04-25T18:37:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.452+0000 7f26e8400700 1 -- 172.21.15.112:0/3539109169 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e01a31c0 con 0x7f26e0103530 2024-04-25T18:37:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.454+0000 7f26e8400700 1 -- 172.21.15.112:0/3539109169 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e0069730 con 0x7f26e0103530 2024-04-25T18:37:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.455+0000 7f26de000700 1 -- 172.21.15.112:0/3539109169 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f26d4029070 con 0x7f26e0103530 2024-04-25T18:37:47.457 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44935]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:37:47.457 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44935]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.457 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44935]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.458 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44935]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:47.458 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44938]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:37:47.458 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44938]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.458 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44938]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.458 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44938]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:47.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.457+0000 7f26de000700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f26c80a3c30 0x7f26c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:37:47.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.457+0000 7f26de000700 1 -- 172.21.15.112:0/3539109169 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f26d40cfe00 con 0x7f26e0103530 2024-04-25T18:37:47.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.457+0000 7f26e5000700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f26c80a3c30 0x7f26c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:37:47.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.458+0000 7f26e5000700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f26c80a3c30 0x7f26c80a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009be0 tx=0x7f26cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:37:47.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.461+0000 7f26de000700 1 -- 172.21.15.112:0/3539109169 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26d405dff0 con 0x7f26e0103530 2024-04-25T18:37:47.596 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40886]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:37:47.596 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40886]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.596 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40886]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.596 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40886]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:47.597 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40890]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:37:47.597 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40890]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.597 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40890]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.597 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40890]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:47.682 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42206]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:37:47.682 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42206]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.682 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42206]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.682 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42206]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:47.683 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42209]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:37:47.683 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42209]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.683 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42209]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.683 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42209]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:47.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42212]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:37:47.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42212]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42212]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42212]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:47.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42215]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:37:47.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42215]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42215]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 sudo[42215]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:47.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:47.704+0000 7f26e8400700 1 -- 172.21.15.112:0/3539109169 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f26e0062e80 con 0x7f26c80a3c30 2024-04-25T18:37:47.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44963]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:37:47.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44963]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44963]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44963]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:47.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44966]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:37:47.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44966]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44966]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 sudo[44966]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40893]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:37:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40893]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40893]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40893]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40896]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:37:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40896]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:37:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40896]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:37:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 sudo[40896]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:37:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:37:48.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:37:48.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:37:48.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:37:48.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:37:48.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:37:48.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:37:48.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:37:48.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:37:48.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:37:48.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:37:48.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:48.435 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:47 smithi171 ceph-mon[33266]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T18:37:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:37:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T18:37:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:37:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:37:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:37:48.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:48.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:47 smithi161 ceph-mon[32609]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T18:37:48.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:37:48.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T18:37:48.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:48.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:37:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:37:48.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:37:48.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:37:48.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:37:48.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:37:48.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:37:48.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:47 smithi112 ceph-mon[27864]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T18:37:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:49 smithi171 ceph-mon[33266]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:37:49.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:49 smithi161 ceph-mon[32609]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:37:49.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:49 smithi112 ceph-mon[27864]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:37:50.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:37:50.035+0000 7f26de000700 1 -- 172.21.15.112:0/3539109169 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f26d4009d10 con 0x7f26e0103530 2024-04-25T18:37:50.214 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:50 smithi112 ceph-mon[27864]: pgmap v99: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:50 smithi171 ceph-mon[33266]: pgmap v99: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:50 smithi161 ceph-mon[32609]: pgmap v99: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:51.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:51 smithi171 ceph-mon[33266]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T18:37:51.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:51 smithi161 ceph-mon[32609]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T18:37:51.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:51 smithi112 ceph-mon[27864]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T18:37:52.348 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:52 smithi112 ceph-mon[27864]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:52.348 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:52 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/4260785963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7aae8cd-fec7-46d9-8773-25eaaca3cc78"}]: dispatch 2024-04-25T18:37:52.348 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:52 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7aae8cd-fec7-46d9-8773-25eaaca3cc78"}]: dispatch 2024-04-25T18:37:52.348 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:52 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7aae8cd-fec7-46d9-8773-25eaaca3cc78"}]': finished 2024-04-25T18:37:52.348 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:52 smithi112 ceph-mon[27864]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T18:37:52.348 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:52 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:37:52.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:52 smithi171 ceph-mon[33266]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:52.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:52 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/4260785963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7aae8cd-fec7-46d9-8773-25eaaca3cc78"}]: dispatch 2024-04-25T18:37:52.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:52 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7aae8cd-fec7-46d9-8773-25eaaca3cc78"}]: dispatch 2024-04-25T18:37:52.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:52 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7aae8cd-fec7-46d9-8773-25eaaca3cc78"}]': finished 2024-04-25T18:37:52.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:52 smithi171 ceph-mon[33266]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T18:37:52.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:52 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:37:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:52 smithi161 ceph-mon[32609]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:52 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/4260785963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7aae8cd-fec7-46d9-8773-25eaaca3cc78"}]: dispatch 2024-04-25T18:37:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:52 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7aae8cd-fec7-46d9-8773-25eaaca3cc78"}]: dispatch 2024-04-25T18:37:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:52 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7aae8cd-fec7-46d9-8773-25eaaca3cc78"}]': finished 2024-04-25T18:37:52.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:52 smithi161 ceph-mon[32609]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T18:37:52.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:52 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:37:53.374 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:53 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/237017513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:37:53.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:53 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/237017513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:37:53.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:53 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/237017513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:37:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:54 smithi171 ceph-mon[33266]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:54 smithi161 ceph-mon[32609]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:54 smithi112 ceph-mon[27864]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:56.374 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:56 smithi112 ceph-mon[27864]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:56.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:56 smithi171 ceph-mon[33266]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:56.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:56 smithi161 ceph-mon[32609]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:58.406 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:37:58 smithi112 ceph-mon[27864]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:58.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:37:58 smithi171 ceph-mon[33266]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:37:58.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:37:58 smithi161 ceph-mon[32609]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:00 smithi171 ceph-mon[33266]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:00 smithi161 ceph-mon[32609]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:00.472 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:00 smithi112 ceph-mon[27864]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:02.416 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:02 smithi112 ceph-mon[27864]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:02.417 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:02 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T18:38:02.417 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:02 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:02 smithi171 ceph-mon[33266]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:02 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T18:38:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:02 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:02 smithi161 ceph-mon[32609]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:02 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T18:38:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:02 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:03.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:03 smithi171 ceph-mon[33266]: Deploying daemon osd.3 on smithi112 2024-04-25T18:38:03.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:03 smithi161 ceph-mon[32609]: Deploying daemon osd.3 on smithi112 2024-04-25T18:38:03.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:03 smithi112 ceph-mon[27864]: Deploying daemon osd.3 on smithi112 2024-04-25T18:38:04.324 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:04 smithi112 ceph-mon[27864]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:04.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:04 smithi171 ceph-mon[33266]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:04.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:04 smithi161 ceph-mon[32609]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:06.410 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:06 smithi112 ceph-mon[27864]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:06 smithi171 ceph-mon[33266]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:06 smithi161 ceph-mon[32609]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:08.420 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:08 smithi112 ceph-mon[27864]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:08.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:08 smithi171 ceph-mon[33266]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:08 smithi161 ceph-mon[32609]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:10 smithi112 ceph-mon[27864]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:10.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:10 smithi171 ceph-mon[33266]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:10.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:10 smithi161 ceph-mon[32609]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:11.464 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:11 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:11.464 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:11 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:11.464 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:11 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:11.464 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:11 smithi171 ceph-mon[33266]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:11.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:11 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:11.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:11 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:11.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:11 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:11.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:11 smithi161 ceph-mon[32609]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:11.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:11.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:11.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:11.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:11 smithi112 ceph-mon[27864]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:14.384 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:14 smithi112 ceph-mon[27864]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:14.384 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:14.384 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:14.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:14 smithi171 ceph-mon[33266]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:14.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:14.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:14 smithi161 ceph-mon[32609]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:15.143 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:15 smithi112 ceph-mon[27864]: from='osd.3 [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:38:15.144 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:15 smithi112 ceph-mon[27864]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:38:15.144 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:15 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:15.144 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:15 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:15.144 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:15 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:15.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.351+0000 7f26de000700 1 -- 172.21.15.112:0/3539109169 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f26e0062e80 con 0x7f26c80a3c30 2024-04-25T18:38:15.354 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 3 on host 'smithi112' 2024-04-25T18:38:15.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.354+0000 7f26e8400700 1 -- 172.21.15.112:0/3539109169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f26c80a3c30 msgr2=0x7f26c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:38:15.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.355+0000 7f26e8400700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f26c80a3c30 0x7f26c80a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009be0 tx=0x7f26cc009260 comp rx=0 tx=0).stop 2024-04-25T18:38:15.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.355+0000 7f26e8400700 1 -- 172.21.15.112:0/3539109169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26e0103530 msgr2=0x7f26e019aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:38:15.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.355+0000 7f26e8400700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26e0103530 0x7f26e019aed0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f26d4000c00 tx=0x7f26d4009580 comp rx=0 tx=0).stop 2024-04-25T18:38:15.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.355+0000 7f26e8400700 1 -- 172.21.15.112:0/3539109169 shutdown_connections 2024-04-25T18:38:15.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.355+0000 7f26e8400700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f26c80a3c30 0x7f26c80a60f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:15.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.355+0000 7f26e8400700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26e0108790 0x7f26e01a2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:15.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.355+0000 7f26e8400700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26e0103530 0x7f26e019aed0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:15.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.355+0000 7f26e8400700 1 --2- 172.21.15.112:0/3539109169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26e0105e60 0x7f26e019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:15.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.355+0000 7f26e8400700 1 -- 172.21.15.112:0/3539109169 >> 172.21.15.112:0/3539109169 conn(0x7f26e00fcf40 msgr2=0x7f26e0102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:38:15.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.355+0000 7f26e8400700 1 -- 172.21.15.112:0/3539109169 shutdown_connections 2024-04-25T18:38:15.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:38:15.355+0000 7f26e8400700 1 -- 172.21.15.112:0/3539109169 wait complete. 2024-04-25T18:38:15.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:15 smithi171 ceph-mon[33266]: from='osd.3 [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:38:15.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:15 smithi171 ceph-mon[33266]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:38:15.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:15 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:15.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:15 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:15.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:15 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:15.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:15 smithi161 ceph-mon[32609]: from='osd.3 [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:38:15.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:15 smithi161 ceph-mon[32609]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:38:15.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:15 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:15.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:15 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:15.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:15 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:16.239 DEBUG:teuthology.orchestra.run.smithi112:osd.3> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T18:38:16.244 INFO:tasks.cephadm:Deploying osd.4 on smithi161 with /dev/vg_nvme/lv_3... 2024-04-25T18:38:16.244 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T18:38:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T18:38:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: from='osd.3 [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:38:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T18:38:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:38:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:38:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T18:38:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:16.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:16.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T18:38:16.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: from='osd.3 [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:38:16.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T18:38:16.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:16.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:38:16.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:16.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:16.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:16.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:38:16.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T18:38:16.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T18:38:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: from='osd.3 [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:38:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T18:38:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:38:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:38:16.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T18:38:16.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:16.487 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:38:17.092 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:38:17 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3[47225]: 2024-04-25T18:38:17.065+0000 7ff504000700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T18:38:17.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:17 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:17.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:17 smithi161 ceph-mon[32609]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T18:38:17.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:17 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:17.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:17 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:17.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:17 smithi171 ceph-mon[33266]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T18:38:17.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:17 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:17.474 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:38:17 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3[47225]: 2024-04-25T18:38:17.108+0000 7ff4f9e00700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:38:17.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:17 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:17.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:17 smithi112 ceph-mon[27864]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T18:38:17.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:17 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:18 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:38:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:18 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:38:18.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:18 smithi161 ceph-mon[32609]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:18.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:18 smithi161 ceph-mon[32609]: from='osd.3 ' entity='osd.3' 2024-04-25T18:38:18.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:18 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:18.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:18 smithi161 ceph-mon[32609]: osd.3 [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156] boot 2024-04-25T18:38:18.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:18 smithi161 ceph-mon[32609]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T18:38:18.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:18 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:18.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:18 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:38:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:18 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:38:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:18 smithi112 ceph-mon[27864]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:18 smithi112 ceph-mon[27864]: from='osd.3 ' entity='osd.3' 2024-04-25T18:38:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:18 smithi112 ceph-mon[27864]: osd.3 [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156] boot 2024-04-25T18:38:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:18 smithi112 ceph-mon[27864]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T18:38:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:18.419 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T18:38:18.420 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T18:38:18.420 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-04-25T18:38:18.420 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-04-25T18:38:18.420 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113987 s, 920 MB/s 2024-04-25T18:38:18.420 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 2024-04-25T18:38:18.420 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-04-25T18:38:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:18 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:38:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:18 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:38:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:18 smithi171 ceph-mon[33266]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:38:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:18 smithi171 ceph-mon[33266]: from='osd.3 ' entity='osd.3' 2024-04-25T18:38:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:18 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:18 smithi171 ceph-mon[33266]: osd.3 [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156] boot 2024-04-25T18:38:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:18 smithi171 ceph-mon[33266]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T18:38:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:18 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:38:19.044 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-04-25T18:38:19.284 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:38:19.688 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:19 smithi161 ceph-mon[32609]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T18:38:19.688 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:19 smithi161 ceph-mon[32609]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:19.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:19 smithi112 ceph-mon[27864]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T18:38:19.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:19 smithi112 ceph-mon[27864]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:19.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:19 smithi171 ceph-mon[33266]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T18:38:19.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:19 smithi171 ceph-mon[33266]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:20.649 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:20 smithi161 ceph-mon[32609]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T18:38:20.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.667+0000 7f483d835700 1 -- 172.21.15.161:0/1041302577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48381019a0 msgr2=0x7f4838101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:38:20.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.667+0000 7f483d835700 1 --2- 172.21.15.161:0/1041302577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48381019a0 0x7f4838101dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f48280099c0 tx=0x7f4828009cd0 comp rx=0 tx=0).stop 2024-04-25T18:38:20.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.667+0000 7f483d835700 1 -- 172.21.15.161:0/1041302577 shutdown_connections 2024-04-25T18:38:20.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.667+0000 7f483d835700 1 --2- 172.21.15.161:0/1041302577 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4838103540 0x7f4838109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:20.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.667+0000 7f483d835700 1 --2- 172.21.15.161:0/1041302577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48381019a0 0x7f4838101dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:20.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.667+0000 7f483d835700 1 --2- 172.21.15.161:0/1041302577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838102ba0 0x7f4838103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:20.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.667+0000 7f483d835700 1 -- 172.21.15.161:0/1041302577 >> 172.21.15.161:0/1041302577 conn(0x7f48380fcf40 msgr2=0x7f48380ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:38:20.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.667+0000 7f483d835700 1 -- 172.21.15.161:0/1041302577 shutdown_connections 2024-04-25T18:38:20.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.667+0000 7f483d835700 1 -- 172.21.15.161:0/1041302577 wait complete. 2024-04-25T18:38:20.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.667+0000 7f483d835700 1 Processor -- start 2024-04-25T18:38:20.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.667+0000 7f483d835700 1 -- start start 2024-04-25T18:38:20.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f483d835700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48381019a0 0x7f483806bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:38:20.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f483d835700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4838102ba0 0x7f483819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:38:20.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f483d835700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838103540 0x7f483819ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:38:20.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f483d835700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f483810d490 con 0x7f4838103540 2024-04-25T18:38:20.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f483d835700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f483810d310 con 0x7f48381019a0 2024-04-25T18:38:20.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f483d835700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f483810d610 con 0x7f4838102ba0 2024-04-25T18:38:20.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f4836ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48381019a0 0x7f483806bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:38:20.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f4836ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48381019a0 0x7f483806bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:37344/0 (socket says 172.21.15.161:37344) 2024-04-25T18:38:20.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f4836ffd700 1 -- 172.21.15.161:0/1996703303 learned_addr learned my addr 172.21.15.161:0/1996703303 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-04-25T18:38:20.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f482ffff700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4838102ba0 0x7f483819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:38:20.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f4836ffd700 1 -- 172.21.15.161:0/1996703303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4838102ba0 msgr2=0x7f483819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:38:20.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f4836ffd700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4838102ba0 0x7f483819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:20.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f4836ffd700 1 -- 172.21.15.161:0/1996703303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838103540 msgr2=0x7f483819ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:38:20.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f4836ffd700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838103540 0x7f483819ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:20.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f4836ffd700 1 -- 172.21.15.161:0/1996703303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4838197720 con 0x7f48381019a0 2024-04-25T18:38:20.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f4836ffd700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48381019a0 0x7f483806bcd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4828009f50 tx=0x7f48280057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:38:20.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f483c833700 1 -- 172.21.15.161:0/1996703303 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828027070 con 0x7f48381019a0 2024-04-25T18:38:20.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f483c833700 1 -- 172.21.15.161:0/1996703303 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4828022470 con 0x7f48381019a0 2024-04-25T18:38:20.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f483c833700 1 -- 172.21.15.161:0/1996703303 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482800f6c0 con 0x7f48381019a0 2024-04-25T18:38:20.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.668+0000 7f483d835700 1 -- 172.21.15.161:0/1996703303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48381979b0 con 0x7f48381019a0 2024-04-25T18:38:20.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.669+0000 7f483d835700 1 -- 172.21.15.161:0/1996703303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4838197e90 con 0x7f48381019a0 2024-04-25T18:38:20.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.669+0000 7f483d835700 1 -- 172.21.15.161:0/1996703303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4838069730 con 0x7f48381019a0 2024-04-25T18:38:20.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.670+0000 7f483c833700 1 -- 172.21.15.161:0/1996703303 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f482802a070 con 0x7f48381019a0 2024-04-25T18:38:20.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.671+0000 7f483c833700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48240a3c40 0x7f48240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:38:20.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.671+0000 7f483c833700 1 -- 172.21.15.161:0/1996703303 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7f48280d05a0 con 0x7f48381019a0 2024-04-25T18:38:20.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.672+0000 7f482ffff700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48240a3c40 0x7f48240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:38:20.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.673+0000 7f482ffff700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48240a3c40 0x7f48240a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f483806c250 tx=0x7f4820009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:38:20.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.675+0000 7f483c833700 1 -- 172.21.15.161:0/1996703303 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4828031080 con 0x7f48381019a0 2024-04-25T18:38:20.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:20 smithi112 ceph-mon[27864]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T18:38:20.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:20.882+0000 7f483d835700 1 -- 172.21.15.161:0/1996703303 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f48381088f0 con 0x7f48240a3c40 2024-04-25T18:38:20.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:20 smithi171 ceph-mon[33266]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T18:38:21.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T18:38:21.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:38:21.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:38:21.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:38:21.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:21.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:21.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:21.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:21.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:21.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:21.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:21.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:21.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:21.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:21 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:21.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T18:38:21.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:38:21.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:38:21.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:38:21.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:21.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:21.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:21.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:21.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:21.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:21.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:21.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:21.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:21.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:21 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:21.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T18:38:21.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:38:21.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:38:21.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:38:21.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:21.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:21.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:21.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:21.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:21.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:21.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:21.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:21.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:21.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:21 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:22.667 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:22 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi112 2024-04-25T18:38:22.667 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:22 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T18:38:22.668 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:22 smithi161 ceph-mon[32609]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T18:38:22.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:22 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi112 2024-04-25T18:38:22.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:22 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T18:38:22.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:22 smithi112 ceph-mon[27864]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T18:38:22.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:22 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi112 2024-04-25T18:38:22.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:22 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T18:38:22.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:22 smithi171 ceph-mon[33266]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T18:38:23.901 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:23 smithi161 ceph-mon[32609]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:23.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:23 smithi171 ceph-mon[33266]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:23.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:23 smithi112 ceph-mon[27864]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:24.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:24 smithi171 ceph-mon[33266]: from='client.? 172.21.15.161:0/2027353628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1"}]: dispatch 2024-04-25T18:38:24.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:24 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1"}]: dispatch 2024-04-25T18:38:24.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:24 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1"}]': finished 2024-04-25T18:38:24.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:24 smithi171 ceph-mon[33266]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T18:38:24.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:24 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:24.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:24 smithi171 ceph-mon[33266]: from='client.? 172.21.15.161:0/3891046089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:38:24.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:24 smithi161 ceph-mon[32609]: from='client.? 172.21.15.161:0/2027353628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1"}]: dispatch 2024-04-25T18:38:24.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:24 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1"}]: dispatch 2024-04-25T18:38:24.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:24 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1"}]': finished 2024-04-25T18:38:24.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:24 smithi161 ceph-mon[32609]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T18:38:24.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:24 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:24.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:24 smithi161 ceph-mon[32609]: from='client.? 172.21.15.161:0/3891046089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:38:24.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:24 smithi112 ceph-mon[27864]: from='client.? 172.21.15.161:0/2027353628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1"}]: dispatch 2024-04-25T18:38:24.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:24 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1"}]: dispatch 2024-04-25T18:38:24.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:24 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1"}]': finished 2024-04-25T18:38:24.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:24 smithi112 ceph-mon[27864]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T18:38:24.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:24 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:24.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:24 smithi112 ceph-mon[27864]: from='client.? 172.21.15.161:0/3891046089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:38:25.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:25 smithi112 ceph-mon[27864]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T18:38:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:25 smithi171 ceph-mon[33266]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T18:38:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:25 smithi161 ceph-mon[32609]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T18:38:28.427 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:28 smithi112 ceph-mon[27864]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T18:38:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:28 smithi171 ceph-mon[33266]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T18:38:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:28 smithi161 ceph-mon[32609]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T18:38:30.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:30 smithi161 ceph-mon[32609]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:38:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:30 smithi171 ceph-mon[33266]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:38:30.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:30 smithi112 ceph-mon[27864]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:38:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T18:38:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T18:38:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:31.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T18:38:31.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:32.431 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:32 smithi161 ceph-mon[32609]: Deploying daemon osd.4 on smithi161 2024-04-25T18:38:32.431 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:32 smithi161 ceph-mon[32609]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:38:32.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:32 smithi171 ceph-mon[33266]: Deploying daemon osd.4 on smithi161 2024-04-25T18:38:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:32 smithi171 ceph-mon[33266]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:38:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:32 smithi112 ceph-mon[27864]: Deploying daemon osd.4 on smithi161 2024-04-25T18:38:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:32 smithi112 ceph-mon[27864]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:38:34.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:34 smithi161 ceph-mon[32609]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:38:34.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:34 smithi171 ceph-mon[33266]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:38:34.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:34 smithi112 ceph-mon[27864]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:38:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:36 smithi171 ceph-mon[33266]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T18:38:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:36 smithi161 ceph-mon[32609]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T18:38:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:36 smithi112 ceph-mon[27864]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T18:38:37.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:37.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:37.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:37.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:37.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:37.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:37.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:37.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:37.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:38 smithi171 ceph-mon[33266]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:38 smithi161 ceph-mon[32609]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:38 smithi112 ceph-mon[27864]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:39.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.615+0000 7f483c833700 1 -- 172.21.15.161:0/1996703303 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f48381088f0 con 0x7f48240a3c40 2024-04-25T18:38:39.618 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 4 on host 'smithi161' 2024-04-25T18:38:39.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 -- 172.21.15.161:0/1996703303 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48240a3c40 msgr2=0x7f48240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:38:39.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48240a3c40 0x7f48240a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f483806c250 tx=0x7f4820009190 comp rx=0 tx=0).stop 2024-04-25T18:38:39.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 -- 172.21.15.161:0/1996703303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48381019a0 msgr2=0x7f483806bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:38:39.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48381019a0 0x7f483806bcd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4828009f50 tx=0x7f48280057a0 comp rx=0 tx=0).stop 2024-04-25T18:38:39.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 -- 172.21.15.161:0/1996703303 shutdown_connections 2024-04-25T18:38:39.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48240a3c40 0x7f48240a6100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:39.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838103540 0x7f483819ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:39.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48381019a0 0x7f483806bcd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:39.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 --2- 172.21.15.161:0/1996703303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4838102ba0 0x7f483819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:39.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 -- 172.21.15.161:0/1996703303 >> 172.21.15.161:0/1996703303 conn(0x7f48380fcf40 msgr2=0x7f4838105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:38:39.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 -- 172.21.15.161:0/1996703303 shutdown_connections 2024-04-25T18:38:39.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:38:39.618+0000 7f483d835700 1 -- 172.21.15.161:0/1996703303 wait complete. 2024-04-25T18:38:39.941 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:39.942 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:39.942 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:39.942 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:39.942 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:39.942 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:39 smithi161 ceph-mon[32609]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:39.942 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:39.942 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:39.942 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:39.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:39.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:39 smithi112 ceph-mon[27864]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:39.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:40.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:40.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:40.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:40.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:40.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:40.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:39 smithi171 ceph-mon[33266]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:40.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:40.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:40.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:38:40.268 DEBUG:teuthology.orchestra.run.smithi161:osd.4> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T18:38:40.270 INFO:tasks.cephadm:Deploying osd.5 on smithi171 with /dev/vg_nvme/lv_3... 2024-04-25T18:38:40.270 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T18:38:40.516 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:38:40.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:40 smithi171 ceph-mon[33266]: from='osd.4 [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:38:40.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:40 smithi171 ceph-mon[33266]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:38:40.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:40 smithi161 ceph-mon[32609]: from='osd.4 [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:38:40.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:40 smithi161 ceph-mon[32609]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:38:40.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:40 smithi112 ceph-mon[27864]: from='osd.4 [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:38:40.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:40 smithi112 ceph-mon[27864]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:38:41.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:41 smithi171 ceph-mon[33266]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T18:38:41.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:41 smithi171 ceph-mon[33266]: from='osd.4 [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:38:41.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:41 smithi171 ceph-mon[33266]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T18:38:41.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:41 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:41.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:41 smithi171 ceph-mon[33266]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:38:41.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:41 smithi171 ceph-mon[33266]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:41.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:41 smithi171 ceph-mon[33266]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:38:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:41 smithi161 ceph-mon[32609]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T18:38:42.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:41 smithi161 ceph-mon[32609]: from='osd.4 [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:38:42.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:41 smithi161 ceph-mon[32609]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T18:38:42.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:41 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:42.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:41 smithi161 ceph-mon[32609]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:38:42.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:41 smithi161 ceph-mon[32609]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:42.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:41 smithi161 ceph-mon[32609]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:38:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:41 smithi112 ceph-mon[27864]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T18:38:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:41 smithi112 ceph-mon[27864]: from='osd.4 [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:38:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:41 smithi112 ceph-mon[27864]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T18:38:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:41 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:42.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:41 smithi112 ceph-mon[27864]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:38:42.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:41 smithi112 ceph-mon[27864]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:42.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:41 smithi112 ceph-mon[27864]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:38:42.567 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T18:38:42.567 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T18:38:42.568 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-25T18:38:42.568 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-25T18:38:42.568 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0144315 s, 727 MB/s 2024-04-25T18:38:42.568 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-25T18:38:42.897 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:42 smithi161 ceph-mon[32609]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T18:38:42.897 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:42.897 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:42 smithi161 ceph-mon[32609]: from='osd.4 ' entity='osd.4' 2024-04-25T18:38:42.897 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:42.898 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:38:42 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:38:42.658+0000 7f7b8644e700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T18:38:42.898 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:38:42 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:38:42.790+0000 7f7b7e239700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:38:43.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:42 smithi171 ceph-mon[33266]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T18:38:43.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:43.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:42 smithi171 ceph-mon[33266]: from='osd.4 ' entity='osd.4' 2024-04-25T18:38:43.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:43.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:42 smithi112 ceph-mon[27864]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T18:38:43.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:43.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:42 smithi112 ceph-mon[27864]: from='osd.4 ' entity='osd.4' 2024-04-25T18:38:43.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:43.255 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_3 2024-04-25T18:38:43.532 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:38:43.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:38:43.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:38:43.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:43.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:43.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:43.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:43.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:43.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:43.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:43.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:43.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:43.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:38:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:38:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:44.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:44.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:44.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:44.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:44.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:44.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:44.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:38:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:38:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:38:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:38:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:38:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:38:44.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:44.921 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:44 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi161 2024-04-25T18:38:44.921 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:44 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi161 to 8544M 2024-04-25T18:38:44.921 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:44 smithi171 ceph-mon[33266]: osd.4 [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306] boot 2024-04-25T18:38:44.921 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:44 smithi171 ceph-mon[33266]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T18:38:44.921 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:44 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:45.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.138+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2124203564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc2c103ca0 msgr2=0x7fbc2c104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:38:45.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.138+0000 7fbc33ef6700 1 --2- 172.21.15.171:0/2124203564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc2c103ca0 0x7fbc2c104100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009990 tx=0x7fbc1c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:38:45.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2124203564 shutdown_connections 2024-04-25T18:38:45.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 --2- 172.21.15.171:0/2124203564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c104640 0x7fbc2c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:45.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 --2- 172.21.15.171:0/2124203564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc2c102aa0 0x7fbc2c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:45.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 --2- 172.21.15.171:0/2124203564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc2c103ca0 0x7fbc2c104100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:45.140 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2124203564 >> 172.21.15.171:0/2124203564 conn(0x7fbc2c0fe040 msgr2=0x7fbc2c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:38:45.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2124203564 shutdown_connections 2024-04-25T18:38:45.141 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2124203564 wait complete. 2024-04-25T18:38:45.142 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 Processor -- start 2024-04-25T18:38:45.142 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 -- start start 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc2c102aa0 0x7fbc2c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc2c103ca0 0x7fbc2c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c104640 0x7fbc2c1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c10e5d0 con 0x7fbc2c102aa0 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c10e450 con 0x7fbc2c104640 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.139+0000 7fbc33ef6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c10e750 con 0x7fbc2c103ca0 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc31491700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc2c103ca0 0x7fbc2c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc31491700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc2c103ca0 0x7fbc2c1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:34286/0 (socket says 172.21.15.171:34286) 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc31491700 1 -- 172.21.15.171:0/2351109405 learned_addr learned my addr 172.21.15.171:0/2351109405 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc32493700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c104640 0x7fbc2c1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc31491700 1 -- 172.21.15.171:0/2351109405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c104640 msgr2=0x7fbc2c1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:38:45.143 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc31491700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c104640 0x7fbc2c1a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:45.144 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc31491700 1 -- 172.21.15.171:0/2351109405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc2c102aa0 msgr2=0x7fbc2c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:38:45.144 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc31491700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc2c102aa0 0x7fbc2c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:38:45.144 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc31491700 1 -- 172.21.15.171:0/2351109405 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c1a0900 con 0x7fbc2c103ca0 2024-04-25T18:38:45.144 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc32493700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c104640 0x7fbc2c1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:38:45.144 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc31491700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc2c103ca0 0x7fbc2c1991c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0036f0 tx=0x7fbc1c0039f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:38:45.144 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc22ffd700 1 -- 172.21.15.171:0/2351109405 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c027070 con 0x7fbc2c103ca0 2024-04-25T18:38:45.144 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc22ffd700 1 -- 172.21.15.171:0/2351109405 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc1c003f60 con 0x7fbc2c103ca0 2024-04-25T18:38:45.144 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2351109405 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c1a0b90 con 0x7fbc2c103ca0 2024-04-25T18:38:45.144 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc22ffd700 1 -- 172.21.15.171:0/2351109405 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c022e60 con 0x7fbc2c103ca0 2024-04-25T18:38:45.144 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.140+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2351109405 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c1a0fe0 con 0x7fbc2c103ca0 2024-04-25T18:38:45.144 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.141+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2351109405 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2c02df80 con 0x7fbc2c103ca0 2024-04-25T18:38:45.145 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.142+0000 7fbc22ffd700 1 -- 172.21.15.171:0/2351109405 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fbc1c02a070 con 0x7fbc2c103ca0 2024-04-25T18:38:45.145 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.143+0000 7fbc22ffd700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbc180a3c40 0x7fbc180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:38:45.145 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.143+0000 7fbc22ffd700 1 -- 172.21.15.171:0/2351109405 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4101+0+0 (secure 0 0 0) 0x7fbc1c0d0950 con 0x7fbc2c103ca0 2024-04-25T18:38:45.145 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.143+0000 7fbc31c92700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbc180a3c40 0x7fbc180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:38:45.145 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.144+0000 7fbc31c92700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbc180a3c40 0x7fbc180a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c199b60 tx=0x7fbc28006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:38:45.148 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.147+0000 7fbc22ffd700 1 -- 172.21.15.171:0/2351109405 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc1c013070 con 0x7fbc2c103ca0 2024-04-25T18:38:45.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:44 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi161 2024-04-25T18:38:45.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:44 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi161 to 8544M 2024-04-25T18:38:45.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:44 smithi161 ceph-mon[32609]: osd.4 [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306] boot 2024-04-25T18:38:45.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:44 smithi161 ceph-mon[32609]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T18:38:45.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:44 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:44 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi161 2024-04-25T18:38:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:44 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi161 to 8544M 2024-04-25T18:38:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:44 smithi112 ceph-mon[27864]: osd.4 [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306] boot 2024-04-25T18:38:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:44 smithi112 ceph-mon[27864]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T18:38:45.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:44 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:38:45.357 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:38:45.355+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2351109405 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbc2c062e80 con 0x7fbc180a3c40 2024-04-25T18:38:46.121 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:45 smithi171 ceph-mon[33266]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T18:38:46.121 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:45 smithi171 ceph-mon[33266]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:46.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:45 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:38:46.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:45 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:38:46.122 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:45 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:46.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:45 smithi161 ceph-mon[32609]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T18:38:46.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:45 smithi161 ceph-mon[32609]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:46.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:45 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:38:46.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:45 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:38:46.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:45 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:45 smithi112 ceph-mon[27864]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T18:38:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:45 smithi112 ceph-mon[27864]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:45 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:38:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:45 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:38:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:45 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:47.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:46 smithi161 ceph-mon[32609]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:38:47.169 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:46 smithi171 ceph-mon[33266]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:38:47.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:46 smithi112 ceph-mon[27864]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:38:48.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:47 smithi171 ceph-mon[33266]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:48.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:47 smithi161 ceph-mon[32609]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:48.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:47 smithi112 ceph-mon[27864]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:49.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:48 smithi171 ceph-mon[33266]: from='client.? 172.21.15.171:0/771437388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56124d12-bc1f-4cba-8430-938248851776"}]: dispatch 2024-04-25T18:38:49.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:48 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56124d12-bc1f-4cba-8430-938248851776"}]: dispatch 2024-04-25T18:38:49.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:48 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56124d12-bc1f-4cba-8430-938248851776"}]': finished 2024-04-25T18:38:49.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:48 smithi171 ceph-mon[33266]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T18:38:49.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:38:49.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:48 smithi171 ceph-mon[33266]: from='client.? 172.21.15.171:0/2841507325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:38:49.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:48 smithi161 ceph-mon[32609]: from='client.? 172.21.15.171:0/771437388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56124d12-bc1f-4cba-8430-938248851776"}]: dispatch 2024-04-25T18:38:49.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:48 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56124d12-bc1f-4cba-8430-938248851776"}]: dispatch 2024-04-25T18:38:49.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:48 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56124d12-bc1f-4cba-8430-938248851776"}]': finished 2024-04-25T18:38:49.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:48 smithi161 ceph-mon[32609]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T18:38:49.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:38:49.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:48 smithi161 ceph-mon[32609]: from='client.? 172.21.15.171:0/2841507325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:38:49.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:48 smithi112 ceph-mon[27864]: from='client.? 172.21.15.171:0/771437388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56124d12-bc1f-4cba-8430-938248851776"}]: dispatch 2024-04-25T18:38:49.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:48 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56124d12-bc1f-4cba-8430-938248851776"}]: dispatch 2024-04-25T18:38:49.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:48 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56124d12-bc1f-4cba-8430-938248851776"}]': finished 2024-04-25T18:38:49.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:48 smithi112 ceph-mon[27864]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T18:38:49.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:38:49.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:48 smithi112 ceph-mon[27864]: from='client.? 172.21.15.171:0/2841507325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:38:50.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:49 smithi171 ceph-mon[33266]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:50.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:49 smithi161 ceph-mon[32609]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:50.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:49 smithi112 ceph-mon[27864]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:52.342 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:52 smithi171 ceph-mon[33266]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:52 smithi161 ceph-mon[32609]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:52 smithi112 ceph-mon[27864]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:54.347 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:54 smithi171 ceph-mon[33266]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:54 smithi161 ceph-mon[32609]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:54 smithi112 ceph-mon[27864]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:56.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:56 smithi171 ceph-mon[33266]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:56.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T18:38:56.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:56.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:56 smithi161 ceph-mon[32609]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:56.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T18:38:56.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:56.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:56 smithi112 ceph-mon[27864]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:56.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T18:38:56.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:38:57.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:57 smithi171 ceph-mon[33266]: Deploying daemon osd.5 on smithi171 2024-04-25T18:38:57.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:57 smithi161 ceph-mon[32609]: Deploying daemon osd.5 on smithi171 2024-04-25T18:38:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:57 smithi112 ceph-mon[27864]: Deploying daemon osd.5 on smithi171 2024-04-25T18:38:58.374 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:38:58 smithi112 ceph-mon[27864]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:58.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:38:58 smithi161 ceph-mon[32609]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:38:58.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:38:58 smithi171 ceph-mon[33266]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:00.416 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:00 smithi171 ceph-mon[33266]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:00 smithi161 ceph-mon[32609]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:00.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:00 smithi112 ceph-mon[27864]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:02.398 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:02 smithi171 ceph-mon[33266]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:02.399 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:02 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:02.399 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:02 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:02.399 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:02 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:02 smithi161 ceph-mon[32609]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:02 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:02 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:02 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:02 smithi112 ceph-mon[27864]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:02 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:02 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:02 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:04.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:04 smithi171 ceph-mon[33266]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:04.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:04 smithi161 ceph-mon[32609]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:04.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:04 smithi112 ceph-mon[27864]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:04.790 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.788+0000 7fbc22ffd700 1 -- 172.21.15.171:0/2351109405 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbc2c062e80 con 0x7fbc180a3c40 2024-04-25T18:39:04.790 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 5 on host 'smithi171' 2024-04-25T18:39:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2351109405 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbc180a3c40 msgr2=0x7fbc180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:39:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbc180a3c40 0x7fbc180a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c199b60 tx=0x7fbc28006d20 comp rx=0 tx=0).stop 2024-04-25T18:39:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2351109405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc2c103ca0 msgr2=0x7fbc2c1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:39:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc2c103ca0 0x7fbc2c1991c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0036f0 tx=0x7fbc1c0039f0 comp rx=0 tx=0).stop 2024-04-25T18:39:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2351109405 shutdown_connections 2024-04-25T18:39:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbc180a3c40 0x7fbc180a6100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c104640 0x7fbc2c1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:04.792 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc2c102aa0 0x7fbc2c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 --2- 172.21.15.171:0/2351109405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc2c103ca0 0x7fbc2c1991c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2351109405 >> 172.21.15.171:0/2351109405 conn(0x7fbc2c0fe040 msgr2=0x7fbc2c107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:39:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2351109405 shutdown_connections 2024-04-25T18:39:04.793 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:39:04.791+0000 7fbc33ef6700 1 -- 172.21.15.171:0/2351109405 wait complete. 2024-04-25T18:39:05.432 DEBUG:teuthology.orchestra.run.smithi171:osd.5> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T18:39:05.434 INFO:tasks.cephadm:Deploying osd.6 on smithi112 with /dev/vg_nvme/lv_2... 2024-04-25T18:39:05.435 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T18:39:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:39:05.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:05 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:05 smithi112 ceph-mon[27864]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:05 smithi112 ceph-mon[27864]: from='osd.5 [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:39:05.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:05 smithi112 ceph-mon[27864]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:39:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:05 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:05 smithi171 ceph-mon[33266]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:05 smithi171 ceph-mon[33266]: from='osd.5 [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:39:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:05 smithi171 ceph-mon[33266]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:39:06.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:05 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:06.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:05 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:06.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:05 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:06.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:05 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:06.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:05 smithi161 ceph-mon[32609]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:06.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:05 smithi161 ceph-mon[32609]: from='osd.5 [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:39:06.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:05 smithi161 ceph-mon[32609]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:39:06.918 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:06 smithi171 ceph-mon[33266]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T18:39:06.919 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:06 smithi171 ceph-mon[33266]: from='osd.5 [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:39:06.919 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:06 smithi171 ceph-mon[33266]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T18:39:06.919 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:06.919 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:06 smithi171 ceph-mon[33266]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:39:06.919 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:39:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:39:06.853+0000 7fcbbf963700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T18:39:06.919 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:39:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:39:06.863+0000 7fcbbb756700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:39:07.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:06 smithi161 ceph-mon[32609]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T18:39:07.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:06 smithi161 ceph-mon[32609]: from='osd.5 [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:39:07.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:06 smithi161 ceph-mon[32609]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T18:39:07.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:07.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:06 smithi161 ceph-mon[32609]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:39:07.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:06 smithi112 ceph-mon[27864]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T18:39:07.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:06 smithi112 ceph-mon[27864]: from='osd.5 [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:39:07.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:06 smithi112 ceph-mon[27864]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T18:39:07.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:07.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:06 smithi112 ceph-mon[27864]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:39:08.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:07 smithi112 ceph-mon[27864]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:39:08.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:07 smithi112 ceph-mon[27864]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T18:39:08.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:07 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:08.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:07 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:08.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:07 smithi112 ceph-mon[27864]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:08.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:07 smithi112 ceph-mon[27864]: osd.5 [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313] boot 2024-04-25T18:39:08.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:07 smithi112 ceph-mon[27864]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T18:39:08.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:07 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:08.060 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:07 smithi171 ceph-mon[33266]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:39:08.060 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:07 smithi171 ceph-mon[33266]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T18:39:08.060 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:07 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:08.060 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:07 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:08.060 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:07 smithi171 ceph-mon[33266]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:08.060 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:07 smithi171 ceph-mon[33266]: osd.5 [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313] boot 2024-04-25T18:39:08.060 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:07 smithi171 ceph-mon[33266]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T18:39:08.060 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:07 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:07 smithi161 ceph-mon[32609]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:39:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:07 smithi161 ceph-mon[32609]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T18:39:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:07 smithi161 ceph-mon[32609]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:39:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:07 smithi161 ceph-mon[32609]: osd.5 [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313] boot 2024-04-25T18:39:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:07 smithi161 ceph-mon[32609]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T18:39:08.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:39:09.035 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T18:39:09.035 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T18:39:09.035 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T18:39:09.035 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T18:39:09.035 INFO:teuthology.orchestra.run.smithi112.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0339905 s, 308 MB/s 2024-04-25T18:39:09.036 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T18:39:09.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:39:09.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:39:09.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:09.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:09.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T18:39:09.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:09.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:09.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:09.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:09.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:09.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:39:09.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:09.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:09.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:09.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:09.976 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-04-25T18:39:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:09 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi171 2024-04-25T18:39:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:09 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-25T18:39:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:09 smithi171 ceph-mon[33266]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:09 smithi171 ceph-mon[33266]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T18:39:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:09 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi171 2024-04-25T18:39:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:09 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-25T18:39:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:09 smithi161 ceph-mon[32609]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:09 smithi161 ceph-mon[32609]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T18:39:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:09 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi171 2024-04-25T18:39:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:09 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-25T18:39:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:09 smithi112 ceph-mon[27864]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:09 smithi112 ceph-mon[27864]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T18:39:10.411 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:39:11.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:11 smithi171 ceph-mon[33266]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T18:39:11.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:11 smithi171 ceph-mon[33266]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:11.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:11 smithi161 ceph-mon[32609]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T18:39:11.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:11 smithi161 ceph-mon[32609]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:11.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:11 smithi112 ceph-mon[27864]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T18:39:11.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:11 smithi112 ceph-mon[27864]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:12.641 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:12 smithi112 ceph-mon[27864]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T18:39:12.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:12 smithi161 ceph-mon[32609]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T18:39:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.807+0000 7f5eafa00700 1 -- 172.21.15.112:0/3387242389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea8104ce0 msgr2=0x7f5ea8105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:39:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.807+0000 7f5eafa00700 1 --2- 172.21.15.112:0/3387242389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea8104ce0 0x7f5ea8105100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0099c0 tx=0x7f5e9c009cd0 comp rx=0 tx=0).stop 2024-04-25T18:39:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.808+0000 7f5eafa00700 1 -- 172.21.15.112:0/3387242389 shutdown_connections 2024-04-25T18:39:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.808+0000 7f5eafa00700 1 --2- 172.21.15.112:0/3387242389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ea8106880 0x7f5ea810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.808+0000 7f5eafa00700 1 --2- 172.21.15.112:0/3387242389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea8104ce0 0x7f5ea8105100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.808+0000 7f5eafa00700 1 --2- 172.21.15.112:0/3387242389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ea8105ee0 0x7f5ea8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.808+0000 7f5eafa00700 1 -- 172.21.15.112:0/3387242389 >> 172.21.15.112:0/3387242389 conn(0x7f5ea81002a0 msgr2=0x7f5ea81026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:39:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.808+0000 7f5eafa00700 1 -- 172.21.15.112:0/3387242389 shutdown_connections 2024-04-25T18:39:12.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.808+0000 7f5eafa00700 1 -- 172.21.15.112:0/3387242389 wait complete. 2024-04-25T18:39:12.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.809+0000 7f5eafa00700 1 Processor -- start 2024-04-25T18:39:12.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.809+0000 7f5eafa00700 1 -- start start 2024-04-25T18:39:12.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.810+0000 7f5eafa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ea8104ce0 0x7f5ea819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:39:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.810+0000 7f5eafa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ea8105ee0 0x7f5ea819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:39:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.810+0000 7f5eafa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea8106880 0x7f5ea81a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:39:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.810+0000 7f5eafa00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea8110750 con 0x7f5ea8106880 2024-04-25T18:39:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.810+0000 7f5eafa00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea81105d0 con 0x7f5ea8104ce0 2024-04-25T18:39:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.810+0000 7f5eafa00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea81108d0 con 0x7f5ea8105ee0 2024-04-25T18:39:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.810+0000 7f5ead000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ea8104ce0 0x7f5ea819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:39:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.810+0000 7f5ead000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ea8104ce0 0x7f5ea819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:35214/0 (socket says 172.21.15.112:35214) 2024-04-25T18:39:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.810+0000 7f5ead000700 1 -- 172.21.15.112:0/818965500 learned_addr learned my addr 172.21.15.112:0/818965500 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:39:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.810+0000 7f5eada00700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea8106880 0x7f5ea81a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:39:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.810+0000 7f5ea7e00700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ea8105ee0 0x7f5ea819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:39:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.811+0000 7f5ead000700 1 -- 172.21.15.112:0/818965500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ea8105ee0 msgr2=0x7f5ea819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:39:12.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.811+0000 7f5ead000700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ea8105ee0 0x7f5ea819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:12.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.811+0000 7f5ead000700 1 -- 172.21.15.112:0/818965500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea8106880 msgr2=0x7f5ea81a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:39:12.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.811+0000 7f5ead000700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea8106880 0x7f5ea81a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:12.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.811+0000 7f5ead000700 1 -- 172.21.15.112:0/818965500 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea81a2a30 con 0x7f5ea8104ce0 2024-04-25T18:39:12.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.811+0000 7f5ead000700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ea8104ce0 0x7f5ea819add0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0036f0 tx=0x7f5e9c00f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:39:12.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.811+0000 7f5ea5600700 1 -- 172.21.15.112:0/818965500 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c009e60 con 0x7f5ea8104ce0 2024-04-25T18:39:12.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.811+0000 7f5ea5600700 1 -- 172.21.15.112:0/818965500 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9c005940 con 0x7f5ea8104ce0 2024-04-25T18:39:12.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.811+0000 7f5eafa00700 1 -- 172.21.15.112:0/818965500 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea81a2cc0 con 0x7f5ea8104ce0 2024-04-25T18:39:12.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.811+0000 7f5ea5600700 1 -- 172.21.15.112:0/818965500 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0054f0 con 0x7f5ea8104ce0 2024-04-25T18:39:12.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.812+0000 7f5eafa00700 1 -- 172.21.15.112:0/818965500 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea81a31a0 con 0x7f5ea8104ce0 2024-04-25T18:39:12.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.813+0000 7f5eafa00700 1 -- 172.21.15.112:0/818965500 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea8069730 con 0x7f5ea8104ce0 2024-04-25T18:39:12.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.815+0000 7f5ea5600700 1 -- 172.21.15.112:0/818965500 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5e9c00fb80 con 0x7f5ea8104ce0 2024-04-25T18:39:12.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.816+0000 7f5ea5600700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5e900a3cc0 0x7f5e900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:39:12.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.816+0000 7f5ea5600700 1 -- 172.21.15.112:0/818965500 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4533+0+0 (secure 0 0 0) 0x7f5e9c0d0280 con 0x7f5ea8104ce0 2024-04-25T18:39:12.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.816+0000 7f5ea7e00700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5e900a3cc0 0x7f5e900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:39:12.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.817+0000 7f5ea7e00700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5e900a3cc0 0x7f5e900a6180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009c60 tx=0x7f5e94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:39:12.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:12.821+0000 7f5ea5600700 1 -- 172.21.15.112:0/818965500 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e9c013070 con 0x7f5ea8104ce0 2024-04-25T18:39:12.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:12 smithi171 ceph-mon[33266]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T18:39:13.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:13.058+0000 7f5eafa00700 1 -- 172.21.15.112:0/818965500 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5ea8062e80 con 0x7f5e900a3cc0 2024-04-25T18:39:13.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:13 smithi112 ceph-mon[27864]: pgmap v163: 1 pgs: 1 remapped+peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:13.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:13 smithi112 ceph-mon[27864]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:39:13.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:13 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:39:13.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:13 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:39:13.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:13 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:13.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:13 smithi171 ceph-mon[33266]: pgmap v163: 1 pgs: 1 remapped+peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:13.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:13 smithi171 ceph-mon[33266]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:39:13.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:13 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:39:13.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:13 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:39:13.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:13 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:13.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:13 smithi161 ceph-mon[32609]: pgmap v163: 1 pgs: 1 remapped+peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:13.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:13 smithi161 ceph-mon[32609]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:39:13.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:13 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:39:13.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:13 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:39:13.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:13 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:16 smithi171 ceph-mon[33266]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:39:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:16 smithi161 ceph-mon[32609]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:39:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:16 smithi112 ceph-mon[27864]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:39:18.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:18 smithi112 ceph-mon[27864]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:39:18.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:18 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2354243995' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9563a9d3-352b-4686-975c-282ec167248b"}]: dispatch 2024-04-25T18:39:18.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:18 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9563a9d3-352b-4686-975c-282ec167248b"}]: dispatch 2024-04-25T18:39:18.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:18 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9563a9d3-352b-4686-975c-282ec167248b"}]': finished 2024-04-25T18:39:18.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:18 smithi112 ceph-mon[27864]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T18:39:18.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:18 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:18 smithi171 ceph-mon[33266]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:39:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:18 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2354243995' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9563a9d3-352b-4686-975c-282ec167248b"}]: dispatch 2024-04-25T18:39:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:18 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9563a9d3-352b-4686-975c-282ec167248b"}]: dispatch 2024-04-25T18:39:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:18 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9563a9d3-352b-4686-975c-282ec167248b"}]': finished 2024-04-25T18:39:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:18 smithi171 ceph-mon[33266]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T18:39:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:18 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:18 smithi161 ceph-mon[32609]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:39:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:18 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2354243995' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9563a9d3-352b-4686-975c-282ec167248b"}]: dispatch 2024-04-25T18:39:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:18 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9563a9d3-352b-4686-975c-282ec167248b"}]: dispatch 2024-04-25T18:39:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:18 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9563a9d3-352b-4686-975c-282ec167248b"}]': finished 2024-04-25T18:39:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:18 smithi161 ceph-mon[32609]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T18:39:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:18 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:19.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:19 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2679286256' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:39:19.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:19 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2679286256' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:39:19.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:19 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2679286256' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:39:20.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:20 smithi171 ceph-mon[33266]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:39:20.466 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:20 smithi161 ceph-mon[32609]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:39:20.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:20 smithi112 ceph-mon[27864]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:39:22.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:22 smithi171 ceph-mon[33266]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:39:22.468 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:22 smithi112 ceph-mon[27864]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:39:22.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:22 smithi161 ceph-mon[32609]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:39:24.213 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:24 smithi112 ceph-mon[27864]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:39:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:24 smithi171 ceph-mon[33266]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:39:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:24 smithi161 ceph-mon[32609]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:39:26.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:26 smithi171 ceph-mon[33266]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:26.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:26 smithi161 ceph-mon[32609]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:26.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:26 smithi112 ceph-mon[27864]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:28 smithi171 ceph-mon[33266]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:28 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T18:39:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:28 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:28 smithi161 ceph-mon[32609]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:28 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T18:39:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:28 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:28 smithi112 ceph-mon[27864]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:28 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T18:39:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:28 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:29.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:29 smithi171 ceph-mon[33266]: Deploying daemon osd.6 on smithi112 2024-04-25T18:39:29.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:29 smithi161 ceph-mon[32609]: Deploying daemon osd.6 on smithi112 2024-04-25T18:39:29.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:29 smithi112 ceph-mon[27864]: Deploying daemon osd.6 on smithi112 2024-04-25T18:39:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:30 smithi171 ceph-mon[33266]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:30 smithi161 ceph-mon[32609]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:30.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:30 smithi112 ceph-mon[27864]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:32 smithi171 ceph-mon[33266]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:32 smithi161 ceph-mon[32609]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:32 smithi112 ceph-mon[27864]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:34.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:34 smithi171 ceph-mon[33266]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:34 smithi161 ceph-mon[32609]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:34.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:34 smithi112 ceph-mon[27864]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:36.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:36 smithi112 ceph-mon[27864]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:36 smithi171 ceph-mon[33266]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:36 smithi161 ceph-mon[32609]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:37.678 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:37.678 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:37.678 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:37.678 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:37 smithi112 ceph-mon[27864]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:37.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:37.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:37.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:37.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:37 smithi161 ceph-mon[32609]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:37.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:37.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:37.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:37.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:37 smithi171 ceph-mon[33266]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:40.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:40 smithi112 ceph-mon[27864]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:40.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:40 smithi171 ceph-mon[33266]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:40.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:40 smithi161 ceph-mon[32609]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:41.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:41 smithi171 ceph-mon[33266]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:39:41.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:41 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:41.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:41 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:41.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:41 smithi161 ceph-mon[32609]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:39:41.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:41 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:41.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:41 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:41.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:41 smithi112 ceph-mon[27864]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:39:41.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:41 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:41.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:41 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:42.122 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 6 on host 'smithi112' 2024-04-25T18:39:42.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.119+0000 7f5ea5600700 1 -- 172.21.15.112:0/818965500 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5ea8062e80 con 0x7f5e900a3cc0 2024-04-25T18:39:42.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.123+0000 7f5eafa00700 1 -- 172.21.15.112:0/818965500 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5e900a3cc0 msgr2=0x7f5e900a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:39:42.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.123+0000 7f5eafa00700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5e900a3cc0 0x7f5e900a6180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009c60 tx=0x7f5e94009260 comp rx=0 tx=0).stop 2024-04-25T18:39:42.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.123+0000 7f5eafa00700 1 -- 172.21.15.112:0/818965500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ea8104ce0 msgr2=0x7f5ea819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:39:42.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.123+0000 7f5eafa00700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ea8104ce0 0x7f5ea819add0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0036f0 tx=0x7f5e9c00f710 comp rx=0 tx=0).stop 2024-04-25T18:39:42.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.124+0000 7f5eafa00700 1 -- 172.21.15.112:0/818965500 shutdown_connections 2024-04-25T18:39:42.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.124+0000 7f5eafa00700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5e900a3cc0 0x7f5e900a6180 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:42.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.124+0000 7f5eafa00700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ea8106880 0x7f5ea81a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:42.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.124+0000 7f5eafa00700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ea8104ce0 0x7f5ea819add0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:42.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.124+0000 7f5eafa00700 1 --2- 172.21.15.112:0/818965500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5ea8105ee0 0x7f5ea819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:42.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.124+0000 7f5eafa00700 1 -- 172.21.15.112:0/818965500 >> 172.21.15.112:0/818965500 conn(0x7f5ea81002a0 msgr2=0x7f5ea8109300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:39:42.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.124+0000 7f5eafa00700 1 -- 172.21.15.112:0/818965500 shutdown_connections 2024-04-25T18:39:42.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:39:42.124+0000 7f5eafa00700 1 -- 172.21.15.112:0/818965500 wait complete. 2024-04-25T18:39:42.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:42 smithi171 ceph-mon[33266]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:42.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:42 smithi171 ceph-mon[33266]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T18:39:42.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:42 smithi171 ceph-mon[33266]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T18:39:42.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:42 smithi171 ceph-mon[33266]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:39:42.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:42.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:42.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:42.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:42.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:42.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:42 smithi161 ceph-mon[32609]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:42 smithi161 ceph-mon[32609]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T18:39:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:42 smithi161 ceph-mon[32609]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T18:39:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:42 smithi161 ceph-mon[32609]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:39:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:42.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:42.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:42.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:42.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:42.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:42.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:42 smithi112 ceph-mon[27864]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:42.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:42 smithi112 ceph-mon[27864]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T18:39:42.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:42 smithi112 ceph-mon[27864]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T18:39:42.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:42 smithi112 ceph-mon[27864]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:39:42.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:42.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:42.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:42.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:42.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:42.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:42.958 DEBUG:teuthology.orchestra.run.smithi112:osd.6> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T18:39:42.962 INFO:tasks.cephadm:Deploying osd.7 on smithi161 with /dev/vg_nvme/lv_2... 2024-04-25T18:39:42.963 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T18:39:43.210 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:39:43.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:43 smithi112 ceph-mon[27864]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:39:43.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:43 smithi112 ceph-mon[27864]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T18:39:43.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:43.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:43.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:43 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:43.332 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:43 smithi161 ceph-mon[32609]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:39:43.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:43 smithi161 ceph-mon[32609]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T18:39:43.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:43.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:43.333 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:43 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:43.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:43 smithi171 ceph-mon[33266]: from='osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:39:43.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:43 smithi171 ceph-mon[33266]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T18:39:43.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:43.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:43.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:43 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:39:44.373 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:44 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:39:44.373 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:44 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:39:44.373 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:44 smithi161 ceph-mon[32609]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:44.374 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:44 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:44.374 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:44 smithi161 ceph-mon[32609]: osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933] boot 2024-04-25T18:39:44.374 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:44 smithi161 ceph-mon[32609]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T18:39:44.374 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:44 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:44 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:39:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:44 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:39:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:44 smithi171 ceph-mon[33266]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:44 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:44 smithi171 ceph-mon[33266]: osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933] boot 2024-04-25T18:39:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:44 smithi171 ceph-mon[33266]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T18:39:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:44 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:44 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:39:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:44 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:39:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:44 smithi112 ceph-mon[27864]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:39:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:44 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:44 smithi112 ceph-mon[27864]: osd.6 [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933] boot 2024-04-25T18:39:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:44 smithi112 ceph-mon[27864]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T18:39:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:44 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:39:45.058 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T18:39:45.058 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T18:39:45.058 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-04-25T18:39:45.059 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-04-25T18:39:45.059 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115046 s, 911 MB/s 2024-04-25T18:39:45.059 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 2024-04-25T18:39:45.059 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-04-25T18:39:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:45 smithi112 ceph-mon[27864]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T18:39:45.360 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:45 smithi161 ceph-mon[32609]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T18:39:45.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:45 smithi171 ceph-mon[33266]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T18:39:45.692 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-04-25T18:39:45.929 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:39:46.319 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:46 smithi161 ceph-mon[32609]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:46 smithi171 ceph-mon[33266]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:46.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:46 smithi112 ceph-mon[27864]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:47.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.348+0000 7ff4cba2b700 1 -- 172.21.15.161:0/223444472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c4104630 msgr2=0x7ff4c410aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:39:47.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.348+0000 7ff4cba2b700 1 --2- 172.21.15.161:0/223444472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c4104630 0x7ff4c410aee0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00b0d0 tx=0x7ff4bc00b3e0 comp rx=0 tx=0).stop 2024-04-25T18:39:47.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.349+0000 7ff4cba2b700 1 -- 172.21.15.161:0/223444472 shutdown_connections 2024-04-25T18:39:47.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.349+0000 7ff4cba2b700 1 --2- 172.21.15.161:0/223444472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c4104630 0x7ff4c410aee0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:47.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.349+0000 7ff4cba2b700 1 --2- 172.21.15.161:0/223444472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4102a90 0x7ff4c4102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:47.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.349+0000 7ff4cba2b700 1 --2- 172.21.15.161:0/223444472 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4103c90 0x7ff4c41040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:47.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.349+0000 7ff4cba2b700 1 -- 172.21.15.161:0/223444472 >> 172.21.15.161:0/223444472 conn(0x7ff4c40fe030 msgr2=0x7ff4c4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:39:47.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.349+0000 7ff4cba2b700 1 -- 172.21.15.161:0/223444472 shutdown_connections 2024-04-25T18:39:47.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.349+0000 7ff4cba2b700 1 -- 172.21.15.161:0/223444472 wait complete. 2024-04-25T18:39:47.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.349+0000 7ff4cba2b700 1 Processor -- start 2024-04-25T18:39:47.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.349+0000 7ff4cba2b700 1 -- start start 2024-04-25T18:39:47.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.349+0000 7ff4cba2b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4102a90 0x7ff4c4198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:39:47.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.349+0000 7ff4cba2b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4103c90 0x7ff4c4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:39:47.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4cba2b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c4104630 0x7ff4c41a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:39:47.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4cba2b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c410e5c0 con 0x7ff4c4103c90 2024-04-25T18:39:47.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4cba2b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c410e440 con 0x7ff4c4104630 2024-04-25T18:39:47.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4cba2b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c410e740 con 0x7ff4c4102a90 2024-04-25T18:39:47.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4c9fc8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c4104630 0x7ff4c41a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:39:47.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4c9fc8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c4104630 0x7ff4c41a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:36888/0 (socket says 172.21.15.161:36888) 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4c9fc8700 1 -- 172.21.15.161:0/1758592584 learned_addr learned my addr 172.21.15.161:0/1758592584 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4c97c7700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4102a90 0x7ff4c4198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4c9fc8700 1 -- 172.21.15.161:0/1758592584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4102a90 msgr2=0x7ff4c4198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4c9fc8700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4102a90 0x7ff4c4198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4c9fc8700 1 -- 172.21.15.161:0/1758592584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4103c90 msgr2=0x7ff4c4199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4c9fc8700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4103c90 0x7ff4c4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4c9fc8700 1 -- 172.21.15.161:0/1758592584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c41a0870 con 0x7ff4c4104630 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4c9fc8700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c4104630 0x7ff4c41a01a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009040 tx=0x7ff4bc004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4ba7fc700 1 -- 172.21.15.161:0/1758592584 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc01f030 con 0x7ff4c4104630 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4ba7fc700 1 -- 172.21.15.161:0/1758592584 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4bc003a00 con 0x7ff4c4104630 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4cba2b700 1 -- 172.21.15.161:0/1758592584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c41a0b00 con 0x7ff4c4104630 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4ba7fc700 1 -- 172.21.15.161:0/1758592584 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0047c0 con 0x7ff4c4104630 2024-04-25T18:39:47.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.350+0000 7ff4cba2b700 1 -- 172.21.15.161:0/1758592584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c41a0fd0 con 0x7ff4c4104630 2024-04-25T18:39:47.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.351+0000 7ff4cba2b700 1 -- 172.21.15.161:0/1758592584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c4069730 con 0x7ff4c4104630 2024-04-25T18:39:47.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.352+0000 7ff4ba7fc700 1 -- 172.21.15.161:0/1758592584 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff4bc003bb0 con 0x7ff4c4104630 2024-04-25T18:39:47.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.353+0000 7ff4ba7fc700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff4b00a3cc0 0x7ff4b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:39:47.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.353+0000 7ff4ba7fc700 1 -- 172.21.15.161:0/1758592584 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7ff4bc0d08a0 con 0x7ff4c4104630 2024-04-25T18:39:47.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.353+0000 7ff4c97c7700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff4b00a3cc0 0x7ff4b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:39:47.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.354+0000 7ff4c97c7700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff4b00a3cc0 0x7ff4b00a6180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4c419a100 tx=0x7ff4c0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:39:47.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.358+0000 7ff4ba7fc700 1 -- 172.21.15.161:0/1758592584 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4bc09e020 con 0x7ff4c4104630 2024-04-25T18:39:47.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:39:47.567+0000 7ff4cba2b700 1 -- 172.21.15.161:0/1758592584 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff4c4062e80 con 0x7ff4b00a3cc0 2024-04-25T18:39:48.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:48.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:39:48.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:39:48.408 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:48.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:48.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:48.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:48.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:48.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:48.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:48.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:48.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:48.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:39:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:39:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:39:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:39:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:39:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:48.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:39:48.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:39:49.287 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:49 smithi161 ceph-mon[32609]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:39:49.287 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:49 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi112 2024-04-25T18:39:49.287 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:49 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T18:39:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:49 smithi171 ceph-mon[33266]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:39:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:49 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi112 2024-04-25T18:39:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:49 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T18:39:49.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:49 smithi112 ceph-mon[27864]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:39:49.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:49 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi112 2024-04-25T18:39:49.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:49 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T18:39:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:50 smithi171 ceph-mon[33266]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:50 smithi171 ceph-mon[33266]: from='client.? 172.21.15.161:0/598721264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2d06e86-6aac-40f4-956e-18fe0b0e3543"}]: dispatch 2024-04-25T18:39:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:50 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2d06e86-6aac-40f4-956e-18fe0b0e3543"}]: dispatch 2024-04-25T18:39:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:50 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2d06e86-6aac-40f4-956e-18fe0b0e3543"}]': finished 2024-04-25T18:39:50.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:50 smithi171 ceph-mon[33266]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T18:39:50.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:39:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:50 smithi161 ceph-mon[32609]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:50.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:50 smithi161 ceph-mon[32609]: from='client.? 172.21.15.161:0/598721264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2d06e86-6aac-40f4-956e-18fe0b0e3543"}]: dispatch 2024-04-25T18:39:50.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:50 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2d06e86-6aac-40f4-956e-18fe0b0e3543"}]: dispatch 2024-04-25T18:39:50.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:50 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2d06e86-6aac-40f4-956e-18fe0b0e3543"}]': finished 2024-04-25T18:39:50.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:50 smithi161 ceph-mon[32609]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T18:39:50.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:39:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:50 smithi112 ceph-mon[27864]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:50 smithi112 ceph-mon[27864]: from='client.? 172.21.15.161:0/598721264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2d06e86-6aac-40f4-956e-18fe0b0e3543"}]: dispatch 2024-04-25T18:39:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:50 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2d06e86-6aac-40f4-956e-18fe0b0e3543"}]: dispatch 2024-04-25T18:39:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:50 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2d06e86-6aac-40f4-956e-18fe0b0e3543"}]': finished 2024-04-25T18:39:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:50 smithi112 ceph-mon[27864]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T18:39:50.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:39:51.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:51 smithi171 ceph-mon[33266]: from='client.? 172.21.15.161:0/3557831465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:39:51.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:51 smithi161 ceph-mon[32609]: from='client.? 172.21.15.161:0/3557831465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:39:51.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:51 smithi112 ceph-mon[27864]: from='client.? 172.21.15.161:0/3557831465' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:39:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:52 smithi161 ceph-mon[32609]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:52 smithi112 ceph-mon[27864]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:52.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:52 smithi171 ceph-mon[33266]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:53.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:53 smithi161 ceph-mon[32609]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:53.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:53 smithi112 ceph-mon[27864]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:53.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:53 smithi171 ceph-mon[33266]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:56.347 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:56 smithi161 ceph-mon[32609]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:56.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:56 smithi171 ceph-mon[33266]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:56.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:56 smithi112 ceph-mon[27864]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:58.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:58 smithi161 ceph-mon[32609]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:58.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:58 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T18:39:58.409 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:58 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:58.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:58 smithi171 ceph-mon[33266]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:58.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:58 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T18:39:58.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:58 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:58.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:58 smithi112 ceph-mon[27864]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:39:58.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:58 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T18:39:58.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:58 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:39:59.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:39:59 smithi112 ceph-mon[27864]: Deploying daemon osd.7 on smithi161 2024-04-25T18:39:59.341 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:39:59 smithi161 ceph-mon[32609]: Deploying daemon osd.7 on smithi161 2024-04-25T18:39:59.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:39:59 smithi171 ceph-mon[33266]: Deploying daemon osd.7 on smithi161 2024-04-25T18:40:00.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:00 smithi112 ceph-mon[27864]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:00.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:00 smithi112 ceph-mon[27864]: overall HEALTH_OK 2024-04-25T18:40:00.302 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:00 smithi161 ceph-mon[32609]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:00.303 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:00 smithi161 ceph-mon[32609]: overall HEALTH_OK 2024-04-25T18:40:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:00 smithi171 ceph-mon[33266]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:00 smithi171 ceph-mon[33266]: overall HEALTH_OK 2024-04-25T18:40:02.343 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:02 smithi161 ceph-mon[32609]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:02 smithi171 ceph-mon[33266]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:02 smithi112 ceph-mon[27864]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:04.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:03 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:04.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:03 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:04.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:03 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:04.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:03 smithi171 ceph-mon[33266]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:04.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:03 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:04.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:03 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:04.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:03 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:04.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:03 smithi161 ceph-mon[32609]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:04.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:03 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:04.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:03 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:04.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:03 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:04.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:03 smithi112 ceph-mon[27864]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:06.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:06 smithi112 ceph-mon[27864]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:06.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:06.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:06.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:06.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:06.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:06.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:06 smithi161 ceph-mon[32609]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:06.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:06.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:06.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:06.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:06.361 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:06 smithi171 ceph-mon[33266]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:06.461 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 7 on host 'smithi161' 2024-04-25T18:40:06.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.459+0000 7ff4ba7fc700 1 -- 172.21.15.161:0/1758592584 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff4c4062e80 con 0x7ff4b00a3cc0 2024-04-25T18:40:06.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.461+0000 7ff4cba2b700 1 -- 172.21.15.161:0/1758592584 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff4b00a3cc0 msgr2=0x7ff4b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:40:06.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.461+0000 7ff4cba2b700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff4b00a3cc0 0x7ff4b00a6180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4c419a100 tx=0x7ff4c0009190 comp rx=0 tx=0).stop 2024-04-25T18:40:06.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.461+0000 7ff4cba2b700 1 -- 172.21.15.161:0/1758592584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c4104630 msgr2=0x7ff4c41a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:40:06.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.461+0000 7ff4cba2b700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c4104630 0x7ff4c41a01a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009040 tx=0x7ff4bc004d50 comp rx=0 tx=0).stop 2024-04-25T18:40:06.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.461+0000 7ff4cba2b700 1 -- 172.21.15.161:0/1758592584 shutdown_connections 2024-04-25T18:40:06.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.461+0000 7ff4cba2b700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff4b00a3cc0 0x7ff4b00a6180 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:06.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.461+0000 7ff4cba2b700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c4104630 0x7ff4c41a01a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:06.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.461+0000 7ff4cba2b700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4102a90 0x7ff4c4198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:06.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.461+0000 7ff4cba2b700 1 --2- 172.21.15.161:0/1758592584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4103c90 0x7ff4c4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:06.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.461+0000 7ff4cba2b700 1 -- 172.21.15.161:0/1758592584 >> 172.21.15.161:0/1758592584 conn(0x7ff4c40fe030 msgr2=0x7ff4c4106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:40:06.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.462+0000 7ff4cba2b700 1 -- 172.21.15.161:0/1758592584 shutdown_connections 2024-04-25T18:40:06.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:40:06.462+0000 7ff4cba2b700 1 -- 172.21.15.161:0/1758592584 wait complete. 2024-04-25T18:40:07.099 DEBUG:teuthology.orchestra.run.smithi161:osd.7> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T18:40:07.101 INFO:tasks.cephadm:Deploying osd.8 on smithi171 with /dev/vg_nvme/lv_2... 2024-04-25T18:40:07.102 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T18:40:07.352 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:40:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:07 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:07 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:07 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:07 smithi171 ceph-mon[33266]: from='osd.7 [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:40:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:07 smithi171 ceph-mon[33266]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:40:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:07 smithi171 ceph-mon[33266]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:07.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:07.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:07.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:07.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:07 smithi161 ceph-mon[32609]: from='osd.7 [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:40:07.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:07 smithi161 ceph-mon[32609]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:40:07.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:07 smithi161 ceph-mon[32609]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:07.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:07 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:07.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:07 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:07.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:07 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:07.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:07 smithi112 ceph-mon[27864]: from='osd.7 [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:40:07.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:07 smithi112 ceph-mon[27864]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:40:07.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:07 smithi112 ceph-mon[27864]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:08.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:08 smithi161 ceph-mon[32609]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T18:40:08.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:08 smithi161 ceph-mon[32609]: from='osd.7 [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:40:08.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:08 smithi161 ceph-mon[32609]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T18:40:08.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:08.582 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:08 smithi161 ceph-mon[32609]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:40:08.582 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:40:08 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:40:08.429+0000 7f97ed282700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T18:40:08.583 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:40:08 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:40:08.440+0000 7f97e9075700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:40:08.689 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:08 smithi171 ceph-mon[33266]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T18:40:08.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:08 smithi171 ceph-mon[33266]: from='osd.7 [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:40:08.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:08 smithi171 ceph-mon[33266]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T18:40:08.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:08.690 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:08 smithi171 ceph-mon[33266]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:40:08.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:08 smithi112 ceph-mon[27864]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T18:40:08.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:08 smithi112 ceph-mon[27864]: from='osd.7 [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:40:08.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:08 smithi112 ceph-mon[27864]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T18:40:08.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:08.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:08 smithi112 ceph-mon[27864]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:40:09.240 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T18:40:09.240 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T18:40:09.240 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-25T18:40:09.241 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-25T18:40:09.241 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113874 s, 921 MB/s 2024-04-25T18:40:09.241 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-25T18:40:09.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:09 smithi161 ceph-mon[32609]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:40:09.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:09 smithi161 ceph-mon[32609]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T18:40:09.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:09.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:09 smithi161 ceph-mon[32609]: osdmap e53: 8 total, 7 up, 8 in 2024-04-25T18:40:09.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:09.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:09 smithi161 ceph-mon[32609]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:09.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:09.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:09 smithi171 ceph-mon[33266]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:40:09.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:09 smithi171 ceph-mon[33266]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T18:40:09.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:09.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:09 smithi171 ceph-mon[33266]: osdmap e53: 8 total, 7 up, 8 in 2024-04-25T18:40:09.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:09.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:09 smithi171 ceph-mon[33266]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:09.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:09.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:09 smithi112 ceph-mon[27864]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:40:09.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:09 smithi112 ceph-mon[27864]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T18:40:09.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:09.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:09 smithi112 ceph-mon[27864]: osdmap e53: 8 total, 7 up, 8 in 2024-04-25T18:40:09.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:09.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:09 smithi112 ceph-mon[27864]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:40:09.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:09.854 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_2 2024-04-25T18:40:10.112 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:40:10.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:40:10.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:40:10.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: osd.7 [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138] boot 2024-04-25T18:40:10.866 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi161 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi161 to 5696M 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:10.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:10 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:10.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:40:10.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:40:10.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: osd.7 [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138] boot 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi161 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi161 to 5696M 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:10.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:10 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:10.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:40:10.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:40:10.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: osd.7 [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138] boot 2024-04-25T18:40:10.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T18:40:10.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:40:10.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi161 2024-04-25T18:40:10.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:10.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:10.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:10.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:10.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:10.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi161 to 5696M 2024-04-25T18:40:10.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:10.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:10.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:10.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:10 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:11.642 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.639+0000 7f9e76a31700 1 -- 172.21.15.171:0/3956786706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e70103550 msgr2=0x7f9e70109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.639+0000 7f9e76a31700 1 --2- 172.21.15.171:0/3956786706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e70103550 0x7f9e70109e00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800b0a0 tx=0x7f9e6800b3b0 comp rx=0 tx=0).stop 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.639+0000 7f9e76a31700 1 -- 172.21.15.171:0/3956786706 shutdown_connections 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.639+0000 7f9e76a31700 1 --2- 172.21.15.171:0/3956786706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e70103550 0x7f9e70109e00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.639+0000 7f9e76a31700 1 --2- 172.21.15.171:0/3956786706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e701019b0 0x7f9e70101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.639+0000 7f9e76a31700 1 --2- 172.21.15.171:0/3956786706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e70102bb0 0x7f9e70103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.639+0000 7f9e76a31700 1 -- 172.21.15.171:0/3956786706 >> 172.21.15.171:0/3956786706 conn(0x7f9e700fcf70 msgr2=0x7f9e700ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.639+0000 7f9e76a31700 1 -- 172.21.15.171:0/3956786706 shutdown_connections 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.639+0000 7f9e76a31700 1 -- 172.21.15.171:0/3956786706 wait complete. 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.640+0000 7f9e76a31700 1 Processor -- start 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.640+0000 7f9e76a31700 1 -- start start 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.640+0000 7f9e76a31700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e701019b0 0x7f9e70105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:40:11.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.640+0000 7f9e76a31700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e70102bb0 0x7f9e70103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.640+0000 7f9e76a31700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e70103550 0x7f9e701044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.640+0000 7f9e76a31700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7010d4e0 con 0x7f9e70102bb0 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.640+0000 7f9e76a31700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7010d360 con 0x7f9e701019b0 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.640+0000 7f9e76a31700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7010d660 con 0x7f9e70103550 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.640+0000 7f9e74fce700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e70103550 0x7f9e701044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.640+0000 7f9e74fce700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e70103550 0x7f9e701044d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:35966/0 (socket says 172.21.15.171:35966) 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.640+0000 7f9e74fce700 1 -- 172.21.15.171:0/3387308097 learned_addr learned my addr 172.21.15.171:0/3387308097 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e6ffff700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e701019b0 0x7f9e70105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e74fce700 1 -- 172.21.15.171:0/3387308097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e701019b0 msgr2=0x7f9e70105940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e74fce700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e701019b0 0x7f9e70105940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e74fce700 1 -- 172.21.15.171:0/3387308097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e70102bb0 msgr2=0x7f9e70103f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e74fce700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e70102bb0 0x7f9e70103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:11.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e74fce700 1 -- 172.21.15.171:0/3387308097 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e70104d80 con 0x7f9e70103550 2024-04-25T18:40:11.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e6ffff700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e701019b0 0x7f9e70105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:40:11.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e74fce700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e70103550 0x7f9e701044d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800b070 tx=0x7f9e6800cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:40:11.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e6d7fa700 1 -- 172.21.15.171:0/3387308097 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68021070 con 0x7f9e70103550 2024-04-25T18:40:11.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e76a31700 1 -- 172.21.15.171:0/3387308097 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e701a2db0 con 0x7f9e70103550 2024-04-25T18:40:11.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e6d7fa700 1 -- 172.21.15.171:0/3387308097 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e68007e20 con 0x7f9e70103550 2024-04-25T18:40:11.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e6d7fa700 1 -- 172.21.15.171:0/3387308097 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68005460 con 0x7f9e70103550 2024-04-25T18:40:11.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.641+0000 7f9e76a31700 1 -- 172.21.15.171:0/3387308097 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e701a3230 con 0x7f9e70103550 2024-04-25T18:40:11.646 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.642+0000 7f9e76a31700 1 -- 172.21.15.171:0/3387308097 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e701a3400 con 0x7f9e70103550 2024-04-25T18:40:11.646 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.643+0000 7f9e6d7fa700 1 -- 172.21.15.171:0/3387308097 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9e68024070 con 0x7f9e70103550 2024-04-25T18:40:11.646 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.644+0000 7f9e6d7fa700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9e580a3c40 0x7f9e580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:40:11.646 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.644+0000 7f9e6d7fa700 1 -- 172.21.15.171:0/3387308097 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5426+0+0 (secure 0 0 0) 0x7f9e6800f020 con 0x7f9e70103550 2024-04-25T18:40:11.646 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.644+0000 7f9e6ffff700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9e580a3c40 0x7f9e580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:40:11.647 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.645+0000 7f9e6ffff700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9e580a3c40 0x7f9e580a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e70102a10 tx=0x7f9e60008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:40:11.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.648+0000 7f9e6d7fa700 1 -- 172.21.15.171:0/3387308097 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e6802b020 con 0x7f9e70103550 2024-04-25T18:40:11.856 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:11.855+0000 7f9e76a31700 1 -- 172.21.15.171:0/3387308097 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9e700ff160 con 0x7f9e580a3c40 2024-04-25T18:40:11.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:11 smithi171 ceph-mon[33266]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T18:40:11.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:11 smithi171 ceph-mon[33266]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:11.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:11 smithi161 ceph-mon[32609]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T18:40:11.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:11 smithi161 ceph-mon[32609]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:11.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:11 smithi112 ceph-mon[27864]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T18:40:11.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:11 smithi112 ceph-mon[27864]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:12.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:12 smithi171 ceph-mon[33266]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T18:40:12.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:12 smithi171 ceph-mon[33266]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:40:12.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:12 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:40:12.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:12 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:40:12.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:12 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:12.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:12 smithi161 ceph-mon[32609]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T18:40:12.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:12 smithi161 ceph-mon[32609]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:40:12.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:12 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:40:12.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:12 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:40:12.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:12 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:12.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:12 smithi112 ceph-mon[27864]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T18:40:12.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:12 smithi112 ceph-mon[27864]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:40:12.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:12 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:40:12.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:12 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:40:12.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:12 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:13.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:13 smithi171 ceph-mon[33266]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T18:40:13.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:13 smithi171 ceph-mon[33266]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:13.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:13 smithi161 ceph-mon[32609]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T18:40:13.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:13 smithi161 ceph-mon[32609]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:13.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:13 smithi112 ceph-mon[27864]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T18:40:13.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:13 smithi112 ceph-mon[27864]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:14.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:14 smithi171 ceph-mon[33266]: osdmap e58: 8 total, 8 up, 8 in 2024-04-25T18:40:14.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:14 smithi161 ceph-mon[32609]: osdmap e58: 8 total, 8 up, 8 in 2024-04-25T18:40:14.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:14 smithi112 ceph-mon[27864]: osdmap e58: 8 total, 8 up, 8 in 2024-04-25T18:40:15.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:15 smithi171 ceph-mon[33266]: from='client.? 172.21.15.171:0/2446461467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f006e45f-2bb9-45d0-9020-be7c33c6a825"}]: dispatch 2024-04-25T18:40:15.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:15 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f006e45f-2bb9-45d0-9020-be7c33c6a825"}]: dispatch 2024-04-25T18:40:15.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:15 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f006e45f-2bb9-45d0-9020-be7c33c6a825"}]': finished 2024-04-25T18:40:15.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:15 smithi171 ceph-mon[33266]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T18:40:15.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:15 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:15.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:15 smithi171 ceph-mon[33266]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:15.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:15 smithi171 ceph-mon[33266]: from='client.? 172.21.15.171:0/3840698523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:40:15.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:15 smithi161 ceph-mon[32609]: from='client.? 172.21.15.171:0/2446461467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f006e45f-2bb9-45d0-9020-be7c33c6a825"}]: dispatch 2024-04-25T18:40:15.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:15 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f006e45f-2bb9-45d0-9020-be7c33c6a825"}]: dispatch 2024-04-25T18:40:15.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:15 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f006e45f-2bb9-45d0-9020-be7c33c6a825"}]': finished 2024-04-25T18:40:15.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:15 smithi161 ceph-mon[32609]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T18:40:15.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:15 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:15.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:15 smithi161 ceph-mon[32609]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:15.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:15 smithi161 ceph-mon[32609]: from='client.? 172.21.15.171:0/3840698523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:40:15.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:15 smithi112 ceph-mon[27864]: from='client.? 172.21.15.171:0/2446461467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f006e45f-2bb9-45d0-9020-be7c33c6a825"}]: dispatch 2024-04-25T18:40:15.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:15 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f006e45f-2bb9-45d0-9020-be7c33c6a825"}]: dispatch 2024-04-25T18:40:15.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:15 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f006e45f-2bb9-45d0-9020-be7c33c6a825"}]': finished 2024-04-25T18:40:15.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:15 smithi112 ceph-mon[27864]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T18:40:15.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:15 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:15.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:15 smithi112 ceph-mon[27864]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:15.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:15 smithi112 ceph-mon[27864]: from='client.? 172.21.15.171:0/3840698523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:40:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:18 smithi171 ceph-mon[33266]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:18 smithi161 ceph-mon[32609]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:18 smithi112 ceph-mon[27864]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:20.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:20 smithi171 ceph-mon[33266]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:20.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:20 smithi161 ceph-mon[32609]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:20.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:20 smithi112 ceph-mon[27864]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:22.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:22 smithi171 ceph-mon[33266]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:22.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T18:40:22.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:22 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:22.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:22 smithi161 ceph-mon[32609]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:22.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T18:40:22.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:22 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:22.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:22 smithi112 ceph-mon[27864]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:22.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T18:40:22.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:22 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:23.225 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:23 smithi171 ceph-mon[33266]: Deploying daemon osd.8 on smithi171 2024-04-25T18:40:23.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:23 smithi161 ceph-mon[32609]: Deploying daemon osd.8 on smithi171 2024-04-25T18:40:23.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:23 smithi112 ceph-mon[27864]: Deploying daemon osd.8 on smithi171 2024-04-25T18:40:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:24 smithi171 ceph-mon[33266]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:24 smithi161 ceph-mon[32609]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:24.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:24 smithi112 ceph-mon[27864]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:26.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:26 smithi171 ceph-mon[33266]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:26.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:26 smithi161 ceph-mon[32609]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:26.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:26 smithi112 ceph-mon[27864]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:28.403 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:28 smithi171 ceph-mon[33266]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:28.403 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:28 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:28.404 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:28 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:28.404 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:28 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:28 smithi161 ceph-mon[32609]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:28 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:28 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:28 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:28 smithi112 ceph-mon[27864]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:28 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:28 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:28 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:30.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:30 smithi112 ceph-mon[27864]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:30.317 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:30 smithi171 ceph-mon[33266]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:30 smithi161 ceph-mon[32609]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:31.041 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.039+0000 7f9e6d7fa700 1 -- 172.21.15.171:0/3387308097 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9e700ff160 con 0x7f9e580a3c40 2024-04-25T18:40:31.041 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 8 on host 'smithi171' 2024-04-25T18:40:31.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 -- 172.21.15.171:0/3387308097 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9e580a3c40 msgr2=0x7f9e580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:40:31.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9e580a3c40 0x7f9e580a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e70102a10 tx=0x7f9e60008040 comp rx=0 tx=0).stop 2024-04-25T18:40:31.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 -- 172.21.15.171:0/3387308097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e70103550 msgr2=0x7f9e701044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:40:31.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e70103550 0x7f9e701044d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800b070 tx=0x7f9e6800cf80 comp rx=0 tx=0).stop 2024-04-25T18:40:31.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 -- 172.21.15.171:0/3387308097 shutdown_connections 2024-04-25T18:40:31.044 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9e580a3c40 0x7f9e580a6100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:31.044 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e70103550 0x7f9e701044d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:31.044 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e701019b0 0x7f9e70105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:31.044 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 --2- 172.21.15.171:0/3387308097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e70102bb0 0x7f9e70103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:31.044 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 -- 172.21.15.171:0/3387308097 >> 172.21.15.171:0/3387308097 conn(0x7f9e700fcf70 msgr2=0x7f9e700fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:40:31.044 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 -- 172.21.15.171:0/3387308097 shutdown_connections 2024-04-25T18:40:31.044 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:40:31.042+0000 7f9e76a31700 1 -- 172.21.15.171:0/3387308097 wait complete. 2024-04-25T18:40:31.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:31.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:31.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:31.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:31 smithi171 ceph-mon[33266]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:31.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:31 smithi171 ceph-mon[33266]: from='osd.8 [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:40:31.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:31 smithi171 ceph-mon[33266]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:40:31.719 DEBUG:teuthology.orchestra.run.smithi171:osd.8> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T18:40:31.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:31.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:31.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:31.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:31 smithi161 ceph-mon[32609]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:31.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:31 smithi161 ceph-mon[32609]: from='osd.8 [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:40:31.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:31 smithi161 ceph-mon[32609]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:40:31.722 INFO:tasks.cephadm:Deploying osd.9 on smithi112 with /dev/vg_nvme/lv_1... 2024-04-25T18:40:31.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T18:40:31.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:31.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:31.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:31.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:40:31.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:31 smithi112 ceph-mon[27864]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:31.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:31 smithi112 ceph-mon[27864]: from='osd.8 [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:40:31.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:31 smithi112 ceph-mon[27864]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:40:32.172 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:40:33.137 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:33 smithi171 ceph-mon[33266]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T18:40:33.137 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:33 smithi171 ceph-mon[33266]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T18:40:33.137 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:33 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:33.137 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:33 smithi171 ceph-mon[33266]: from='osd.8 [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:40:33.137 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:33 smithi171 ceph-mon[33266]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:40:33.138 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:40:33 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:40:33.013+0000 7fcccf5cd700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T18:40:33.138 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:40:33 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:40:33.027+0000 7fcccb3c0700 -1 osd.8 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:40:33.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:33 smithi112 ceph-mon[27864]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T18:40:33.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:33 smithi112 ceph-mon[27864]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T18:40:33.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:33 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:33.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:33 smithi112 ceph-mon[27864]: from='osd.8 [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:40:33.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:33 smithi112 ceph-mon[27864]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:40:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:33 smithi161 ceph-mon[32609]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T18:40:33.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:33 smithi161 ceph-mon[32609]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T18:40:33.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:33 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:33.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:33 smithi161 ceph-mon[32609]: from='osd.8 [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:40:33.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:33 smithi161 ceph-mon[32609]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:40:34.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:34 smithi171 ceph-mon[33266]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:40:34.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:34 smithi171 ceph-mon[33266]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T18:40:34.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:34 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:34.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:34 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:34.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:34 smithi171 ceph-mon[33266]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:34.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:34 smithi171 ceph-mon[33266]: osd.8 [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571] boot 2024-04-25T18:40:34.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:34 smithi171 ceph-mon[33266]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T18:40:34.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:34 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:34.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:34 smithi112 ceph-mon[27864]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:40:34.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:34 smithi112 ceph-mon[27864]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T18:40:34.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:34 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:34.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:34 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:34.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:34 smithi112 ceph-mon[27864]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:34.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:34 smithi112 ceph-mon[27864]: osd.8 [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571] boot 2024-04-25T18:40:34.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:34 smithi112 ceph-mon[27864]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T18:40:34.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:34 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:34 smithi161 ceph-mon[32609]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:40:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:34 smithi161 ceph-mon[32609]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T18:40:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:34 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:34 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:34 smithi161 ceph-mon[32609]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:40:34.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:34 smithi161 ceph-mon[32609]: osd.8 [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571] boot 2024-04-25T18:40:34.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:34 smithi161 ceph-mon[32609]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T18:40:34.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:34 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:40:35.336 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T18:40:35.336 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T18:40:35.336 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T18:40:35.336 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T18:40:35.336 INFO:teuthology.orchestra.run.smithi112.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0362083 s, 290 MB/s 2024-04-25T18:40:35.336 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:35.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:35 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:35.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:40:35.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:40:35.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T18:40:35.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:35.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:35.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:35.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:35.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:35.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:35.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:35.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:35.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:35 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:35.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:40:35.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:40:35.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T18:40:35.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:35.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:35.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:35.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:35.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:40:35.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:35.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:35.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:40:35.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:35 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:40:36.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-04-25T18:40:36.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:36 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi171 2024-04-25T18:40:36.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:36 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-25T18:40:36.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:36 smithi112 ceph-mon[27864]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:36.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:36 smithi112 ceph-mon[27864]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T18:40:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:36 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi171 2024-04-25T18:40:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:36 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-25T18:40:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:36 smithi171 ceph-mon[33266]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:36 smithi171 ceph-mon[33266]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T18:40:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:36 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi171 2024-04-25T18:40:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:36 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-25T18:40:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:36 smithi161 ceph-mon[32609]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:36 smithi161 ceph-mon[32609]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T18:40:36.696 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:40:37.868 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:37 smithi112 ceph-mon[27864]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T18:40:37.868 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:37 smithi112 ceph-mon[27864]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:37.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:37 smithi171 ceph-mon[33266]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T18:40:37.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:37 smithi171 ceph-mon[33266]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:37.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:37 smithi161 ceph-mon[32609]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T18:40:37.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:37 smithi161 ceph-mon[32609]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:38.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:38 smithi112 ceph-mon[27864]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T18:40:38.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:38 smithi171 ceph-mon[33266]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T18:40:38.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:38 smithi161 ceph-mon[32609]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T18:40:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.059+0000 7f7644200700 1 -- 172.21.15.112:0/2498469300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c104690 msgr2=0x7f763c10af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:40:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.059+0000 7f7644200700 1 --2- 172.21.15.112:0/2498469300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c104690 0x7f763c10af40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7634009fc0 tx=0x7f763400b330 comp rx=0 tx=0).stop 2024-04-25T18:40:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.059+0000 7f7644200700 1 -- 172.21.15.112:0/2498469300 shutdown_connections 2024-04-25T18:40:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.059+0000 7f7644200700 1 --2- 172.21.15.112:0/2498469300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c104690 0x7f763c10af40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.059+0000 7f7644200700 1 --2- 172.21.15.112:0/2498469300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c102af0 0x7f763c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.059+0000 7f7644200700 1 --2- 172.21.15.112:0/2498469300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f763c103cf0 0x7f763c104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.059+0000 7f7644200700 1 -- 172.21.15.112:0/2498469300 >> 172.21.15.112:0/2498469300 conn(0x7f763c0fe090 msgr2=0x7f763c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:40:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.059+0000 7f7644200700 1 -- 172.21.15.112:0/2498469300 shutdown_connections 2024-04-25T18:40:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.059+0000 7f7644200700 1 -- 172.21.15.112:0/2498469300 wait complete. 2024-04-25T18:40:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.060+0000 7f7644200700 1 Processor -- start 2024-04-25T18:40:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.060+0000 7f7644200700 1 -- start start 2024-04-25T18:40:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.060+0000 7f7644200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c102af0 0x7f763c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:40:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.060+0000 7f7644200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c103cf0 0x7f763c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:40:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7644200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f763c104690 0x7f763c1a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:40:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7644200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f763c10e5c0 con 0x7f763c102af0 2024-04-25T18:40:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7644200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f763c10e440 con 0x7f763c103cf0 2024-04-25T18:40:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7644200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f763c10e740 con 0x7f763c104690 2024-04-25T18:40:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7641800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c102af0 0x7f763c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:40:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7641800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c102af0 0x7f763c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50134/0 (socket says 172.21.15.112:50134) 2024-04-25T18:40:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7641800700 1 -- 172.21.15.112:0/807575300 learned_addr learned my addr 172.21.15.112:0/807575300 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:40:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7640e00700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c103cf0 0x7f763c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:40:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7642200700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f763c104690 0x7f763c1a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:40:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7641800700 1 -- 172.21.15.112:0/807575300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f763c104690 msgr2=0x7f763c1a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:40:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7641800700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f763c104690 0x7f763c1a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7641800700 1 -- 172.21.15.112:0/807575300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c103cf0 msgr2=0x7f763c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:40:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7641800700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c103cf0 0x7f763c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:40:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7641800700 1 -- 172.21.15.112:0/807575300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763c1a2a00 con 0x7f763c102af0 2024-04-25T18:40:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.061+0000 7f7642200700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f763c104690 0x7f763c1a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:40:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.062+0000 7f7641800700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c102af0 0x7f763c19adb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f763000e970 tx=0x7f763000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:40:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.062+0000 7f763a000700 1 -- 172.21.15.112:0/807575300 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763000cbd0 con 0x7f763c102af0 2024-04-25T18:40:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.062+0000 7f763a000700 1 -- 172.21.15.112:0/807575300 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76300044d0 con 0x7f763c102af0 2024-04-25T18:40:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.062+0000 7f7644200700 1 -- 172.21.15.112:0/807575300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763c1a2cf0 con 0x7f763c102af0 2024-04-25T18:40:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.062+0000 7f763a000700 1 -- 172.21.15.112:0/807575300 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630003de0 con 0x7f763c102af0 2024-04-25T18:40:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.062+0000 7f7644200700 1 -- 172.21.15.112:0/807575300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763c1a31c0 con 0x7f763c102af0 2024-04-25T18:40:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.064+0000 7f7644200700 1 -- 172.21.15.112:0/807575300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f763c069730 con 0x7f763c102af0 2024-04-25T18:40:39.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.065+0000 7f763a000700 1 -- 172.21.15.112:0/807575300 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7630003f80 con 0x7f763c102af0 2024-04-25T18:40:39.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.067+0000 7f763a000700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76240a3cc0 0x7f76240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:40:39.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.067+0000 7f763a000700 1 -- 172.21.15.112:0/807575300 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5829+0+0 (secure 0 0 0) 0x7f76300cfca0 con 0x7f763c102af0 2024-04-25T18:40:39.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.067+0000 7f7640e00700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76240a3cc0 0x7f76240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:40:39.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.068+0000 7f7640e00700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76240a3cc0 0x7f76240a6180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7628005fd0 tx=0x7f7628009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:40:39.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.072+0000 7f763a000700 1 -- 172.21.15.112:0/807575300 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f763009cf90 con 0x7f763c102af0 2024-04-25T18:40:39.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:40:39.309+0000 7f7644200700 1 -- 172.21.15.112:0/807575300 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f763c076390 con 0x7f76240a3cc0 2024-04-25T18:40:39.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:39 smithi112 ceph-mon[27864]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:39.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:40:39.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:40:39.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:39.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:39 smithi171 ceph-mon[33266]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:39.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:40:39.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:40:39.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:39 smithi161 ceph-mon[32609]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:40:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:40:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:40.833 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:40 smithi171 ceph-mon[33266]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:40:40.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:40 smithi161 ceph-mon[32609]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:40:40.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:40 smithi112 ceph-mon[27864]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:40:41.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:41 smithi171 ceph-mon[33266]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:40:41.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:41 smithi112 ceph-mon[27864]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:40:41.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:41 smithi161 ceph-mon[32609]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:40:44.352 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:44 smithi112 ceph-mon[27864]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:40:44.352 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:44 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/910535986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5a96491-4151-4633-b3bc-2c214fe51c43"}]: dispatch 2024-04-25T18:40:44.352 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:44 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/910535986' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5a96491-4151-4633-b3bc-2c214fe51c43"}]': finished 2024-04-25T18:40:44.352 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:44 smithi112 ceph-mon[27864]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T18:40:44.352 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:44 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:40:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:44 smithi171 ceph-mon[33266]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:40:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:44 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/910535986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5a96491-4151-4633-b3bc-2c214fe51c43"}]: dispatch 2024-04-25T18:40:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:44 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/910535986' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5a96491-4151-4633-b3bc-2c214fe51c43"}]': finished 2024-04-25T18:40:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:44 smithi171 ceph-mon[33266]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T18:40:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:44 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:40:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:44 smithi161 ceph-mon[32609]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:40:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:44 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/910535986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5a96491-4151-4633-b3bc-2c214fe51c43"}]: dispatch 2024-04-25T18:40:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:44 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/910535986' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5a96491-4151-4633-b3bc-2c214fe51c43"}]': finished 2024-04-25T18:40:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:44 smithi161 ceph-mon[32609]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T18:40:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:44 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:40:45.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:45 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1783583792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:40:45.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:45 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1783583792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:40:45.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:45 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1783583792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:40:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:46 smithi171 ceph-mon[33266]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:40:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:46 smithi161 ceph-mon[32609]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:40:46.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:46 smithi112 ceph-mon[27864]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:40:48.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:48 smithi112 ceph-mon[27864]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:40:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:48 smithi171 ceph-mon[33266]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:40:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:48 smithi161 ceph-mon[32609]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T18:40:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:50 smithi171 ceph-mon[33266]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:40:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:50 smithi161 ceph-mon[32609]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:40:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:50 smithi112 ceph-mon[27864]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:40:52.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:52 smithi171 ceph-mon[33266]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:52 smithi161 ceph-mon[32609]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:52 smithi112 ceph-mon[27864]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:54 smithi171 ceph-mon[33266]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:54 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T18:40:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:54 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:54 smithi161 ceph-mon[32609]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:54 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T18:40:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:54 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:54 smithi112 ceph-mon[27864]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:54 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T18:40:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:54 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:40:55.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:55 smithi112 ceph-mon[27864]: Deploying daemon osd.9 on smithi112 2024-04-25T18:40:55.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:55 smithi171 ceph-mon[33266]: Deploying daemon osd.9 on smithi112 2024-04-25T18:40:55.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:55 smithi161 ceph-mon[32609]: Deploying daemon osd.9 on smithi112 2024-04-25T18:40:56.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:56 smithi171 ceph-mon[33266]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:56.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:56 smithi161 ceph-mon[32609]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:56.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:56 smithi112 ceph-mon[27864]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:58.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:40:58 smithi171 ceph-mon[33266]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:58.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:40:58 smithi161 ceph-mon[32609]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:40:58.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:40:58 smithi112 ceph-mon[27864]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:00 smithi171 ceph-mon[33266]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:00.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:00 smithi112 ceph-mon[27864]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:00 smithi161 ceph-mon[32609]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:02 smithi171 ceph-mon[33266]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:02 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:02 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:02 smithi161 ceph-mon[32609]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:02 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:02 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:02 smithi112 ceph-mon[27864]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:02 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:02 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:03.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:03 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:03.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:03 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:03.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:03 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:04.433 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:04 smithi112 ceph-mon[27864]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:04.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:04 smithi161 ceph-mon[32609]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:04.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:04 smithi171 ceph-mon[33266]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:06 smithi161 ceph-mon[32609]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:06 smithi161 ceph-mon[32609]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T18:41:06.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:06 smithi112 ceph-mon[27864]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:06.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:06 smithi112 ceph-mon[27864]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T18:41:06.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:06 smithi171 ceph-mon[33266]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:06.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:06 smithi171 ceph-mon[33266]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T18:41:07.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:07 smithi161 ceph-mon[32609]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T18:41:07.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:07 smithi161 ceph-mon[32609]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T18:41:07.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:07 smithi161 ceph-mon[32609]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:41:07.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:07.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:07.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:07 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:07.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:07 smithi161 ceph-mon[32609]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:07.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:07 smithi112 ceph-mon[27864]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T18:41:07.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:07 smithi112 ceph-mon[27864]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T18:41:07.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:07 smithi112 ceph-mon[27864]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:41:07.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:07 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:07.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:07 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:07.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:07 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:07.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:07 smithi112 ceph-mon[27864]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:07.672 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 9 on host 'smithi112' 2024-04-25T18:41:07.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.669+0000 7f763a000700 1 -- 172.21.15.112:0/807575300 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f763c076390 con 0x7f76240a3cc0 2024-04-25T18:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.674+0000 7f7644200700 1 -- 172.21.15.112:0/807575300 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76240a3cc0 msgr2=0x7f76240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.674+0000 7f7644200700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76240a3cc0 0x7f76240a6180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7628005fd0 tx=0x7f7628009310 comp rx=0 tx=0).stop 2024-04-25T18:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.674+0000 7f7644200700 1 -- 172.21.15.112:0/807575300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c102af0 msgr2=0x7f763c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.674+0000 7f7644200700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c102af0 0x7f763c19adb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f763000e970 tx=0x7f763000ed30 comp rx=0 tx=0).stop 2024-04-25T18:41:07.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.674+0000 7f7644200700 1 -- 172.21.15.112:0/807575300 shutdown_connections 2024-04-25T18:41:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.674+0000 7f7644200700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76240a3cc0 0x7f76240a6180 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.674+0000 7f7644200700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f763c104690 0x7f763c1a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.674+0000 7f7644200700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c102af0 0x7f763c19adb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.674+0000 7f7644200700 1 --2- 172.21.15.112:0/807575300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c103cf0 0x7f763c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.674+0000 7f7644200700 1 -- 172.21.15.112:0/807575300 >> 172.21.15.112:0/807575300 conn(0x7f763c0fe090 msgr2=0x7f763c074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:41:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.675+0000 7f7644200700 1 -- 172.21.15.112:0/807575300 shutdown_connections 2024-04-25T18:41:07.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:41:07.675+0000 7f7644200700 1 -- 172.21.15.112:0/807575300 wait complete. 2024-04-25T18:41:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:07 smithi171 ceph-mon[33266]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T18:41:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:07 smithi171 ceph-mon[33266]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T18:41:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:07 smithi171 ceph-mon[33266]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T18:41:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:07 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:07 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:07 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:07 smithi171 ceph-mon[33266]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:08.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:08 smithi112 ceph-mon[27864]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:41:08.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:08 smithi112 ceph-mon[27864]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T18:41:08.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:08.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:08.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:08.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:08.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:08.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:08.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:08.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:08 smithi161 ceph-mon[32609]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:41:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:08 smithi161 ceph-mon[32609]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T18:41:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:08.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:08.520 DEBUG:teuthology.orchestra.run.smithi112:osd.9> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T18:41:08.525 INFO:tasks.cephadm:Deploying osd.10 on smithi161 with /dev/vg_nvme/lv_1... 2024-04-25T18:41:08.526 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T18:41:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:08 smithi171 ceph-mon[33266]: from='osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T18:41:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:08 smithi171 ceph-mon[33266]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T18:41:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:08.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:08.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:08.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:08.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:08.771 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:41:09.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:09 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:41:09.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:09 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:41:09.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:09 smithi161 ceph-mon[32609]: osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528] boot 2024-04-25T18:41:09.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:09 smithi161 ceph-mon[32609]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T18:41:09.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:09 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:09.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:09 smithi161 ceph-mon[32609]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:09.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:09 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:41:09.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:09 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:41:09.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:09 smithi112 ceph-mon[27864]: osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528] boot 2024-04-25T18:41:09.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:09 smithi112 ceph-mon[27864]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T18:41:09.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:09 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:09.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:09 smithi112 ceph-mon[27864]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:09.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:09 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:41:09.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:09 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:41:09.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:09 smithi171 ceph-mon[33266]: osd.9 [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528] boot 2024-04-25T18:41:09.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:09 smithi171 ceph-mon[33266]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T18:41:09.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:09 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:41:09.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:09 smithi171 ceph-mon[33266]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:41:10.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:10 smithi161 ceph-mon[32609]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T18:41:10.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:10 smithi112 ceph-mon[27864]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T18:41:10.583 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:10 smithi171 ceph-mon[33266]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T18:41:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T18:41:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T18:41:10.631 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-04-25T18:41:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-04-25T18:41:10.631 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115281 s, 910 MB/s 2024-04-25T18:41:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-04-25T18:41:11.291 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:11 smithi161 ceph-mon[32609]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:11.292 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-04-25T18:41:11.537 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:41:11.596 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:11 smithi171 ceph-mon[33266]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:11.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:11 smithi112 ceph-mon[27864]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:13.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.099+0000 7fa4b3a40700 1 -- 172.21.15.161:0/3212476963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac103530 msgr2=0x7fa4ac109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:13.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.099+0000 7fa4b3a40700 1 --2- 172.21.15.161:0/3212476963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac103530 0x7fa4ac109de0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400b0d0 tx=0x7fa4a400b3e0 comp rx=0 tx=0).stop 2024-04-25T18:41:13.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.099+0000 7fa4b3a40700 1 -- 172.21.15.161:0/3212476963 shutdown_connections 2024-04-25T18:41:13.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.099+0000 7fa4b3a40700 1 --2- 172.21.15.161:0/3212476963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac103530 0x7fa4ac109de0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:13.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.099+0000 7fa4b3a40700 1 --2- 172.21.15.161:0/3212476963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4ac101990 0x7fa4ac101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:13.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.099+0000 7fa4b3a40700 1 --2- 172.21.15.161:0/3212476963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4ac102b90 0x7fa4ac102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:13.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.099+0000 7fa4b3a40700 1 -- 172.21.15.161:0/3212476963 >> 172.21.15.161:0/3212476963 conn(0x7fa4ac0fcf70 msgr2=0x7fa4ac0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:41:13.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.099+0000 7fa4b3a40700 1 -- 172.21.15.161:0/3212476963 shutdown_connections 2024-04-25T18:41:13.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.099+0000 7fa4b3a40700 1 -- 172.21.15.161:0/3212476963 wait complete. 2024-04-25T18:41:13.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.099+0000 7fa4b3a40700 1 Processor -- start 2024-04-25T18:41:13.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.099+0000 7fa4b3a40700 1 -- start start 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b3a40700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac101990 0x7fa4ac198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b3a40700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4ac102b90 0x7fa4ac199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b3a40700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4ac103530 0x7fa4ac1a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b3a40700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac10d480 con 0x7fa4ac102b90 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b3a40700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac10d300 con 0x7fa4ac101990 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b3a40700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac10d600 con 0x7fa4ac103530 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b17dc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac101990 0x7fa4ac198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b17dc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac101990 0x7fa4ac198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32926/0 (socket says 172.21.15.161:32926) 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b17dc700 1 -- 172.21.15.161:0/821411600 learned_addr learned my addr 172.21.15.161:0/821411600 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b17dc700 1 -- 172.21.15.161:0/821411600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4ac103530 msgr2=0x7fa4ac1a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b17dc700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4ac103530 0x7fa4ac1a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:13.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b17dc700 1 -- 172.21.15.161:0/821411600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4ac102b90 msgr2=0x7fa4ac199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:13.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b17dc700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4ac102b90 0x7fa4ac199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:13.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b17dc700 1 -- 172.21.15.161:0/821411600 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ac1a0820 con 0x7fa4ac101990 2024-04-25T18:41:13.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b17dc700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac101990 0x7fa4ac198bc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00d980 tx=0x7fa49c00dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:41:13.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4a27fc700 1 -- 172.21.15.161:0/821411600 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c017030 con 0x7fa4ac101990 2024-04-25T18:41:13.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b3a40700 1 -- 172.21.15.161:0/821411600 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ac1a0b10 con 0x7fa4ac101990 2024-04-25T18:41:13.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4a27fc700 1 -- 172.21.15.161:0/821411600 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa49c0044d0 con 0x7fa4ac101990 2024-04-25T18:41:13.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4a27fc700 1 -- 172.21.15.161:0/821411600 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c003aa0 con 0x7fa4ac101990 2024-04-25T18:41:13.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.100+0000 7fa4b3a40700 1 -- 172.21.15.161:0/821411600 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ac1a1120 con 0x7fa4ac101990 2024-04-25T18:41:13.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.101+0000 7fa4b3a40700 1 -- 172.21.15.161:0/821411600 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ac069730 con 0x7fa4ac101990 2024-04-25T18:41:13.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.102+0000 7fa4a27fc700 1 -- 172.21.15.161:0/821411600 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa49c01a070 con 0x7fa4ac101990 2024-04-25T18:41:13.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.103+0000 7fa4a27fc700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4980a3c40 0x7fa4980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:41:13.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.103+0000 7fa4a27fc700 1 -- 172.21.15.161:0/821411600 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6261+0+0 (secure 0 0 0) 0x7fa49c0cf5f0 con 0x7fa4ac101990 2024-04-25T18:41:13.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.103+0000 7fa4b0fdb700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4980a3c40 0x7fa4980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:41:13.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.104+0000 7fa4b0fdb700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4980a3c40 0x7fa4980a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80097b0 tx=0x7fa4a8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:41:13.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.108+0000 7fa4a27fc700 1 -- 172.21.15.161:0/821411600 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa49c09c7d0 con 0x7fa4ac101990 2024-04-25T18:41:13.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:13.312+0000 7fa4b3a40700 1 -- 172.21.15.161:0/821411600 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa4ac0fdcf0 con 0x7fa4980a3c40 2024-04-25T18:41:14.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:14.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:41:14.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:41:14.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:14.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:14.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:14.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:14.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:14.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:14.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:14.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:41:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:41:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:14.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:14.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:14.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:14.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:14.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:41:14.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:41:14.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:14.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:14.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:14.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:14.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:14.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:14.476 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:14.476 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:15.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:15 smithi161 ceph-mon[32609]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:41:15.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:15 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi112 2024-04-25T18:41:15.158 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:15 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T18:41:15.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:15 smithi171 ceph-mon[33266]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:41:15.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:15 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi112 2024-04-25T18:41:15.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:15 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T18:41:15.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:15 smithi112 ceph-mon[27864]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:41:15.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:15 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi112 2024-04-25T18:41:15.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:15 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T18:41:16.395 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:16 smithi161 ceph-mon[32609]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:16.395 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:16 smithi161 ceph-mon[32609]: from='client.? 172.21.15.161:0/671085365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244eda3e-25a0-490e-879f-c1e2726c1be5"}]: dispatch 2024-04-25T18:41:16.395 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:16 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244eda3e-25a0-490e-879f-c1e2726c1be5"}]: dispatch 2024-04-25T18:41:16.395 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:16 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "244eda3e-25a0-490e-879f-c1e2726c1be5"}]': finished 2024-04-25T18:41:16.395 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:16 smithi161 ceph-mon[32609]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T18:41:16.395 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:16 smithi171 ceph-mon[33266]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:16 smithi171 ceph-mon[33266]: from='client.? 172.21.15.161:0/671085365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244eda3e-25a0-490e-879f-c1e2726c1be5"}]: dispatch 2024-04-25T18:41:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:16 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244eda3e-25a0-490e-879f-c1e2726c1be5"}]: dispatch 2024-04-25T18:41:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:16 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "244eda3e-25a0-490e-879f-c1e2726c1be5"}]': finished 2024-04-25T18:41:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:16 smithi171 ceph-mon[33266]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T18:41:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:16 smithi112 ceph-mon[27864]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:16 smithi112 ceph-mon[27864]: from='client.? 172.21.15.161:0/671085365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244eda3e-25a0-490e-879f-c1e2726c1be5"}]: dispatch 2024-04-25T18:41:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:16 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244eda3e-25a0-490e-879f-c1e2726c1be5"}]: dispatch 2024-04-25T18:41:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:16 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "244eda3e-25a0-490e-879f-c1e2726c1be5"}]': finished 2024-04-25T18:41:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:16 smithi112 ceph-mon[27864]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T18:41:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:17.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:17 smithi171 ceph-mon[33266]: from='client.? 172.21.15.161:0/4108995996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:41:17.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:17 smithi161 ceph-mon[32609]: from='client.? 172.21.15.161:0/4108995996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:41:17.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:17 smithi112 ceph-mon[27864]: from='client.? 172.21.15.161:0/4108995996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:41:18.392 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:18 smithi161 ceph-mon[32609]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:18 smithi171 ceph-mon[33266]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:18 smithi112 ceph-mon[27864]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:20.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:20 smithi161 ceph-mon[32609]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:20.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:20 smithi171 ceph-mon[33266]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:20.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:20 smithi112 ceph-mon[27864]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:22.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:22 smithi161 ceph-mon[32609]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:22.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:22 smithi171 ceph-mon[33266]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:22.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:22 smithi112 ceph-mon[27864]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:24.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:24 smithi161 ceph-mon[32609]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:24.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:24 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T18:41:24.349 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:24 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:24 smithi171 ceph-mon[33266]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:24 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T18:41:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:24 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:24.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:24 smithi112 ceph-mon[27864]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:24.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:24 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T18:41:24.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:24 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:25.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:25 smithi161 ceph-mon[32609]: Deploying daemon osd.10 on smithi161 2024-04-25T18:41:25.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:25 smithi171 ceph-mon[33266]: Deploying daemon osd.10 on smithi161 2024-04-25T18:41:25.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:25 smithi112 ceph-mon[27864]: Deploying daemon osd.10 on smithi161 2024-04-25T18:41:26.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:26 smithi171 ceph-mon[33266]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:26.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:26 smithi161 ceph-mon[32609]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:26.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:26 smithi112 ceph-mon[27864]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:28.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:28 smithi161 ceph-mon[32609]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:28 smithi171 ceph-mon[33266]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:28 smithi112 ceph-mon[27864]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:29.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:29 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:29.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:29 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:29.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:29 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:29.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:29 smithi112 ceph-mon[27864]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:30.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:29 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:30.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:29 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:30.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:29 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:30.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:29 smithi171 ceph-mon[33266]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:30.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:29 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:30.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:29 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:30.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:29 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:30.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:29 smithi161 ceph-mon[32609]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:32.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:32 smithi161 ceph-mon[32609]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:32.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:32 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:32.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:32 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:32.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:32 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:32.351 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:32 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:32.352 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:32 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:32 smithi171 ceph-mon[33266]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:32 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:32 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:32 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:32 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:32 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:32.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.448+0000 7fa4a27fc700 1 -- 172.21.15.161:0/821411600 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa4ac0fdcf0 con 0x7fa4980a3c40 2024-04-25T18:41:32.450 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 10 on host 'smithi161' 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 -- 172.21.15.161:0/821411600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4980a3c40 msgr2=0x7fa4980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4980a3c40 0x7fa4980a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80097b0 tx=0x7fa4a8006d20 comp rx=0 tx=0).stop 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 -- 172.21.15.161:0/821411600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac101990 msgr2=0x7fa4ac198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac101990 0x7fa4ac198bc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00d980 tx=0x7fa49c00dd40 comp rx=0 tx=0).stop 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 -- 172.21.15.161:0/821411600 shutdown_connections 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4980a3c40 0x7fa4980a6100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4ac103530 0x7fa4ac1a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac101990 0x7fa4ac198bc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 --2- 172.21.15.161:0/821411600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa4ac102b90 0x7fa4ac199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 -- 172.21.15.161:0/821411600 >> 172.21.15.161:0/821411600 conn(0x7fa4ac0fcf70 msgr2=0x7fa4ac0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 -- 172.21.15.161:0/821411600 shutdown_connections 2024-04-25T18:41:32.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-04-25T18:41:32.450+0000 7fa4b3a40700 1 -- 172.21.15.161:0/821411600 wait complete. 2024-04-25T18:41:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:32 smithi112 ceph-mon[27864]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:32 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:32 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:32.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:32 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:32.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:32 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:32.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:32 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:33.070 DEBUG:teuthology.orchestra.run.smithi161:osd.10> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T18:41:33.072 INFO:tasks.cephadm:Deploying osd.11 on smithi171 with /dev/vg_nvme/lv_1... 2024-04-25T18:41:33.072 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T18:41:33.320 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:41:33.680 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:33 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:33.680 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:33 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:33.680 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:33 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:33.680 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:33 smithi171 ceph-mon[33266]: from='osd.10 [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:41:33.680 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:33 smithi171 ceph-mon[33266]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:41:33.680 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:33 smithi171 ceph-mon[33266]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:33.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:33 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:33.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:33 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:33.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:33 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:33.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:33 smithi161 ceph-mon[32609]: from='osd.10 [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:41:33.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:33 smithi161 ceph-mon[32609]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:41:33.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:33 smithi161 ceph-mon[32609]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:33.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:33 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:33.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:33 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:33.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:33 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:33.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:33 smithi112 ceph-mon[27864]: from='osd.10 [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:41:33.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:33 smithi112 ceph-mon[27864]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:41:33.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:33 smithi112 ceph-mon[27864]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:34.669 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:34 smithi171 ceph-mon[33266]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T18:41:34.670 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:34 smithi171 ceph-mon[33266]: from='osd.10 [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:41:34.670 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:34 smithi171 ceph-mon[33266]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T18:41:34.670 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:34 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:34.670 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:34 smithi171 ceph-mon[33266]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:41:34.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:34 smithi161 ceph-mon[32609]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T18:41:34.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:34 smithi161 ceph-mon[32609]: from='osd.10 [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:41:34.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:34 smithi161 ceph-mon[32609]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T18:41:34.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:34 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:34.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:34 smithi161 ceph-mon[32609]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:41:34.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:34 smithi112 ceph-mon[27864]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T18:41:34.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:34 smithi112 ceph-mon[27864]: from='osd.10 [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:41:34.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:34 smithi112 ceph-mon[27864]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T18:41:34.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:34 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:34.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:34 smithi112 ceph-mon[27864]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-04-25T18:41:35.220 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T18:41:35.220 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T18:41:35.220 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-25T18:41:35.220 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-25T18:41:35.220 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135956 s, 771 MB/s 2024-04-25T18:41:35.220 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-25T18:41:35.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:35 smithi171 ceph-mon[33266]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:41:35.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:35 smithi171 ceph-mon[33266]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T18:41:35.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:35 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:35.521 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:35 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:35.521 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:35 smithi171 ceph-mon[33266]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:35.521 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:35 smithi171 ceph-mon[33266]: from='osd.10 ' entity='osd.10' 2024-04-25T18:41:35.692 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:35 smithi161 ceph-mon[32609]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:41:35.692 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:35 smithi161 ceph-mon[32609]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T18:41:35.693 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:35 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:35.693 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:35 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:35.693 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:35 smithi161 ceph-mon[32609]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:35.693 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:35 smithi161 ceph-mon[32609]: from='osd.10 ' entity='osd.10' 2024-04-25T18:41:35.693 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:41:35 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:41:35.349+0000 7f2d8a0e8700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T18:41:35.693 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:41:35 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:41:35.501+0000 7f2d81ed3700 -1 osd.10 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:41:35.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:35 smithi112 ceph-mon[27864]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-04-25T18:41:35.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:35 smithi112 ceph-mon[27864]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T18:41:35.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:35 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:35.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:35 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:35.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:35 smithi112 ceph-mon[27864]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:35.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:35 smithi112 ceph-mon[27864]: from='osd.10 ' entity='osd.10' 2024-04-25T18:41:35.911 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_1 2024-04-25T18:41:36.153 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:41:36.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:41:36.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:41:36.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:36.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: osdmap e75: 11 total, 10 up, 11 in 2024-04-25T18:41:36.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:36.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi161 2024-04-25T18:41:36.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:36.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:36.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi161 to 4272M 2024-04-25T18:41:36.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:36.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:36.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:36.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:36.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:41:36.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:41:36.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:36.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: osdmap e75: 11 total, 10 up, 11 in 2024-04-25T18:41:36.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi161 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi161 to 4272M 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: osdmap e75: 11 total, 10 up, 11 in 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi161 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:41:36.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi161 to 4272M 2024-04-25T18:41:36.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:36.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:36.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:36.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:37.572 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:37.572 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:37 smithi171 ceph-mon[33266]: osd.10 [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765] boot 2024-04-25T18:41:37.573 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:37 smithi171 ceph-mon[33266]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T18:41:37.573 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:37.573 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:37 smithi171 ceph-mon[33266]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:37.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:37.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:37 smithi161 ceph-mon[32609]: osd.10 [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765] boot 2024-04-25T18:41:37.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:37 smithi161 ceph-mon[32609]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T18:41:37.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:37.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:37 smithi161 ceph-mon[32609]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:37.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:37.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:37 smithi112 ceph-mon[27864]: osd.10 [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765] boot 2024-04-25T18:41:37.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:37 smithi112 ceph-mon[27864]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T18:41:37.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:41:37.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:37 smithi112 ceph-mon[27864]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:41:37.764 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.761+0000 7f04f760e700 1 -- 172.21.15.171:0/3756904258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04f0102aa0 msgr2=0x7f04f0102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:37.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.761+0000 7f04f760e700 1 --2- 172.21.15.171:0/3756904258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04f0102aa0 0x7f04f0102ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009990 tx=0x7f04ec009ca0 comp rx=0 tx=0).stop 2024-04-25T18:41:37.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 -- 172.21.15.171:0/3756904258 shutdown_connections 2024-04-25T18:41:37.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 --2- 172.21.15.171:0/3756904258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04f0104640 0x7f04f010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:37.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 --2- 172.21.15.171:0/3756904258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04f0102aa0 0x7f04f0102ec0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:37.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 --2- 172.21.15.171:0/3756904258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f0103ca0 0x7f04f0104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:37.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 -- 172.21.15.171:0/3756904258 >> 172.21.15.171:0/3756904258 conn(0x7f04f00fe040 msgr2=0x7f04f0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:41:37.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 -- 172.21.15.171:0/3756904258 shutdown_connections 2024-04-25T18:41:37.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 -- 172.21.15.171:0/3756904258 wait complete. 2024-04-25T18:41:37.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 Processor -- start 2024-04-25T18:41:37.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 -- start start 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04f0102aa0 0x7f04f0198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f0103ca0 0x7f04f0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04f0104640 0x7f04f01a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04f010e660 con 0x7f04f0103ca0 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f04f010e4e0 con 0x7f04f0104640 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f760e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f04f010e7e0 con 0x7f04f0102aa0 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.762+0000 7f04f53aa700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04f0102aa0 0x7f04f0198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04f53aa700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04f0102aa0 0x7f04f0198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:39170/0 (socket says 172.21.15.171:39170) 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04f53aa700 1 -- 172.21.15.171:0/3068760735 learned_addr learned my addr 172.21.15.171:0/3068760735 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04f53aa700 1 -- 172.21.15.171:0/3068760735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04f0104640 msgr2=0x7f04f01a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04f53aa700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04f0104640 0x7f04f01a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:37.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04f53aa700 1 -- 172.21.15.171:0/3068760735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f0103ca0 msgr2=0x7f04f0199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:37.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04f53aa700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f0103ca0 0x7f04f0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:37.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04f53aa700 1 -- 172.21.15.171:0/3068760735 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f01a0970 con 0x7f04f0102aa0 2024-04-25T18:41:37.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04f53aa700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04f0102aa0 0x7f04f0198d20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04ec005f50 tx=0x7f04ec0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:41:37.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04e67fc700 1 -- 172.21.15.171:0/3068760735 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec027070 con 0x7f04f0102aa0 2024-04-25T18:41:37.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04e67fc700 1 -- 172.21.15.171:0/3068760735 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ec022470 con 0x7f04f0102aa0 2024-04-25T18:41:37.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04f760e700 1 -- 172.21.15.171:0/3068760735 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f01a0c00 con 0x7f04f0102aa0 2024-04-25T18:41:37.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04e67fc700 1 -- 172.21.15.171:0/3068760735 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec00f6c0 con 0x7f04f0102aa0 2024-04-25T18:41:37.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.763+0000 7f04f760e700 1 -- 172.21.15.171:0/3068760735 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f01a10d0 con 0x7f04f0102aa0 2024-04-25T18:41:37.768 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.764+0000 7f04f760e700 1 -- 172.21.15.171:0/3068760735 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f002df80 con 0x7f04f0102aa0 2024-04-25T18:41:37.768 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.765+0000 7f04e67fc700 1 -- 172.21.15.171:0/3068760735 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f04ec02a070 con 0x7f04f0102aa0 2024-04-25T18:41:37.768 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.766+0000 7f04e67fc700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04dc0a3c40 0x7f04dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:41:37.768 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.766+0000 7f04e67fc700 1 -- 172.21.15.171:0/3068760735 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6723+0+0 (secure 0 0 0) 0x7f04ec0d0d40 con 0x7f04f0102aa0 2024-04-25T18:41:37.768 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.766+0000 7f04f4ba9700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04dc0a3c40 0x7f04dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:41:37.769 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.767+0000 7f04f4ba9700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04dc0a3c40 0x7f04dc0a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f04e00097b0 tx=0x7f04e0006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:41:37.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.771+0000 7f04e67fc700 1 -- 172.21.15.171:0/3068760735 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04ec013070 con 0x7f04f0102aa0 2024-04-25T18:41:37.979 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:37.978+0000 7f04f760e700 1 -- 172.21.15.171:0/3068760735 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f04f0062e80 con 0x7f04dc0a3c40 2024-04-25T18:41:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:38 smithi161 ceph-mon[32609]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T18:41:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:38 smithi161 ceph-mon[32609]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:41:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:38 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:41:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:38 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:41:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:38 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:38.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:38 smithi112 ceph-mon[27864]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T18:41:38.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:38 smithi112 ceph-mon[27864]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:41:38.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:38 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:41:38.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:38 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:41:38.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:38 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:38.763 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:38 smithi171 ceph-mon[33266]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T18:41:38.763 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:38 smithi171 ceph-mon[33266]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:41:38.763 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:38 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:41:38.763 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:38 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:41:38.763 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:38 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:39.558 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:39 smithi171 ceph-mon[33266]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T18:41:39.558 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:39 smithi171 ceph-mon[33266]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:39 smithi161 ceph-mon[32609]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T18:41:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:39 smithi161 ceph-mon[32609]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:39.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:39 smithi112 ceph-mon[27864]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T18:41:39.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:39 smithi112 ceph-mon[27864]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:40.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:40 smithi112 ceph-mon[27864]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T18:41:40.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:40 smithi171 ceph-mon[33266]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T18:41:40.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:40 smithi161 ceph-mon[32609]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T18:41:41.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:41 smithi171 ceph-mon[33266]: osdmap e80: 11 total, 11 up, 11 in 2024-04-25T18:41:41.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:41 smithi171 ceph-mon[33266]: from='client.? 172.21.15.171:0/3992789747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356e5c54-bbc3-45ee-bda2-85164ce3cb92"}]: dispatch 2024-04-25T18:41:41.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:41 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356e5c54-bbc3-45ee-bda2-85164ce3cb92"}]: dispatch 2024-04-25T18:41:41.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:41 smithi171 ceph-mon[33266]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356e5c54-bbc3-45ee-bda2-85164ce3cb92"}]': finished 2024-04-25T18:41:41.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:41 smithi171 ceph-mon[33266]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T18:41:41.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:41 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:41:41.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:41 smithi171 ceph-mon[33266]: pgmap v274: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 159 KiB/s, 0 objects/s recovering 2024-04-25T18:41:41.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:41 smithi171 ceph-mon[33266]: from='client.? 172.21.15.171:0/3288589045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:41:41.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:41 smithi161 ceph-mon[32609]: osdmap e80: 11 total, 11 up, 11 in 2024-04-25T18:41:41.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:41 smithi161 ceph-mon[32609]: from='client.? 172.21.15.171:0/3992789747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356e5c54-bbc3-45ee-bda2-85164ce3cb92"}]: dispatch 2024-04-25T18:41:41.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:41 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356e5c54-bbc3-45ee-bda2-85164ce3cb92"}]: dispatch 2024-04-25T18:41:41.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:41 smithi161 ceph-mon[32609]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356e5c54-bbc3-45ee-bda2-85164ce3cb92"}]': finished 2024-04-25T18:41:41.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:41 smithi161 ceph-mon[32609]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T18:41:41.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:41 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:41:41.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:41 smithi161 ceph-mon[32609]: pgmap v274: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 159 KiB/s, 0 objects/s recovering 2024-04-25T18:41:41.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:41 smithi161 ceph-mon[32609]: from='client.? 172.21.15.171:0/3288589045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:41:41.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:41 smithi112 ceph-mon[27864]: osdmap e80: 11 total, 11 up, 11 in 2024-04-25T18:41:41.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:41 smithi112 ceph-mon[27864]: from='client.? 172.21.15.171:0/3992789747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356e5c54-bbc3-45ee-bda2-85164ce3cb92"}]: dispatch 2024-04-25T18:41:41.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:41 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "356e5c54-bbc3-45ee-bda2-85164ce3cb92"}]: dispatch 2024-04-25T18:41:41.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:41 smithi112 ceph-mon[27864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "356e5c54-bbc3-45ee-bda2-85164ce3cb92"}]': finished 2024-04-25T18:41:41.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:41 smithi112 ceph-mon[27864]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T18:41:41.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:41 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:41:41.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:41 smithi112 ceph-mon[27864]: pgmap v274: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 159 KiB/s, 0 objects/s recovering 2024-04-25T18:41:41.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:41 smithi112 ceph-mon[27864]: from='client.? 172.21.15.171:0/3288589045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:41:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:44 smithi171 ceph-mon[33266]: pgmap v275: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T18:41:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:44 smithi161 ceph-mon[32609]: pgmap v275: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T18:41:44.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:44 smithi112 ceph-mon[27864]: pgmap v275: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-25T18:41:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:46 smithi171 ceph-mon[33266]: pgmap v276: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:41:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:46 smithi161 ceph-mon[32609]: pgmap v276: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:41:46.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:46 smithi112 ceph-mon[27864]: pgmap v276: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:41:48.155 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:48 smithi171 ceph-mon[33266]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:41:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:48 smithi161 ceph-mon[32609]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:41:48.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:48 smithi112 ceph-mon[27864]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:41:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:49 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T18:41:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:49 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:49 smithi171 ceph-mon[33266]: Deploying daemon osd.11 on smithi171 2024-04-25T18:41:49.469 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:49 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T18:41:49.469 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:49 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:49.469 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:49 smithi161 ceph-mon[32609]: Deploying daemon osd.11 on smithi171 2024-04-25T18:41:49.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:49 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T18:41:49.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:49 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:49.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:49 smithi112 ceph-mon[27864]: Deploying daemon osd.11 on smithi171 2024-04-25T18:41:50.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:50 smithi171 ceph-mon[33266]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:50 smithi161 ceph-mon[32609]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:50 smithi112 ceph-mon[27864]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:52.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:52 smithi171 ceph-mon[33266]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:52 smithi161 ceph-mon[32609]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:52 smithi112 ceph-mon[27864]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:54 smithi171 ceph-mon[33266]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:54 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:54 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:54 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:54 smithi161 ceph-mon[32609]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:54 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:54.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:54 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:54.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:54 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:54 smithi112 ceph-mon[27864]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:54 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:54 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:54 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:56.344 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:56 smithi171 ceph-mon[33266]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:56.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:56 smithi161 ceph-mon[32609]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:56.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:56 smithi112 ceph-mon[27864]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:57.768 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 11 on host 'smithi171' 2024-04-25T18:41:57.769 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.767+0000 7f04e67fc700 1 -- 172.21.15.171:0/3068760735 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f04f0062e80 con 0x7f04dc0a3c40 2024-04-25T18:41:57.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.769+0000 7f04f760e700 1 -- 172.21.15.171:0/3068760735 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04dc0a3c40 msgr2=0x7f04dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:57.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.769+0000 7f04f760e700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04dc0a3c40 0x7f04dc0a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f04e00097b0 tx=0x7f04e0006d20 comp rx=0 tx=0).stop 2024-04-25T18:41:57.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.769+0000 7f04f760e700 1 -- 172.21.15.171:0/3068760735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04f0102aa0 msgr2=0x7f04f0198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:41:57.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.769+0000 7f04f760e700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04f0102aa0 0x7f04f0198d20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04ec005f50 tx=0x7f04ec0047b0 comp rx=0 tx=0).stop 2024-04-25T18:41:57.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.769+0000 7f04f760e700 1 -- 172.21.15.171:0/3068760735 shutdown_connections 2024-04-25T18:41:57.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.769+0000 7f04f760e700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f04dc0a3c40 0x7f04dc0a6100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:57.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.769+0000 7f04f760e700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04f0104640 0x7f04f01a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:57.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.769+0000 7f04f760e700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04f0102aa0 0x7f04f0198d20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:57.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.769+0000 7f04f760e700 1 --2- 172.21.15.171:0/3068760735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f0103ca0 0x7f04f0199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:41:57.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.769+0000 7f04f760e700 1 -- 172.21.15.171:0/3068760735 >> 172.21.15.171:0/3068760735 conn(0x7f04f00fe040 msgr2=0x7f04f0107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:41:57.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.769+0000 7f04f760e700 1 -- 172.21.15.171:0/3068760735 shutdown_connections 2024-04-25T18:41:57.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:41:57.770+0000 7f04f760e700 1 -- 172.21.15.171:0/3068760735 wait complete. 2024-04-25T18:41:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:58.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:58.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:57 smithi171 ceph-mon[33266]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:58.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:57 smithi171 ceph-mon[33266]: from='osd.11 [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:41:58.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:57 smithi171 ceph-mon[33266]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:41:58.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:58.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:58.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:58.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:57 smithi161 ceph-mon[32609]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:58.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:57 smithi161 ceph-mon[32609]: from='osd.11 [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:41:58.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:57 smithi161 ceph-mon[32609]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:41:58.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:58.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:41:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:41:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:57 smithi112 ceph-mon[27864]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:41:58.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:57 smithi112 ceph-mon[27864]: from='osd.11 [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:41:58.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:57 smithi112 ceph-mon[27864]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:41:58.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:41:58.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:41:58.417 DEBUG:teuthology.orchestra.run.smithi171:osd.11> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T18:41:58.419 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T18:41:58.420 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T18:41:58.875 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:41:59.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:58 smithi171 ceph-mon[33266]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T18:41:59.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:58 smithi171 ceph-mon[33266]: from='osd.11 [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:41:59.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:58 smithi171 ceph-mon[33266]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T18:41:59.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:58 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:41:59.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:58 smithi171 ceph-mon[33266]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:41:59.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:58 smithi112 ceph-mon[27864]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T18:41:59.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:58 smithi112 ceph-mon[27864]: from='osd.11 [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:41:59.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:58 smithi112 ceph-mon[27864]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T18:41:59.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:58 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:41:59.203 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:58 smithi112 ceph-mon[27864]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:41:59.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:58 smithi161 ceph-mon[32609]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T18:41:59.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:58 smithi161 ceph-mon[32609]: from='osd.11 [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:41:59.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:58 smithi161 ceph-mon[32609]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T18:41:59.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:58 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:41:59.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:58 smithi161 ceph-mon[32609]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T18:42:00.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:59 smithi171 ceph-mon[33266]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:42:00.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:59 smithi171 ceph-mon[33266]: osdmap e83: 12 total, 11 up, 12 in 2024-04-25T18:42:00.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:59 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:00.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:59 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:00.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:59 smithi171 ceph-mon[33266]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:42:00.170 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:41:59 smithi171 ceph-mon[33266]: from='osd.11 ' entity='osd.11' 2024-04-25T18:42:00.171 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:41:59 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:41:59.867+0000 7fd9f08e4700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T18:42:00.171 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:42:00 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:42:00.044+0000 7fd9e7ece700 -1 osd.11 83 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:42:00.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:59 smithi161 ceph-mon[32609]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:42:00.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:59 smithi161 ceph-mon[32609]: osdmap e83: 12 total, 11 up, 12 in 2024-04-25T18:42:00.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:59 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:00.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:59 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:00.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:59 smithi161 ceph-mon[32609]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:42:00.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:41:59 smithi161 ceph-mon[32609]: from='osd.11 ' entity='osd.11' 2024-04-25T18:42:00.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:59 smithi112 ceph-mon[27864]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T18:42:00.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:59 smithi112 ceph-mon[27864]: osdmap e83: 12 total, 11 up, 12 in 2024-04-25T18:42:00.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:59 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:00.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:59 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:00.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:59 smithi112 ceph-mon[27864]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:42:00.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:41:59 smithi112 ceph-mon[27864]: from='osd.11 ' entity='osd.11' 2024-04-25T18:42:01.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:00 smithi171 ceph-mon[33266]: purged_snaps scrub starts 2024-04-25T18:42:01.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:00 smithi171 ceph-mon[33266]: purged_snaps scrub ok 2024-04-25T18:42:01.090 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:00 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:01.091 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:00 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:01.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:00 smithi112 ceph-mon[27864]: purged_snaps scrub starts 2024-04-25T18:42:01.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:00 smithi112 ceph-mon[27864]: purged_snaps scrub ok 2024-04-25T18:42:01.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:00 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:01.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:00 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:01.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.252+0000 7ff7cf800700 1 -- 172.21.15.112:0/3181485859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8106880 msgr2=0x7ff7c810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:01.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.252+0000 7ff7cf800700 1 --2- 172.21.15.112:0/3181485859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8106880 0x7ff7c810d130 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009f90 tx=0x7ff7c000b330 comp rx=0 tx=0).stop 2024-04-25T18:42:01.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.252+0000 7ff7cf800700 1 -- 172.21.15.112:0/3181485859 shutdown_connections 2024-04-25T18:42:01.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.252+0000 7ff7cf800700 1 --2- 172.21.15.112:0/3181485859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8106880 0x7ff7c810d130 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:01.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.252+0000 7ff7cf800700 1 --2- 172.21.15.112:0/3181485859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7c8104ce0 0x7ff7c8105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:01.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.252+0000 7ff7cf800700 1 --2- 172.21.15.112:0/3181485859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c8105ee0 0x7ff7c8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:01.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.252+0000 7ff7cf800700 1 -- 172.21.15.112:0/3181485859 >> 172.21.15.112:0/3181485859 conn(0x7ff7c81002a0 msgr2=0x7ff7c81026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:01.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.252+0000 7ff7cf800700 1 -- 172.21.15.112:0/3181485859 shutdown_connections 2024-04-25T18:42:01.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.252+0000 7ff7cf800700 1 -- 172.21.15.112:0/3181485859 wait complete. 2024-04-25T18:42:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.253+0000 7ff7cf800700 1 Processor -- start 2024-04-25T18:42:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7cf800700 1 -- start start 2024-04-25T18:42:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7cf800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7c8104ce0 0x7ff7c819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7cf800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c8105ee0 0x7ff7c819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7cf800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8106880 0x7ff7c81a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7cf800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81107d0 con 0x7ff7c8105ee0 2024-04-25T18:42:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7cf800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8110650 con 0x7ff7c8104ce0 2024-04-25T18:42:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7cf800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8110950 con 0x7ff7c8106880 2024-04-25T18:42:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7cd800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8106880 0x7ff7c81a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7c7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c8105ee0 0x7ff7c819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:01.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7cd800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8106880 0x7ff7c81a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:33114/0 (socket says 172.21.15.112:33114) 2024-04-25T18:42:01.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7cd800700 1 -- 172.21.15.112:0/508907478 learned_addr learned my addr 172.21.15.112:0/508907478 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:42:01.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.254+0000 7ff7cce00700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7c8104ce0 0x7ff7c819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.255+0000 7ff7cd800700 1 -- 172.21.15.112:0/508907478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7c8104ce0 msgr2=0x7ff7c819adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.255+0000 7ff7cd800700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7c8104ce0 0x7ff7c819adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.255+0000 7ff7cd800700 1 -- 172.21.15.112:0/508907478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c8105ee0 msgr2=0x7ff7c819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.255+0000 7ff7cd800700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c8105ee0 0x7ff7c819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.255+0000 7ff7cd800700 1 -- 172.21.15.112:0/508907478 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c81a2a10 con 0x7ff7c8106880 2024-04-25T18:42:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.255+0000 7ff7cd800700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8106880 0x7ff7c81a2370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00078e0 tx=0x7ff7c0007910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.255+0000 7ff7c5600700 1 -- 172.21.15.112:0/508907478 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c001d670 con 0x7ff7c8106880 2024-04-25T18:42:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.255+0000 7ff7c5600700 1 -- 172.21.15.112:0/508907478 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7c0022070 con 0x7ff7c8106880 2024-04-25T18:42:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.255+0000 7ff7cf800700 1 -- 172.21.15.112:0/508907478 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c81a2ca0 con 0x7ff7c8106880 2024-04-25T18:42:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.255+0000 7ff7c5600700 1 -- 172.21.15.112:0/508907478 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c001bbe0 con 0x7ff7c8106880 2024-04-25T18:42:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.255+0000 7ff7cf800700 1 -- 172.21.15.112:0/508907478 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c81a30f0 con 0x7ff7c8106880 2024-04-25T18:42:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.256+0000 7ff7cf800700 1 -- 172.21.15.112:0/508907478 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c802df80 con 0x7ff7c8106880 2024-04-25T18:42:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.258+0000 7ff7c5600700 1 -- 172.21.15.112:0/508907478 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff7c001d810 con 0x7ff7c8106880 2024-04-25T18:42:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.260+0000 7ff7c5600700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff7b00a3cc0 0x7ff7b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:01.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.260+0000 7ff7c5600700 1 -- 172.21.15.112:0/508907478 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff7c00d12a0 con 0x7ff7c8106880 2024-04-25T18:42:01.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.260+0000 7ff7cce00700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff7b00a3cc0 0x7ff7b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:01.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.261+0000 7ff7cce00700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff7b00a3cc0 0x7ff7b00a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8105d40 tx=0x7ff7bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.262+0000 7ff7c5600700 1 -- 172.21.15.112:0/508907478 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7c009e110 con 0x7ff7c8106880 2024-04-25T18:42:01.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:00 smithi161 ceph-mon[32609]: purged_snaps scrub starts 2024-04-25T18:42:01.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:00 smithi161 ceph-mon[32609]: purged_snaps scrub ok 2024-04-25T18:42:01.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:00 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:01.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:00 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:01.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.486+0000 7ff7cf800700 1 -- 172.21.15.112:0/508907478 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff7c80636f0 con 0x7ff7c8106880 2024-04-25T18:42:01.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.487+0000 7ff7c5600700 1 -- 172.21.15.112:0/508907478 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v84) ==== 74+0+133 (secure 0 0 0) 0x7ff7c009da10 con 0x7ff7c8106880 2024-04-25T18:42:01.488 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:42:01.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 -- 172.21.15.112:0/508907478 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff7b00a3cc0 msgr2=0x7ff7b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:01.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff7b00a3cc0 0x7ff7b00a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8105d40 tx=0x7ff7bc009260 comp rx=0 tx=0).stop 2024-04-25T18:42:01.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 -- 172.21.15.112:0/508907478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8106880 msgr2=0x7ff7c81a2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:01.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8106880 0x7ff7c81a2370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00078e0 tx=0x7ff7c0007910 comp rx=0 tx=0).stop 2024-04-25T18:42:01.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 -- 172.21.15.112:0/508907478 shutdown_connections 2024-04-25T18:42:01.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff7b00a3cc0 0x7ff7b00a6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:01.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8106880 0x7ff7c81a2370 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:01.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7c8104ce0 0x7ff7c819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:01.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 --2- 172.21.15.112:0/508907478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7c8105ee0 0x7ff7c819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:01.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 -- 172.21.15.112:0/508907478 >> 172.21.15.112:0/508907478 conn(0x7ff7c81002a0 msgr2=0x7ff7c81091e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:01.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 -- 172.21.15.112:0/508907478 shutdown_connections 2024-04-25T18:42:01.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:01.491+0000 7ff7cf800700 1 -- 172.21.15.112:0/508907478 wait complete. 2024-04-25T18:42:02.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: osd.11 [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458] boot 2024-04-25T18:42:02.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/508907478' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:42:02.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.366 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":84,"num_osds":12,"num_up_osds":12,"osd_up_since":1714070520,"num_in_osds":12,"osd_in_since":1714070500,"num_remapped_pgs":0} 2024-04-25T18:42:02.367 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:42:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: osd.11 [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458] boot 2024-04-25T18:42:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T18:42:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:42:02.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/508907478' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T18:42:02.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:02.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:42:02.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: osd.11 [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458] boot 2024-04-25T18:42:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T18:42:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/508907478' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:42:02.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:42:03.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:02 smithi112 ceph-mon[27864]: Detected new or changed devices on smithi171 2024-04-25T18:42:03.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:02 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-25T18:42:03.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:02 smithi112 ceph-mon[27864]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T18:42:03.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:02 smithi171 ceph-mon[33266]: Detected new or changed devices on smithi171 2024-04-25T18:42:03.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:02 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-25T18:42:03.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:02 smithi171 ceph-mon[33266]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T18:42:03.474 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:02 smithi161 ceph-mon[32609]: Detected new or changed devices on smithi171 2024-04-25T18:42:03.474 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:02 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-25T18:42:03.474 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:02 smithi161 ceph-mon[32609]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T18:42:04.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:03 smithi112 ceph-mon[27864]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:04.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:04 smithi171 ceph-mon[33266]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:04.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:04 smithi161 ceph-mon[32609]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.132+0000 7f85df800700 1 -- 172.21.15.112:0/2240394902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d8103450 msgr2=0x7f85d810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.132+0000 7f85df800700 1 --2- 172.21.15.112:0/2240394902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d8103450 0x7f85d810f8b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009f90 tx=0x7f85d000b330 comp rx=0 tx=0).stop 2024-04-25T18:42:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.132+0000 7f85df800700 1 -- 172.21.15.112:0/2240394902 shutdown_connections 2024-04-25T18:42:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.132+0000 7f85df800700 1 --2- 172.21.15.112:0/2240394902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d8103450 0x7f85d810f8b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.132+0000 7f85df800700 1 --2- 172.21.15.112:0/2240394902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85d8108a50 0x7f85d8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.132+0000 7f85df800700 1 --2- 172.21.15.112:0/2240394902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85d8102a90 0x7f85d8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:05.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.132+0000 7f85df800700 1 -- 172.21.15.112:0/2240394902 >> 172.21.15.112:0/2240394902 conn(0x7f85d8079ba0 msgr2=0x7f85d8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:05.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.132+0000 7f85df800700 1 -- 172.21.15.112:0/2240394902 shutdown_connections 2024-04-25T18:42:05.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.132+0000 7f85df800700 1 -- 172.21.15.112:0/2240394902 wait complete. 2024-04-25T18:42:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.133+0000 7f85df800700 1 Processor -- start 2024-04-25T18:42:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.133+0000 7f85df800700 1 -- start start 2024-04-25T18:42:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.133+0000 7f85df800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85d8102a90 0x7f85d819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85df800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85d8103450 0x7f85d819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85df800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d8108a50 0x7f85d819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85df800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8112c50 con 0x7f85d8108a50 2024-04-25T18:42:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85df800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8112ad0 con 0x7f85d8102a90 2024-04-25T18:42:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85df800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8112dd0 con 0x7f85d8103450 2024-04-25T18:42:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85dce00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85d8102a90 0x7f85d819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85dce00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85d8102a90 0x7f85d819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:56768/0 (socket says 172.21.15.112:56768) 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85dce00700 1 -- 172.21.15.112:0/2702842783 learned_addr learned my addr 172.21.15.112:0/2702842783 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85d7e00700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85d8103450 0x7f85d819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85dce00700 1 -- 172.21.15.112:0/2702842783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85d8103450 msgr2=0x7f85d819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85dce00700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85d8103450 0x7f85d819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85dce00700 1 -- 172.21.15.112:0/2702842783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d8108a50 msgr2=0x7f85d819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.134+0000 7f85dd800700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d8108a50 0x7f85d819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.135+0000 7f85dce00700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d8108a50 0x7f85d819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.135+0000 7f85dce00700 1 -- 172.21.15.112:0/2702842783 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d81a0370 con 0x7f85d8102a90 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.135+0000 7f85dd800700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d8108a50 0x7f85d819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.135+0000 7f85dce00700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85d8102a90 0x7f85d819b360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00eb30 tx=0x7f85cc00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.135+0000 7f85d5600700 1 -- 172.21.15.112:0/2702842783 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc00cc80 con 0x7f85d8102a90 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.135+0000 7f85d5600700 1 -- 172.21.15.112:0/2702842783 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85cc0044d0 con 0x7f85d8102a90 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.135+0000 7f85d5600700 1 -- 172.21.15.112:0/2702842783 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc010430 con 0x7f85d8102a90 2024-04-25T18:42:05.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.135+0000 7f85df800700 1 -- 172.21.15.112:0/2702842783 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d81a0660 con 0x7f85d8102a90 2024-04-25T18:42:05.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.136+0000 7f85df800700 1 -- 172.21.15.112:0/2702842783 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81a7ee0 con 0x7f85d8102a90 2024-04-25T18:42:05.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.137+0000 7f85df800700 1 -- 172.21.15.112:0/2702842783 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8069730 con 0x7f85d8102a90 2024-04-25T18:42:05.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.139+0000 7f85d5600700 1 -- 172.21.15.112:0/2702842783 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f85cc003ed0 con 0x7f85d8102a90 2024-04-25T18:42:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.140+0000 7f85d5600700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f85c00a3cc0 0x7f85c00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.140+0000 7f85d5600700 1 -- 172.21.15.112:0/2702842783 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f85cc0d0190 con 0x7f85d8102a90 2024-04-25T18:42:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.140+0000 7f85d7e00700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f85c00a3cc0 0x7f85c00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.141+0000 7f85d7e00700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f85c00a3cc0 0x7f85c00a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f85d807a430 tx=0x7f85c4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.144+0000 7f85d5600700 1 -- 172.21.15.112:0/2702842783 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85cc09cf70 con 0x7f85d8102a90 2024-04-25T18:42:05.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.371+0000 7f85df800700 1 -- 172.21.15.112:0/2702842783 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85d819c6c0 con 0x7f85d8102a90 2024-04-25T18:42:05.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.373+0000 7f85d5600700 1 -- 172.21.15.112:0/2702842783 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+18476 (secure 0 0 0) 0x7f85cc09c870 con 0x7f85d8102a90 2024-04-25T18:42:05.375 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:42:05.376 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":85,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","created":"2024-04-25T18:33:37.379089+0000","modified":"2024-04-25T18:42:02.537716+0000","last_up_change":"2024-04-25T18:42:00.961646+0000","last_in_change":"2024-04-25T18:41:40.647162+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:37:45.106199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efb4bc01-e29b-4402-8a7a-3387d1731ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6803","nonce":232254535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6805","nonce":232254535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6809","nonce":232254535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6807","nonce":232254535}]},"public_addr":"172.21.15.112:6803/232254535","cluster_addr":"172.21.15.112:6805/232254535","heartbeat_back_addr":"172.21.15.112:6809/232254535","heartbeat_front_addr":"172.21.15.112:6807/232254535","state":["exists","up"]},{"osd":1,"uuid":"65a83fa2-96a6-4743-8c17-21531f5e885e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6801","nonce":664224744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6803","nonce":664224744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6807","nonce":664224744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6805","nonce":664224744}]},"public_addr":"172.21.15.161:6801/664224744","cluster_addr":"172.21.15.161:6803/664224744","heartbeat_back_addr":"172.21.15.161:6807/664224744","heartbeat_front_addr":"172.21.15.161:6805/664224744","state":["exists","up"]},{"osd":2,"uuid":"7562ec4c-c126-48ec-a51c-56fa461a2e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6801","nonce":345172073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6803","nonce":345172073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6807","nonce":345172073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6805","nonce":345172073}]},"public_addr":"172.21.15.171:6801/345172073","cluster_addr":"172.21.15.171:6803/345172073","heartbeat_back_addr":"172.21.15.171:6807/345172073","heartbeat_front_addr":"172.21.15.171:6805/345172073","state":["exists","up"]},{"osd":3,"uuid":"d7aae8cd-fec7-46d9-8773-25eaaca3cc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6811","nonce":1472407156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6813","nonce":1472407156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6817","nonce":1472407156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6815","nonce":1472407156}]},"public_addr":"172.21.15.112:6811/1472407156","cluster_addr":"172.21.15.112:6813/1472407156","heartbeat_back_addr":"172.21.15.112:6817/1472407156","heartbeat_front_addr":"172.21.15.112:6815/1472407156","state":["exists","up"]},{"osd":4,"uuid":"b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6809","nonce":373204306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6811","nonce":373204306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6815","nonce":373204306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6813","nonce":373204306}]},"public_addr":"172.21.15.161:6809/373204306","cluster_addr":"172.21.15.161:6811/373204306","heartbeat_back_addr":"172.21.15.161:6815/373204306","heartbeat_front_addr":"172.21.15.161:6813/373204306","state":["exists","up"]},{"osd":5,"uuid":"56124d12-bc1f-4cba-8430-938248851776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6809","nonce":2643512313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6811","nonce":2643512313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6815","nonce":2643512313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6813","nonce":2643512313}]},"public_addr":"172.21.15.171:6809/2643512313","cluster_addr":"172.21.15.171:6811/2643512313","heartbeat_back_addr":"172.21.15.171:6815/2643512313","heartbeat_front_addr":"172.21.15.171:6813/2643512313","state":["exists","up"]},{"osd":6,"uuid":"9563a9d3-352b-4686-975c-282ec167248b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6819","nonce":1933654933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6821","nonce":1933654933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6825","nonce":1933654933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6823","nonce":1933654933}]},"public_addr":"172.21.15.112:6819/1933654933","cluster_addr":"172.21.15.112:6821/1933654933","heartbeat_back_addr":"172.21.15.112:6825/1933654933","heartbeat_front_addr":"172.21.15.112:6823/1933654933","state":["exists","up"]},{"osd":7,"uuid":"f2d06e86-6aac-40f4-956e-18fe0b0e3543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6817","nonce":3060201138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6819","nonce":3060201138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6823","nonce":3060201138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6821","nonce":3060201138}]},"public_addr":"172.21.15.161:6817/3060201138","cluster_addr":"172.21.15.161:6819/3060201138","heartbeat_back_addr":"172.21.15.161:6823/3060201138","heartbeat_front_addr":"172.21.15.161:6821/3060201138","state":["exists","up"]},{"osd":8,"uuid":"f006e45f-2bb9-45d0-9020-be7c33c6a825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6817","nonce":835756571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6819","nonce":835756571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6823","nonce":835756571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6821","nonce":835756571}]},"public_addr":"172.21.15.171:6817/835756571","cluster_addr":"172.21.15.171:6819/835756571","heartbeat_back_addr":"172.21.15.171:6823/835756571","heartbeat_front_addr":"172.21.15.171:6821/835756571","state":["exists","up"]},{"osd":9,"uuid":"c5a96491-4151-4633-b3bc-2c214fe51c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6827","nonce":349758528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6829","nonce":349758528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6833","nonce":349758528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6831","nonce":349758528}]},"public_addr":"172.21.15.112:6827/349758528","cluster_addr":"172.21.15.112:6829/349758528","heartbeat_back_addr":"172.21.15.112:6833/349758528","heartbeat_front_addr":"172.21.15.112:6831/349758528","state":["exists","up"]},{"osd":10,"uuid":"244eda3e-25a0-490e-879f-c1e2726c1be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6825","nonce":3013292765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6827","nonce":3013292765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6831","nonce":3013292765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6829","nonce":3013292765}]},"public_addr":"172.21.15.161:6825/3013292765","cluster_addr":"172.21.15.161:6827/3013292765","heartbeat_back_addr":"172.21.15.161:6831/3013292765","heartbeat_front_addr":"172.21.15.161:6829/3013292765","state":["exists","up"]},{"osd":11,"uuid":"356e5c54-bbc3-45ee-bda2-85164ce3cb92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6825","nonce":1010336458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6827","nonce":1010336458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6831","nonce":1010336458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6829","nonce":1010336458}]},"public_addr":"172.21.15.171:6825/1010336458","cluster_addr":"172.21.15.171:6827/1010336458","heartbeat_back_addr":"172.21.15.171:6831/1010336458","heartbeat_front_addr":"172.21.15.171:6829/1010336458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:36:51.024650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:17.026638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:41.239726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:15.148968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:41.029118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:06.225858+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:41.639444+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:07.753400+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:32.292085+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:07.159926+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:33.749782+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:58.594447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/181862246":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/3393454453":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3195850527":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6801/3468460974":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3392048544":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/1329430933":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/735578423":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/3468460974":"2024-04-26T18:34:20.605775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:42:05.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.376+0000 7f85df800700 1 -- 172.21.15.112:0/2702842783 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f85c00a3cc0 msgr2=0x7f85c00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:05.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.376+0000 7f85df800700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f85c00a3cc0 0x7f85c00a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f85d807a430 tx=0x7f85c4009310 comp rx=0 tx=0).stop 2024-04-25T18:42:05.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.376+0000 7f85df800700 1 -- 172.21.15.112:0/2702842783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85d8102a90 msgr2=0x7f85d819b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:05.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.376+0000 7f85df800700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85d8102a90 0x7f85d819b360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00eb30 tx=0x7f85cc00eef0 comp rx=0 tx=0).stop 2024-04-25T18:42:05.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.377+0000 7f85df800700 1 -- 172.21.15.112:0/2702842783 shutdown_connections 2024-04-25T18:42:05.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.377+0000 7f85df800700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f85c00a3cc0 0x7f85c00a6180 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:05.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.377+0000 7f85df800700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d8108a50 0x7f85d819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:05.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.377+0000 7f85df800700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85d8102a90 0x7f85d819b360 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:05.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.377+0000 7f85df800700 1 --2- 172.21.15.112:0/2702842783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85d8103450 0x7f85d819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:05.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.377+0000 7f85df800700 1 -- 172.21.15.112:0/2702842783 >> 172.21.15.112:0/2702842783 conn(0x7f85d8079ba0 msgr2=0x7f85d810df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:05.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.377+0000 7f85df800700 1 -- 172.21.15.112:0/2702842783 shutdown_connections 2024-04-25T18:42:05.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:05.377+0000 7f85df800700 1 -- 172.21.15.112:0/2702842783 wait complete. 2024-04-25T18:42:06.159 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T18:37:45.106199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T18:42:06.160 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T18:42:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:06 smithi171 ceph-mon[33266]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:06 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2702842783' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:42:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:06 smithi161 ceph-mon[32609]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:06 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2702842783' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:42:06.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:06 smithi112 ceph-mon[27864]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:06.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:06 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2702842783' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:42:06.604 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:42:08.377 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:08 smithi112 ceph-mon[27864]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:08.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:08 smithi171 ceph-mon[33266]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:08 smithi161 ceph-mon[32609]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.866+0000 7fe933e00700 1 -- 172.21.15.112:0/544251322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe934103ba0 msgr2=0x7fe934103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.866+0000 7fe933e00700 1 --2- 172.21.15.112:0/544251322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe934103ba0 0x7fe934103f80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe924009990 tx=0x7fe924009ca0 comp rx=0 tx=0).stop 2024-04-25T18:42:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.866+0000 7fe933e00700 1 -- 172.21.15.112:0/544251322 shutdown_connections 2024-04-25T18:42:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.866+0000 7fe933e00700 1 --2- 172.21.15.112:0/544251322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe93410cfd0 0x7fe93410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.866+0000 7fe933e00700 1 --2- 172.21.15.112:0/544251322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe934103ba0 0x7fe934103f80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.866+0000 7fe933e00700 1 --2- 172.21.15.112:0/544251322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9341044c0 0x7fe93410c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.866+0000 7fe933e00700 1 -- 172.21.15.112:0/544251322 >> 172.21.15.112:0/544251322 conn(0x7fe9340fd3f0 msgr2=0x7fe9340ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.866+0000 7fe933e00700 1 -- 172.21.15.112:0/544251322 shutdown_connections 2024-04-25T18:42:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.866+0000 7fe933e00700 1 -- 172.21.15.112:0/544251322 wait complete. 2024-04-25T18:42:08.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.867+0000 7fe933e00700 1 Processor -- start 2024-04-25T18:42:08.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe933e00700 1 -- start start 2024-04-25T18:42:08.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe933e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe934103ba0 0x7fe9341991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:08.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe933e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9341044c0 0x7fe934199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:08.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe933e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe93410cfd0 0x7fe93419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:08.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe933e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe934112c30 con 0x7fe93410cfd0 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe933e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe934112ab0 con 0x7fe9341044c0 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe933e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe934112db0 con 0x7fe934103ba0 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe933400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe93410cfd0 0x7fe93419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe932a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe934103ba0 0x7fe9341991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe932a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe934103ba0 0x7fe9341991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:33160/0 (socket says 172.21.15.112:33160) 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe932a00700 1 -- 172.21.15.112:0/7885847 learned_addr learned my addr 172.21.15.112:0/7885847 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.868+0000 7fe932000700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9341044c0 0x7fe934199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.869+0000 7fe932a00700 1 -- 172.21.15.112:0/7885847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9341044c0 msgr2=0x7fe934199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.869+0000 7fe932a00700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9341044c0 0x7fe934199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.869+0000 7fe932a00700 1 -- 172.21.15.112:0/7885847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe93410cfd0 msgr2=0x7fe93419da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.869+0000 7fe932a00700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe93410cfd0 0x7fe93419da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.869+0000 7fe932a00700 1 -- 172.21.15.112:0/7885847 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93419e180 con 0x7fe934103ba0 2024-04-25T18:42:08.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.869+0000 7fe932a00700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe934103ba0 0x7fe9341991c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe924000c00 tx=0x7fe9240057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:08.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.869+0000 7fe92b400700 1 -- 172.21.15.112:0/7885847 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924027070 con 0x7fe934103ba0 2024-04-25T18:42:08.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.869+0000 7fe92b400700 1 -- 172.21.15.112:0/7885847 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe924022470 con 0x7fe934103ba0 2024-04-25T18:42:08.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.869+0000 7fe933e00700 1 -- 172.21.15.112:0/7885847 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93419e410 con 0x7fe934103ba0 2024-04-25T18:42:08.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.869+0000 7fe92b400700 1 -- 172.21.15.112:0/7885847 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924017650 con 0x7fe934103ba0 2024-04-25T18:42:08.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.870+0000 7fe933e00700 1 -- 172.21.15.112:0/7885847 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9341a5cf0 con 0x7fe934103ba0 2024-04-25T18:42:08.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.871+0000 7fe933e00700 1 -- 172.21.15.112:0/7885847 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe934069730 con 0x7fe934103ba0 2024-04-25T18:42:08.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.873+0000 7fe92b400700 1 -- 172.21.15.112:0/7885847 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe92402a070 con 0x7fe934103ba0 2024-04-25T18:42:08.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.874+0000 7fe92b400700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe9180a3cc0 0x7fe9180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:08.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.874+0000 7fe92b400700 1 -- 172.21.15.112:0/7885847 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fe9240d0c10 con 0x7fe934103ba0 2024-04-25T18:42:08.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.874+0000 7fe932000700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe9180a3cc0 0x7fe9180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:08.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.875+0000 7fe932000700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe9180a3cc0 0x7fe9180a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe93410e5a0 tx=0x7fe920009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:08.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:08.876+0000 7fe92b400700 1 -- 172.21.15.112:0/7885847 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe924013070 con 0x7fe934103ba0 2024-04-25T18:42:09.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.104+0000 7fe933e00700 1 -- 172.21.15.112:0/7885847 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe93402df80 con 0x7fe934103ba0 2024-04-25T18:42:09.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.105+0000 7fe92b400700 1 -- 172.21.15.112:0/7885847 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7fe92409da80 con 0x7fe934103ba0 2024-04-25T18:42:09.107 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T18:42:09.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.108+0000 7fe933e00700 1 -- 172.21.15.112:0/7885847 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe9180a3cc0 msgr2=0x7fe9180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:09.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.108+0000 7fe933e00700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe9180a3cc0 0x7fe9180a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe93410e5a0 tx=0x7fe920009260 comp rx=0 tx=0).stop 2024-04-25T18:42:09.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.109+0000 7fe933e00700 1 -- 172.21.15.112:0/7885847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe934103ba0 msgr2=0x7fe9341991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:09.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.109+0000 7fe933e00700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe934103ba0 0x7fe9341991c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe924000c00 tx=0x7fe9240057c0 comp rx=0 tx=0).stop 2024-04-25T18:42:09.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.109+0000 7fe933e00700 1 -- 172.21.15.112:0/7885847 shutdown_connections 2024-04-25T18:42:09.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.109+0000 7fe933e00700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe9180a3cc0 0x7fe9180a6180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:09.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.109+0000 7fe933e00700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe93410cfd0 0x7fe93419da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:09.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.109+0000 7fe933e00700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe934103ba0 0x7fe9341991c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:09.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.109+0000 7fe933e00700 1 --2- 172.21.15.112:0/7885847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9341044c0 0x7fe934199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:09.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.109+0000 7fe933e00700 1 -- 172.21.15.112:0/7885847 >> 172.21.15.112:0/7885847 conn(0x7fe9340fd3f0 msgr2=0x7fe9340fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:09.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.109+0000 7fe933e00700 1 -- 172.21.15.112:0/7885847 shutdown_connections 2024-04-25T18:42:09.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:09.109+0000 7fe933e00700 1 -- 172.21.15.112:0/7885847 wait complete. 2024-04-25T18:42:09.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:09 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/7885847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:42:09.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:09 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/7885847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:42:09.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:09 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/7885847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:42:09.981 INFO:tasks.cephadm:Adding mds.a on smithi112 2024-04-25T18:42:09.981 INFO:tasks.cephadm:Adding mds.d on smithi112 2024-04-25T18:42:09.981 INFO:tasks.cephadm:Adding mds.g on smithi112 2024-04-25T18:42:09.981 INFO:tasks.cephadm:Adding mds.j on smithi112 2024-04-25T18:42:09.981 INFO:tasks.cephadm:Adding mds.b on smithi161 2024-04-25T18:42:09.981 INFO:tasks.cephadm:Adding mds.e on smithi161 2024-04-25T18:42:09.981 INFO:tasks.cephadm:Adding mds.h on smithi161 2024-04-25T18:42:09.982 INFO:tasks.cephadm:Adding mds.k on smithi161 2024-04-25T18:42:09.982 INFO:tasks.cephadm:Adding mds.c on smithi171 2024-04-25T18:42:09.982 INFO:tasks.cephadm:Adding mds.f on smithi171 2024-04-25T18:42:09.982 INFO:tasks.cephadm:Adding mds.i on smithi171 2024-04-25T18:42:09.982 INFO:tasks.cephadm:Adding mds.l on smithi171 2024-04-25T18:42:09.982 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l' 2024-04-25T18:42:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:10 smithi112 ceph-mon[27864]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:10.232 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:42:10.333 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:10 smithi171 ceph-mon[33266]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:10.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:10 smithi161 ceph-mon[32609]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:11.842 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.793+0000 7fe900d62700 1 -- 172.21.15.171:0/630810948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8fc103570 msgr2=0x7fe8fc105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:11.842 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.793+0000 7fe900d62700 1 --2- 172.21.15.171:0/630810948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8fc103570 0x7fe8fc105960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009990 tx=0x7fe8ec009ca0 comp rx=0 tx=0).stop 2024-04-25T18:42:11.842 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 -- 172.21.15.171:0/630810948 shutdown_connections 2024-04-25T18:42:11.842 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 --2- 172.21.15.171:0/630810948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8fc1087d0 0x7fe8fc10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:11.842 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 --2- 172.21.15.171:0/630810948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8fc103570 0x7fe8fc105960 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 --2- 172.21.15.171:0/630810948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8fc105ea0 0x7fe8fc108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 -- 172.21.15.171:0/630810948 >> 172.21.15.171:0/630810948 conn(0x7fe8fc0fcfa0 msgr2=0x7fe8fc0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 -- 172.21.15.171:0/630810948 shutdown_connections 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 -- 172.21.15.171:0/630810948 wait complete. 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 Processor -- start 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 -- start start 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8fc103570 0x7fe8fc19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8fc105ea0 0x7fe8fc19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.794+0000 7fe900d62700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8fc1087d0 0x7fe8fc1a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe900d62700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc10e6b0 con 0x7fe8fc1087d0 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe900d62700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc10e530 con 0x7fe8fc105ea0 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe900d62700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc10e830 con 0x7fe8fc103570 2024-04-25T18:42:11.843 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8fa59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8fc103570 0x7fe8fc19afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8fa59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8fc103570 0x7fe8fc19afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:54262/0 (socket says 172.21.15.171:54262) 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8fa59c700 1 -- 172.21.15.171:0/2831365059 learned_addr learned my addr 172.21.15.171:0/2831365059 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8f9d9b700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8fc105ea0 0x7fe8fc19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8fa59c700 1 -- 172.21.15.171:0/2831365059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8fc105ea0 msgr2=0x7fe8fc19b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8fa59c700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8fc105ea0 0x7fe8fc19b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8fa59c700 1 -- 172.21.15.171:0/2831365059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8fc1087d0 msgr2=0x7fe8fc1a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8fa59c700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8fc1087d0 0x7fe8fc1a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8fa59c700 1 -- 172.21.15.171:0/2831365059 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc1a2c40 con 0x7fe8fc103570 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8fa59c700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8fc103570 0x7fe8fc19afe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec000c00 tx=0x7fe8ec0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8f37fe700 1 -- 172.21.15.171:0/2831365059 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec027070 con 0x7fe8fc103570 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe900d62700 1 -- 172.21.15.171:0/2831365059 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc1a2ed0 con 0x7fe8fc103570 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8f37fe700 1 -- 172.21.15.171:0/2831365059 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ec004100 con 0x7fe8fc103570 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe8f37fe700 1 -- 172.21.15.171:0/2831365059 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00f460 con 0x7fe8fc103570 2024-04-25T18:42:11.844 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.795+0000 7fe900d62700 1 -- 172.21.15.171:0/2831365059 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc1a3340 con 0x7fe8fc103570 2024-04-25T18:42:11.845 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.796+0000 7fe900d62700 1 -- 172.21.15.171:0/2831365059 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc069730 con 0x7fe8fc103570 2024-04-25T18:42:11.845 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.797+0000 7fe8f37fe700 1 -- 172.21.15.171:0/2831365059 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe8ec02a070 con 0x7fe8fc103570 2024-04-25T18:42:11.845 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.798+0000 7fe8f37fe700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe8e80a3c40 0x7fe8e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:11.845 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.798+0000 7fe8f37fe700 1 -- 172.21.15.171:0/2831365059 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fe8ec0d1170 con 0x7fe8fc103570 2024-04-25T18:42:11.845 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.798+0000 7fe8f9d9b700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe8e80a3c40 0x7fe8e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:11.845 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.799+0000 7fe8f9d9b700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe8e80a3c40 0x7fe8e80a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009a10 tx=0x7fe8e4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:11.845 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:11.801+0000 7fe8f37fe700 1 -- 172.21.15.171:0/2831365059 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8ec013070 con 0x7fe8fc103570 2024-04-25T18:42:12.011 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.010+0000 7fe900d62700 1 -- 172.21.15.171:0/2831365059 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}) -- 0x7fe8fc062f80 con 0x7fe8e80a3c40 2024-04-25T18:42:12.027 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.026+0000 7fe8f37fe700 1 -- 172.21.15.171:0/2831365059 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe8fc062f80 con 0x7fe8e80a3c40 2024-04-25T18:42:12.028 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mds.all update... 2024-04-25T18:42:12.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.028+0000 7fe900d62700 1 -- 172.21.15.171:0/2831365059 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe8e80a3c40 msgr2=0x7fe8e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:12.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.028+0000 7fe900d62700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe8e80a3c40 0x7fe8e80a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009a10 tx=0x7fe8e4009190 comp rx=0 tx=0).stop 2024-04-25T18:42:12.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.028+0000 7fe900d62700 1 -- 172.21.15.171:0/2831365059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8fc103570 msgr2=0x7fe8fc19afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:12.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.028+0000 7fe900d62700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8fc103570 0x7fe8fc19afe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec000c00 tx=0x7fe8ec0047b0 comp rx=0 tx=0).stop 2024-04-25T18:42:12.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.028+0000 7fe900d62700 1 -- 172.21.15.171:0/2831365059 shutdown_connections 2024-04-25T18:42:12.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.028+0000 7fe900d62700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe8e80a3c40 0x7fe8e80a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:12.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.028+0000 7fe900d62700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8fc1087d0 0x7fe8fc1a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:12.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.028+0000 7fe900d62700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8fc103570 0x7fe8fc19afe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:12.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.028+0000 7fe900d62700 1 --2- 172.21.15.171:0/2831365059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8fc105ea0 0x7fe8fc19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:12.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.028+0000 7fe900d62700 1 -- 172.21.15.171:0/2831365059 >> 172.21.15.171:0/2831365059 conn(0x7fe8fc0fcfa0 msgr2=0x7fe8fc102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:12.030 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.028+0000 7fe900d62700 1 -- 172.21.15.171:0/2831365059 shutdown_connections 2024-04-25T18:42:12.031 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T18:42:12.029+0000 7fe900d62700 1 -- 172.21.15.171:0/2831365059 wait complete. 2024-04-25T18:42:12.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:12 smithi171 ceph-mon[33266]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:12.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:12 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:12.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:12 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:42:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:12 smithi161 ceph-mon[32609]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:12 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:12 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:42:12.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:12 smithi112 ceph-mon[27864]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:12.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:12 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:12.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:12 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:42:12.647 DEBUG:teuthology.orchestra.run.smithi112:mds.a> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T18:42:12.654 DEBUG:teuthology.orchestra.run.smithi112:mds.d> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T18:42:12.702 DEBUG:teuthology.orchestra.run.smithi112:mds.g> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T18:42:12.749 DEBUG:teuthology.orchestra.run.smithi112:mds.j> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T18:42:12.795 DEBUG:teuthology.orchestra.run.smithi161:mds.b> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T18:42:12.798 DEBUG:teuthology.orchestra.run.smithi161:mds.e> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T18:42:12.841 DEBUG:teuthology.orchestra.run.smithi161:mds.h> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T18:42:12.884 DEBUG:teuthology.orchestra.run.smithi161:mds.k> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T18:42:12.927 DEBUG:teuthology.orchestra.run.smithi171:mds.c> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T18:42:12.930 DEBUG:teuthology.orchestra.run.smithi171:mds.f> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T18:42:12.974 DEBUG:teuthology.orchestra.run.smithi171:mds.i> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T18:42:13.017 DEBUG:teuthology.orchestra.run.smithi171:mds.l> sudo journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T18:42:13.060 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T18:42:13.060 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:42:13.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:13 smithi171 ceph-mon[33266]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:42:13.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:13 smithi171 ceph-mon[33266]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-25T18:42:13.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:13 smithi161 ceph-mon[32609]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:42:13.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:13 smithi161 ceph-mon[32609]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-25T18:42:13.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:13 smithi112 ceph-mon[27864]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:42:13.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:13 smithi112 ceph-mon[27864]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-25T18:42:13.480 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:42:14.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:14 smithi171 ceph-mon[33266]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:14.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:42:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:14 smithi171 ceph-mon[33266]: Deploying daemon mds.l on smithi171 2024-04-25T18:42:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:14 smithi161 ceph-mon[32609]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:42:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:14.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:14.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:14.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:14 smithi161 ceph-mon[32609]: Deploying daemon mds.l on smithi171 2024-04-25T18:42:14.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:14 smithi112 ceph-mon[27864]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:14.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:14.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:42:14.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:14.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:14.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:14.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:14.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:14 smithi112 ceph-mon[27864]: Deploying daemon mds.l on smithi171 2024-04-25T18:42:15.376 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 18:42:15 smithi171 systemd[1]: Starting Ceph mds.l for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:42:15.657 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 18:42:15 smithi171 podman[63034]: 2024-04-25 18:42:15.278293463 +0000 UTC m=+0.017884536 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:42:15.657 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 18:42:15 smithi171 podman[63034]: 2024-04-25 18:42:15.404337182 +0000 UTC m=+0.143928244 container create a9264b1d0db3be078a41eb9e8a99656bafc069cd7eacb5f8ae19f5ae3876086a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T18:42:15.657 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 18:42:15 smithi171 podman[63034]: 2024-04-25 18:42:15.656528738 +0000 UTC m=+0.396119800 container init a9264b1d0db3be078a41eb9e8a99656bafc069cd7eacb5f8ae19f5ae3876086a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:42:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.845+0000 7f972bc00700 1 -- 172.21.15.112:0/726404335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9724074c70 msgr2=0x7f9724075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.845+0000 7f972bc00700 1 --2- 172.21.15.112:0/726404335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9724074c70 0x7f9724075050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97180098f0 tx=0x7f9718009c00 comp rx=0 tx=0).stop 2024-04-25T18:42:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.846+0000 7f972bc00700 1 -- 172.21.15.112:0/726404335 shutdown_connections 2024-04-25T18:42:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.846+0000 7f972bc00700 1 --2- 172.21.15.112:0/726404335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972410f440 0x7f9724111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.846+0000 7f972bc00700 1 --2- 172.21.15.112:0/726404335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9724074c70 0x7f9724075050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.846+0000 7f972bc00700 1 --2- 172.21.15.112:0/726404335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9724075590 0x7f972410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.846+0000 7f972bc00700 1 -- 172.21.15.112:0/726404335 >> 172.21.15.112:0/726404335 conn(0x7f9724079ba0 msgr2=0x7f9724079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:15.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.846+0000 7f972bc00700 1 -- 172.21.15.112:0/726404335 shutdown_connections 2024-04-25T18:42:15.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.846+0000 7f972bc00700 1 -- 172.21.15.112:0/726404335 wait complete. 2024-04-25T18:42:15.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.847+0000 7f972bc00700 1 Processor -- start 2024-04-25T18:42:15.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.847+0000 7f972bc00700 1 -- start start 2024-04-25T18:42:15.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.848+0000 7f972bc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9724074c70 0x7f972419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:15.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.848+0000 7f972bc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9724075590 0x7f972419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:15.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.848+0000 7f972bc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972410f440 0x7f972419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:15.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.848+0000 7f972bc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9724114e70 con 0x7f972410f440 2024-04-25T18:42:15.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.848+0000 7f972bc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9724114cf0 con 0x7f9724074c70 2024-04-25T18:42:15.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.848+0000 7f972bc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9724114ff0 con 0x7f9724075590 2024-04-25T18:42:15.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.848+0000 7f9723e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9724075590 0x7f972419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:15.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.848+0000 7f9723e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9724075590 0x7f972419b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:57340/0 (socket says 172.21.15.112:57340) 2024-04-25T18:42:15.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.848+0000 7f9723e00700 1 -- 172.21.15.112:0/2175549854 learned_addr learned my addr 172.21.15.112:0/2175549854 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:42:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.848+0000 7f9729c00700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972410f440 0x7f972419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.848+0000 7f9729200700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9724074c70 0x7f972419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.849+0000 7f9723e00700 1 -- 172.21.15.112:0/2175549854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9724074c70 msgr2=0x7f972419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.849+0000 7f9723e00700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9724074c70 0x7f972419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.849+0000 7f9723e00700 1 -- 172.21.15.112:0/2175549854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972410f440 msgr2=0x7f972419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.849+0000 7f9723e00700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972410f440 0x7f972419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.849+0000 7f9723e00700 1 -- 172.21.15.112:0/2175549854 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97241a0370 con 0x7f9724075590 2024-04-25T18:42:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.849+0000 7f9723e00700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9724075590 0x7f972419b8a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f971000e950 tx=0x7f971000ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.849+0000 7f9721600700 1 -- 172.21.15.112:0/2175549854 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971000eea0 con 0x7f9724075590 2024-04-25T18:42:15.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.849+0000 7f9721600700 1 -- 172.21.15.112:0/2175549854 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97100044e0 con 0x7f9724075590 2024-04-25T18:42:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.849+0000 7f9721600700 1 -- 172.21.15.112:0/2175549854 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710003e00 con 0x7f9724075590 2024-04-25T18:42:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.849+0000 7f972bc00700 1 -- 172.21.15.112:0/2175549854 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97241a0660 con 0x7f9724075590 2024-04-25T18:42:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.849+0000 7f972bc00700 1 -- 172.21.15.112:0/2175549854 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97241a7ee0 con 0x7f9724075590 2024-04-25T18:42:15.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.851+0000 7f972bc00700 1 -- 172.21.15.112:0/2175549854 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9724069730 con 0x7f9724075590 2024-04-25T18:42:15.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.852+0000 7f9721600700 1 -- 172.21.15.112:0/2175549854 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9710003fa0 con 0x7f9724075590 2024-04-25T18:42:15.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.854+0000 7f9721600700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f970c0a3cc0 0x7f970c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.854+0000 7f9721600700 1 -- 172.21.15.112:0/2175549854 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f97100d02c0 con 0x7f9724075590 2024-04-25T18:42:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.854+0000 7f9729200700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f970c0a3cc0 0x7f970c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:15.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.855+0000 7f9729200700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f970c0a3cc0 0x7f970c0a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f97180098c0 tx=0x7f9718019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:15.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:15.858+0000 7f9721600700 1 -- 172.21.15.112:0/2175549854 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f971009d0a0 con 0x7f9724075590 2024-04-25T18:42:15.932 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 18:42:15 smithi171 podman[63034]: 2024-04-25 18:42:15.659799107 +0000 UTC m=+0.399390170 container start a9264b1d0db3be078a41eb9e8a99656bafc069cd7eacb5f8ae19f5ae3876086a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T18:42:15.932 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 18:42:15 smithi171 ceph-mds[63053]: -- 172.21.15.171:0/2545039473 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cde49dda40 con 0x55cde49b1000 2024-04-25T18:42:15.932 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 18:42:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-l[63049]: starting mds.l at 2024-04-25T18:42:15.932 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 18:42:15 smithi171 bash[63034]: a9264b1d0db3be078a41eb9e8a99656bafc069cd7eacb5f8ae19f5ae3876086a 2024-04-25T18:42:15.933 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 18:42:15 smithi171 systemd[1]: Started Ceph mds.l for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:16.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.144+0000 7f972bc00700 1 -- 172.21.15.112:0/2175549854 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f972402df80 con 0x7f9724075590 2024-04-25T18:42:16.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.145+0000 7f9721600700 1 -- 172.21.15.112:0/2175549854 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f971009c9a0 con 0x7f9724075590 2024-04-25T18:42:16.147 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:42:16.147 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T18:42:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.148+0000 7f972bc00700 1 -- 172.21.15.112:0/2175549854 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f970c0a3cc0 msgr2=0x7f970c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.148+0000 7f972bc00700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f970c0a3cc0 0x7f970c0a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f97180098c0 tx=0x7f9718019040 comp rx=0 tx=0).stop 2024-04-25T18:42:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.148+0000 7f972bc00700 1 -- 172.21.15.112:0/2175549854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9724075590 msgr2=0x7f972419b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.148+0000 7f972bc00700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9724075590 0x7f972419b8a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f971000e950 tx=0x7f971000ec60 comp rx=0 tx=0).stop 2024-04-25T18:42:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.148+0000 7f972bc00700 1 -- 172.21.15.112:0/2175549854 shutdown_connections 2024-04-25T18:42:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.148+0000 7f972bc00700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f970c0a3cc0 0x7f970c0a6180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.148+0000 7f972bc00700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f972410f440 0x7f972419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.148+0000 7f972bc00700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9724074c70 0x7f972419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.148+0000 7f972bc00700 1 --2- 172.21.15.112:0/2175549854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9724075590 0x7f972419b8a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.148+0000 7f972bc00700 1 -- 172.21.15.112:0/2175549854 >> 172.21.15.112:0/2175549854 conn(0x7f9724079ba0 msgr2=0x7f97240ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.149+0000 7f972bc00700 1 -- 172.21.15.112:0/2175549854 shutdown_connections 2024-04-25T18:42:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:16.149+0000 7f972bc00700 1 -- 172.21.15.112:0/2175549854 wait complete. 2024-04-25T18:42:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-04-25T18:42:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:16 smithi171 ceph-mon[33266]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:16 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.171:6832/1467985525,v1:172.21.15.171:6833/1467985525] up:boot 2024-04-25T18:42:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:16 smithi171 ceph-mon[33266]: fsmap 1 up:standby 2024-04-25T18:42:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:42:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:16.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:16.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:16 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:16.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:16 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2175549854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:16 smithi161 ceph-mon[32609]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:16 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.171:6832/1467985525,v1:172.21.15.171:6833/1467985525] up:boot 2024-04-25T18:42:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:16 smithi161 ceph-mon[32609]: fsmap 1 up:standby 2024-04-25T18:42:16.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:42:16.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:16.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:16.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:16.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:16.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:16.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:16 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:16.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:16 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2175549854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:16 smithi112 ceph-mon[27864]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:16 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.171:6832/1467985525,v1:172.21.15.171:6833/1467985525] up:boot 2024-04-25T18:42:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:16 smithi112 ceph-mon[27864]: fsmap 1 up:standby 2024-04-25T18:42:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:42:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:16 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:16.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:16 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2175549854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:17.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:17 smithi112 ceph-mon[27864]: Deploying daemon mds.k on smithi161 2024-04-25T18:42:17.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:17 smithi171 ceph-mon[33266]: Deploying daemon mds.k on smithi161 2024-04-25T18:42:17.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:17 smithi161 ceph-mon[32609]: Deploying daemon mds.k on smithi161 2024-04-25T18:42:18.176 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 18:42:17 smithi161 systemd[1]: Starting Ceph mds.k for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:42:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:18 smithi171 ceph-mon[33266]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:18.445 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:18 smithi161 ceph-mon[32609]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:18.445 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 18:42:18 smithi161 podman[61863]: 2024-04-25 18:42:18.198494596 +0000 UTC m=+0.114219704 container create fcfccf6165d139fea0bebd47fee996c3163605c531653deda8e76d4cf8a8218a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T18:42:18.446 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 18:42:18 smithi161 podman[61863]: 2024-04-25 18:42:18.102386121 +0000 UTC m=+0.018111242 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:42:18.446 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 18:42:18 smithi161 podman[61863]: 2024-04-25 18:42:18.41750372 +0000 UTC m=+0.333228835 container init fcfccf6165d139fea0bebd47fee996c3163605c531653deda8e76d4cf8a8218a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T18:42:18.446 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 18:42:18 smithi161 podman[61863]: 2024-04-25 18:42:18.420206035 +0000 UTC m=+0.335931143 container start fcfccf6165d139fea0bebd47fee996c3163605c531653deda8e76d4cf8a8218a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T18:42:18.446 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 18:42:18 smithi161 ceph-mds[61882]: -- 172.21.15.161:0/1137916392 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5616717f3a40 con 0x5616717c6800 2024-04-25T18:42:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:18 smithi112 ceph-mon[27864]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:18.721 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 18:42:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-k[61878]: starting mds.k at 2024-04-25T18:42:18.721 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 18:42:18 smithi161 bash[61863]: fcfccf6165d139fea0bebd47fee996c3163605c531653deda8e76d4cf8a8218a 2024-04-25T18:42:18.721 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 18:42:18 smithi161 systemd[1]: Started Ceph mds.k for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:19.042 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:42:19.464 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:42:19.821 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:19 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:19.821 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:19 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:19.821 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:19 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:19.822 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:19 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:19.822 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:19 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:19.822 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:19 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:19.822 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:19 smithi161 ceph-mon[32609]: Deploying daemon mds.j on smithi112 2024-04-25T18:42:19.822 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:19 smithi161 ceph-mon[32609]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:19.822 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:19 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.161:6832/3214695167,v1:172.21.15.161:6833/3214695167] up:boot 2024-04-25T18:42:19.822 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:19 smithi161 ceph-mon[32609]: fsmap 2 up:standby 2024-04-25T18:42:19.822 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:19 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:42:19.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:19 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:19.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:19 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:19.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:19 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:19.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:19 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:19.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:19 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:19.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:19 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:19.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:19 smithi112 ceph-mon[27864]: Deploying daemon mds.j on smithi112 2024-04-25T18:42:19.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:19 smithi112 ceph-mon[27864]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:19.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:19 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.161:6832/3214695167,v1:172.21.15.161:6833/3214695167] up:boot 2024-04-25T18:42:19.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:19 smithi112 ceph-mon[27864]: fsmap 2 up:standby 2024-04-25T18:42:19.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:19 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:42:19.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:19 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:19.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:19 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:19.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:19 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:19.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:19 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:19.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:19 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:19.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:19 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:19.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:19 smithi171 ceph-mon[33266]: Deploying daemon mds.j on smithi112 2024-04-25T18:42:19.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:19 smithi171 ceph-mon[33266]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:19.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:19 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.161:6832/3214695167,v1:172.21.15.161:6833/3214695167] up:boot 2024-04-25T18:42:19.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:19 smithi171 ceph-mon[33266]: fsmap 2 up:standby 2024-04-25T18:42:19.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:19 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:42:22.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:22 smithi112 ceph-mon[27864]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:22.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:22 smithi171 ceph-mon[33266]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:22.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:22 smithi161 ceph-mon[32609]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:23.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.078+0000 7fab6f200700 1 -- 172.21.15.112:0/3137398277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab68109e90 msgr2=0x7fab68111a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:23.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.078+0000 7fab6f200700 1 --2- 172.21.15.112:0/3137398277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab68109e90 0x7fab68111a60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab64009fc0 tx=0x7fab6400b330 comp rx=0 tx=0).stop 2024-04-25T18:42:23.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.078+0000 7fab6f200700 1 -- 172.21.15.112:0/3137398277 shutdown_connections 2024-04-25T18:42:23.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.078+0000 7fab6f200700 1 --2- 172.21.15.112:0/3137398277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab68109e90 0x7fab68111a60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:23.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.078+0000 7fab6f200700 1 --2- 172.21.15.112:0/3137398277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab68104db0 0x7fab68105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.078+0000 7fab6f200700 1 --2- 172.21.15.112:0/3137398277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab68105760 0x7fab681097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.078+0000 7fab6f200700 1 -- 172.21.15.112:0/3137398277 >> 172.21.15.112:0/3137398277 conn(0x7fab68100620 msgr2=0x7fab68102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.078+0000 7fab6f200700 1 -- 172.21.15.112:0/3137398277 shutdown_connections 2024-04-25T18:42:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.078+0000 7fab6f200700 1 -- 172.21.15.112:0/3137398277 wait complete. 2024-04-25T18:42:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.079+0000 7fab6f200700 1 Processor -- start 2024-04-25T18:42:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.079+0000 7fab6f200700 1 -- start start 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.080+0000 7fab6f200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab68104db0 0x7fab6819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.080+0000 7fab6f200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab68105760 0x7fab6819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.080+0000 7fab6f200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab68109e90 0x7fab6819fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.080+0000 7fab6f200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab68114d70 con 0x7fab68104db0 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.080+0000 7fab6f200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fab68114bf0 con 0x7fab68105760 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.080+0000 7fab6f200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fab68114ef0 con 0x7fab68109e90 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.080+0000 7fab6d400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab68105760 0x7fab6819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.080+0000 7fab6de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab68104db0 0x7fab6819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.080+0000 7fab6e800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab68109e90 0x7fab6819fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.080+0000 7fab6d400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab68105760 0x7fab6819b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:41788/0 (socket says 172.21.15.112:41788) 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.080+0000 7fab6d400700 1 -- 172.21.15.112:0/216225708 learned_addr learned my addr 172.21.15.112:0/216225708 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.081+0000 7fab6d400700 1 -- 172.21.15.112:0/216225708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab68109e90 msgr2=0x7fab6819fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:23.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.081+0000 7fab6d400700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab68109e90 0x7fab6819fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.081+0000 7fab6d400700 1 -- 172.21.15.112:0/216225708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab68104db0 msgr2=0x7fab6819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.081+0000 7fab6d400700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab68104db0 0x7fab6819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.081+0000 7fab6d400700 1 -- 172.21.15.112:0/216225708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab681a0210 con 0x7fab68105760 2024-04-25T18:42:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.081+0000 7fab6e800700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab68109e90 0x7fab6819fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:42:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.081+0000 7fab6d400700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab68105760 0x7fab6819b740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab5800ec70 tx=0x7fab5800ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.082+0000 7fab62a00700 1 -- 172.21.15.112:0/216225708 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5800cc90 con 0x7fab68105760 2024-04-25T18:42:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.082+0000 7fab6f200700 1 -- 172.21.15.112:0/216225708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab681a0500 con 0x7fab68105760 2024-04-25T18:42:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.082+0000 7fab62a00700 1 -- 172.21.15.112:0/216225708 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab5800ce30 con 0x7fab68105760 2024-04-25T18:42:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.082+0000 7fab62a00700 1 -- 172.21.15.112:0/216225708 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58010430 con 0x7fab68105760 2024-04-25T18:42:23.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.082+0000 7fab6f200700 1 -- 172.21.15.112:0/216225708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab681a7d80 con 0x7fab68105760 2024-04-25T18:42:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.083+0000 7fab6f200700 1 -- 172.21.15.112:0/216225708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab68069730 con 0x7fab68105760 2024-04-25T18:42:23.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.085+0000 7fab62a00700 1 -- 172.21.15.112:0/216225708 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fab58003680 con 0x7fab68105760 2024-04-25T18:42:23.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.086+0000 7fab62a00700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fab500a3c40 0x7fab500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:23.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.087+0000 7fab62a00700 1 -- 172.21.15.112:0/216225708 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fab580d0260 con 0x7fab68105760 2024-04-25T18:42:23.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.087+0000 7fab6de00700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fab500a3c40 0x7fab500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:23.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.087+0000 7fab6de00700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fab500a3c40 0x7fab500a6100 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009c60 tx=0x7fab5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:23.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.089+0000 7fab62a00700 1 -- 172.21.15.112:0/216225708 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab5809d040 con 0x7fab68105760 2024-04-25T18:42:23.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.420+0000 7fab6f200700 1 -- 172.21.15.112:0/216225708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab6802df80 con 0x7fab68105760 2024-04-25T18:42:23.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.421+0000 7fab62a00700 1 -- 172.21.15.112:0/216225708 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fab5809c940 con 0x7fab68105760 2024-04-25T18:42:23.424 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:42:23.424 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T18:42:23.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.424+0000 7fab6f200700 1 -- 172.21.15.112:0/216225708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fab500a3c40 msgr2=0x7fab500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.424+0000 7fab6f200700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fab500a3c40 0x7fab500a6100 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009c60 tx=0x7fab5c009260 comp rx=0 tx=0).stop 2024-04-25T18:42:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.424+0000 7fab6f200700 1 -- 172.21.15.112:0/216225708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab68105760 msgr2=0x7fab6819b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.424+0000 7fab6f200700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab68105760 0x7fab6819b740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab5800ec70 tx=0x7fab5800ef80 comp rx=0 tx=0).stop 2024-04-25T18:42:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.425+0000 7fab6f200700 1 -- 172.21.15.112:0/216225708 shutdown_connections 2024-04-25T18:42:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.425+0000 7fab6f200700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fab500a3c40 0x7fab500a6100 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.425+0000 7fab6f200700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab68109e90 0x7fab6819fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.425+0000 7fab6f200700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab68104db0 0x7fab6819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.425+0000 7fab6f200700 1 --2- 172.21.15.112:0/216225708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab68105760 0x7fab6819b740 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.425+0000 7fab6f200700 1 -- 172.21.15.112:0/216225708 >> 172.21.15.112:0/216225708 conn(0x7fab68100620 msgr2=0x7fab681011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.425+0000 7fab6f200700 1 -- 172.21.15.112:0/216225708 shutdown_connections 2024-04-25T18:42:23.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:23.425+0000 7fab6f200700 1 -- 172.21.15.112:0/216225708 wait complete. 2024-04-25T18:42:23.428 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 3 2024-04-25T18:42:23.678 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 18:42:23 smithi112 podman[65529]: 2024-04-25 18:42:23.564957076 +0000 UTC m=+0.199590792 container create 8bec2ea380494db5be8516275e6112b5d9788239006a0809fdb8552a3b640bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:42:24.304 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:24 smithi112 ceph-mon[27864]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:24.304 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:24 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/216225708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:24.305 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 18:42:24 smithi112 podman[65529]: 2024-04-25 18:42:24.187319832 +0000 UTC m=+0.821953555 container init 8bec2ea380494db5be8516275e6112b5d9788239006a0809fdb8552a3b640bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T18:42:24.305 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 18:42:24 smithi112 podman[65529]: 2024-04-25 18:42:24.197366701 +0000 UTC m=+0.832000424 container start 8bec2ea380494db5be8516275e6112b5d9788239006a0809fdb8552a3b640bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T18:42:24.305 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 18:42:24 smithi112 ceph-mds[65564]: -- 172.21.15.112:0/1200753923 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b497e31a40 con 0x55b497e04c00 2024-04-25T18:42:24.305 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 18:42:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-j[65559]: starting mds.j at 2024-04-25T18:42:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:24 smithi171 ceph-mon[33266]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:24 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/216225708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:24 smithi161 ceph-mon[32609]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:24 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/216225708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:24.697 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 18:42:24 smithi112 bash[65529]: 8bec2ea380494db5be8516275e6112b5d9788239006a0809fdb8552a3b640bf0 2024-04-25T18:42:24.698 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 18:42:24 smithi112 systemd[1]: Started Ceph mds.j for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:25.717 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:25.718 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:25.718 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:25.718 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:25.718 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:25.718 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:25.718 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:25 smithi112 ceph-mon[27864]: Deploying daemon mds.d on smithi112 2024-04-25T18:42:25.718 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:25 smithi112 ceph-mon[27864]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:25.718 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:25 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.112:6834/139408672,v1:172.21.15.112:6835/139408672] up:boot 2024-04-25T18:42:25.718 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:25 smithi112 ceph-mon[27864]: fsmap 3 up:standby 2024-04-25T18:42:25.718 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:25 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:42:25.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:25.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:25.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:25.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:25.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:25.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:25.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:25 smithi161 ceph-mon[32609]: Deploying daemon mds.d on smithi112 2024-04-25T18:42:25.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:25 smithi161 ceph-mon[32609]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:25.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:25 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.112:6834/139408672,v1:172.21.15.112:6835/139408672] up:boot 2024-04-25T18:42:25.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:25 smithi161 ceph-mon[32609]: fsmap 3 up:standby 2024-04-25T18:42:25.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:25 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:42:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:25 smithi171 ceph-mon[33266]: Deploying daemon mds.d on smithi112 2024-04-25T18:42:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:25 smithi171 ceph-mon[33266]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:25 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.112:6834/139408672,v1:172.21.15.112:6835/139408672] up:boot 2024-04-25T18:42:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:25 smithi171 ceph-mon[33266]: fsmap 3 up:standby 2024-04-25T18:42:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:25 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:42:26.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:42:27.192 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:42:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:28 smithi171 ceph-mon[33266]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:28 smithi161 ceph-mon[32609]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:28 smithi112 ceph-mon[27864]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:28.882 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 18:42:28 smithi112 podman[65947]: 2024-04-25 18:42:28.756753459 +0000 UTC m=+0.036656464 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:42:28.882 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 18:42:28 smithi112 podman[65947]: 2024-04-25 18:42:28.879630872 +0000 UTC m=+0.159533901 container create aa1c2907f74f8b0893f0e19d519d753a82d55b3f7d0c680bc38c9884493297ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:42:29.696 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 18:42:29 smithi112 podman[65947]: 2024-04-25 18:42:29.427143966 +0000 UTC m=+0.707046975 container init aa1c2907f74f8b0893f0e19d519d753a82d55b3f7d0c680bc38c9884493297ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T18:42:29.696 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 18:42:29 smithi112 podman[65947]: 2024-04-25 18:42:29.436544963 +0000 UTC m=+0.716447968 container start aa1c2907f74f8b0893f0e19d519d753a82d55b3f7d0c680bc38c9884493297ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T18:42:29.696 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 18:42:29 smithi112 ceph-mds[65990]: -- 172.21.15.112:0/325637882 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a7d4799a40 con 0x55a7d476cc00 2024-04-25T18:42:29.696 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 18:42:29 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-d[65980]: starting mds.d at 2024-04-25T18:42:29.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.720+0000 7f4444800700 1 -- 172.21.15.112:0/1550586498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443c075590 msgr2=0x7f443c10ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.720+0000 7f4444800700 1 --2- 172.21.15.112:0/1550586498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443c075590 0x7f443c10ef00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f442c009990 tx=0x7f442c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:42:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.720+0000 7f4444800700 1 -- 172.21.15.112:0/1550586498 shutdown_connections 2024-04-25T18:42:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.720+0000 7f4444800700 1 --2- 172.21.15.112:0/1550586498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c10f440 0x7f443c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.720+0000 7f4444800700 1 --2- 172.21.15.112:0/1550586498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f443c074c70 0x7f443c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.720+0000 7f4444800700 1 --2- 172.21.15.112:0/1550586498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443c075590 0x7f443c10ef00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.720+0000 7f4444800700 1 -- 172.21.15.112:0/1550586498 >> 172.21.15.112:0/1550586498 conn(0x7f443c079bf0 msgr2=0x7f443c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.720+0000 7f4444800700 1 -- 172.21.15.112:0/1550586498 shutdown_connections 2024-04-25T18:42:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.720+0000 7f4444800700 1 -- 172.21.15.112:0/1550586498 wait complete. 2024-04-25T18:42:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.721+0000 7f4444800700 1 Processor -- start 2024-04-25T18:42:29.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.721+0000 7f4444800700 1 -- start start 2024-04-25T18:42:29.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.722+0000 7f4444800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f443c074c70 0x7f443c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:29.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.722+0000 7f4444800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443c075590 0x7f443c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:29.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.722+0000 7f4444800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c10f440 0x7f443c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:29.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.722+0000 7f4444800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f443c114ed0 con 0x7f443c10f440 2024-04-25T18:42:29.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.722+0000 7f4444800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f443c114d50 con 0x7f443c075590 2024-04-25T18:42:29.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.722+0000 7f4444800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f443c115050 con 0x7f443c074c70 2024-04-25T18:42:29.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.722+0000 7f4442800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c10f440 0x7f443c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:29.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.722+0000 7f4442800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c10f440 0x7f443c19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47628/0 (socket says 172.21.15.112:47628) 2024-04-25T18:42:29.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.722+0000 7f4442800700 1 -- 172.21.15.112:0/3938664830 learned_addr learned my addr 172.21.15.112:0/3938664830 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:42:29.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.722+0000 7f4441e00700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f443c074c70 0x7f443c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:29.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.722+0000 7f4441400700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443c075590 0x7f443c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.723+0000 7f4441e00700 1 -- 172.21.15.112:0/3938664830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443c075590 msgr2=0x7f443c19b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.723+0000 7f4441e00700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443c075590 0x7f443c19b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.723+0000 7f4441e00700 1 -- 172.21.15.112:0/3938664830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c10f440 msgr2=0x7f443c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.723+0000 7f4441e00700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c10f440 0x7f443c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.723+0000 7f4441e00700 1 -- 172.21.15.112:0/3938664830 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c1a0410 con 0x7f443c074c70 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.723+0000 7f4442800700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c10f440 0x7f443c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.723+0000 7f4441400700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443c075590 0x7f443c19b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.723+0000 7f4441e00700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f443c074c70 0x7f443c19b400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f443000e9f0 tx=0x7f443000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.723+0000 7f4436a00700 1 -- 172.21.15.112:0/3938664830 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443000c9c0 con 0x7f443c074c70 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.723+0000 7f4436a00700 1 -- 172.21.15.112:0/3938664830 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f443000cb60 con 0x7f443c074c70 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.724+0000 7f4444800700 1 -- 172.21.15.112:0/3938664830 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c1a0700 con 0x7f443c074c70 2024-04-25T18:42:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.724+0000 7f4436a00700 1 -- 172.21.15.112:0/3938664830 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430018680 con 0x7f443c074c70 2024-04-25T18:42:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.724+0000 7f4444800700 1 -- 172.21.15.112:0/3938664830 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c1a7f80 con 0x7f443c074c70 2024-04-25T18:42:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.725+0000 7f4444800700 1 -- 172.21.15.112:0/3938664830 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443c069730 con 0x7f443c074c70 2024-04-25T18:42:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.727+0000 7f4436a00700 1 -- 172.21.15.112:0/3938664830 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4430010430 con 0x7f443c074c70 2024-04-25T18:42:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.728+0000 7f4436a00700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f44240a3cc0 0x7f44240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.728+0000 7f4436a00700 1 -- 172.21.15.112:0/3938664830 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f44300d01f0 con 0x7f443c074c70 2024-04-25T18:42:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.729+0000 7f4441400700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f44240a3cc0 0x7f44240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.729+0000 7f4441400700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f44240a3cc0 0x7f44240a6180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f443c111140 tx=0x7f442c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:29.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:29.730+0000 7f4436a00700 1 -- 172.21.15.112:0/3938664830 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f443009cfd0 con 0x7f443c074c70 2024-04-25T18:42:29.974 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 18:42:29 smithi112 bash[65947]: aa1c2907f74f8b0893f0e19d519d753a82d55b3f7d0c680bc38c9884493297ff 2024-04-25T18:42:29.974 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 18:42:29 smithi112 systemd[1]: Started Ceph mds.d for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.025+0000 7f4444800700 1 -- 172.21.15.112:0/3938664830 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f443c02df80 con 0x7f443c074c70 2024-04-25T18:42:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.026+0000 7f4436a00700 1 -- 172.21.15.112:0/3938664830 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f443009c8d0 con 0x7f443c074c70 2024-04-25T18:42:30.027 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:42:30.027 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-25T18:42:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 -- 172.21.15.112:0/3938664830 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f44240a3cc0 msgr2=0x7f44240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f44240a3cc0 0x7f44240a6180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f443c111140 tx=0x7f442c01a040 comp rx=0 tx=0).stop 2024-04-25T18:42:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 -- 172.21.15.112:0/3938664830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f443c074c70 msgr2=0x7f443c19b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f443c074c70 0x7f443c19b400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f443000e9f0 tx=0x7f443000ed00 comp rx=0 tx=0).stop 2024-04-25T18:42:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 -- 172.21.15.112:0/3938664830 shutdown_connections 2024-04-25T18:42:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f44240a3cc0 0x7f44240a6180 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f443c10f440 0x7f443c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f443c074c70 0x7f443c19b400 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 --2- 172.21.15.112:0/3938664830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f443c075590 0x7f443c19b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:30.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 -- 172.21.15.112:0/3938664830 >> 172.21.15.112:0/3938664830 conn(0x7f443c079bf0 msgr2=0x7f443c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:30.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 -- 172.21.15.112:0/3938664830 shutdown_connections 2024-04-25T18:42:30.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:30.030+0000 7f4444800700 1 -- 172.21.15.112:0/3938664830 wait complete. 2024-04-25T18:42:30.032 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 4 2024-04-25T18:42:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:30 smithi171 ceph-mon[33266]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:30 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:30 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:30 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:30 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:30 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:30 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:30 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3938664830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:30.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:30 smithi112 ceph-mon[27864]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:30.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:30.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:30.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:30.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:30.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:30.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:30 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:30.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:30 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3938664830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:30 smithi161 ceph-mon[32609]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:30 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:30 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:30 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:30 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:30.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:30 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:30.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:30 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:30.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:30 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3938664830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:31 smithi171 ceph-mon[33266]: Deploying daemon mds.e on smithi161 2024-04-25T18:42:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:31 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.112:6836/2558651587,v1:172.21.15.112:6837/2558651587] up:boot 2024-04-25T18:42:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:31 smithi171 ceph-mon[33266]: fsmap 4 up:standby 2024-04-25T18:42:31.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:31 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:42:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:31 smithi161 ceph-mon[32609]: Deploying daemon mds.e on smithi161 2024-04-25T18:42:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:31 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.112:6836/2558651587,v1:172.21.15.112:6837/2558651587] up:boot 2024-04-25T18:42:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:31 smithi161 ceph-mon[32609]: fsmap 4 up:standby 2024-04-25T18:42:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:31 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:42:31.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:31 smithi112 ceph-mon[27864]: Deploying daemon mds.e on smithi161 2024-04-25T18:42:31.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:31 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.112:6836/2558651587,v1:172.21.15.112:6837/2558651587] up:boot 2024-04-25T18:42:31.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:31 smithi112 ceph-mon[27864]: fsmap 4 up:standby 2024-04-25T18:42:31.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:31 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:42:32.162 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 18:42:32 smithi161 systemd[1]: Starting Ceph mds.e for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:42:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:32 smithi171 ceph-mon[33266]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:32.443 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:32 smithi161 ceph-mon[32609]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:32.444 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 18:42:32 smithi161 podman[62182]: 2024-04-25 18:42:32.215695187 +0000 UTC m=+0.112493700 container create 4b70ac893a33dd0fee57803a8e6d0e303025d075144f14651314b3b9060340c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T18:42:32.444 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 18:42:32 smithi161 podman[62182]: 2024-04-25 18:42:32.121155026 +0000 UTC m=+0.017953539 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:42:32.444 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 18:42:32 smithi161 podman[62182]: 2024-04-25 18:42:32.443293242 +0000 UTC m=+0.340091756 container init 4b70ac893a33dd0fee57803a8e6d0e303025d075144f14651314b3b9060340c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:42:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:32 smithi112 ceph-mon[27864]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:32.721 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 18:42:32 smithi161 podman[62182]: 2024-04-25 18:42:32.445879359 +0000 UTC m=+0.342677872 container start 4b70ac893a33dd0fee57803a8e6d0e303025d075144f14651314b3b9060340c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T18:42:32.721 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 18:42:32 smithi161 ceph-mds[62201]: -- 172.21.15.161:0/2363040263 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5588402e7a40 con 0x5588402ba800 2024-04-25T18:42:32.721 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 18:42:32 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-e[62197]: starting mds.e at 2024-04-25T18:42:32.721 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 18:42:32 smithi161 bash[62182]: 4b70ac893a33dd0fee57803a8e6d0e303025d075144f14651314b3b9060340c2 2024-04-25T18:42:32.722 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 18:42:32 smithi161 systemd[1]: Started Ceph mds.e for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:32.929 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:42:33.376 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:42:33.881 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:33 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:33.881 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:33 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:33.881 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:33 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:33.882 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:33 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:33.882 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:33 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:33.882 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:33 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:33.882 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:33 smithi112 ceph-mon[27864]: Deploying daemon mds.f on smithi171 2024-04-25T18:42:33.882 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:33 smithi112 ceph-mon[27864]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:33.882 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:33 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.161:6834/3751012536,v1:172.21.15.161:6835/3751012536] up:boot 2024-04-25T18:42:33.882 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:33 smithi112 ceph-mon[27864]: fsmap 5 up:standby 2024-04-25T18:42:33.882 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:33 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:42:33.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:33 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:33.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:33 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:33.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:33 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:33.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:33 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:33.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:33 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:33.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:33 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:33.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:33 smithi171 ceph-mon[33266]: Deploying daemon mds.f on smithi171 2024-04-25T18:42:33.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:33 smithi171 ceph-mon[33266]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:33.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:33 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.161:6834/3751012536,v1:172.21.15.161:6835/3751012536] up:boot 2024-04-25T18:42:33.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:33 smithi171 ceph-mon[33266]: fsmap 5 up:standby 2024-04-25T18:42:33.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:33 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:42:33.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:33 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:33.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:33 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:33.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:33 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:33.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:33 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:33.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:33 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:33.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:33 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:33.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:33 smithi161 ceph-mon[32609]: Deploying daemon mds.f on smithi171 2024-04-25T18:42:33.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:33 smithi161 ceph-mon[32609]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:33.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:33 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.161:6834/3751012536,v1:172.21.15.161:6835/3751012536] up:boot 2024-04-25T18:42:33.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:33 smithi161 ceph-mon[32609]: fsmap 5 up:standby 2024-04-25T18:42:33.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:33 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:42:34.933 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 18:42:34 smithi171 systemd[1]: Starting Ceph mds.f for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:42:35.260 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 18:42:35 smithi171 podman[63343]: 2024-04-25 18:42:34.907504493 +0000 UTC m=+0.017753840 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:42:35.260 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 18:42:35 smithi171 podman[63343]: 2024-04-25 18:42:35.022886489 +0000 UTC m=+0.133135874 container create 36603a081b1af301bc4947de505436c212dced7a0744487be60ce1ce2782cac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T18:42:35.260 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 18:42:35 smithi171 podman[63343]: 2024-04-25 18:42:35.233384449 +0000 UTC m=+0.343633801 container init 36603a081b1af301bc4947de505436c212dced7a0744487be60ce1ce2782cac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T18:42:35.260 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 18:42:35 smithi171 podman[63343]: 2024-04-25 18:42:35.236046393 +0000 UTC m=+0.346295736 container start 36603a081b1af301bc4947de505436c212dced7a0744487be60ce1ce2782cac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T18:42:35.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.645+0000 7f87a5a00700 1 -- 172.21.15.112:0/3756506425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0075590 msgr2=0x7f87a01136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:35.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.645+0000 7f87a5a00700 1 --2- 172.21.15.112:0/3756506425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0075590 0x7f87a01136d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f878c009990 tx=0x7f878c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:42:35.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.645+0000 7f87a5a00700 1 -- 172.21.15.112:0/3756506425 shutdown_connections 2024-04-25T18:42:35.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.645+0000 7f87a5a00700 1 --2- 172.21.15.112:0/3756506425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87a0113c10 0x7f87a0116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:35.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.645+0000 7f87a5a00700 1 --2- 172.21.15.112:0/3756506425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87a0074c70 0x7f87a0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:35.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.645+0000 7f87a5a00700 1 --2- 172.21.15.112:0/3756506425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0075590 0x7f87a01136d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:35.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.645+0000 7f87a5a00700 1 -- 172.21.15.112:0/3756506425 >> 172.21.15.112:0/3756506425 conn(0x7f87a0079ba0 msgr2=0x7f87a0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:35.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.646+0000 7f87a5a00700 1 -- 172.21.15.112:0/3756506425 shutdown_connections 2024-04-25T18:42:35.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.646+0000 7f87a5a00700 1 -- 172.21.15.112:0/3756506425 wait complete. 2024-04-25T18:42:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.647+0000 7f87a5a00700 1 Processor -- start 2024-04-25T18:42:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.647+0000 7f87a5a00700 1 -- start start 2024-04-25T18:42:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.647+0000 7f87a5a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87a0074c70 0x7f87a01a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.647+0000 7f87a5a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0075590 0x7f87a01a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.647+0000 7f87a5a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87a0113c10 0x7f87a019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.647+0000 7f87a5a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0119270 con 0x7f87a0075590 2024-04-25T18:42:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.647+0000 7f87a5a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01190f0 con 0x7f87a0074c70 2024-04-25T18:42:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.647+0000 7f87a5a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01193f0 con 0x7f87a0113c10 2024-04-25T18:42:35.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.647+0000 7f879ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87a0074c70 0x7f87a01a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.648+0000 7f879ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87a0074c70 0x7f87a01a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:33360/0 (socket says 172.21.15.112:33360) 2024-04-25T18:42:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.648+0000 7f879ea00700 1 -- 172.21.15.112:0/1122218169 learned_addr learned my addr 172.21.15.112:0/1122218169 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:42:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.648+0000 7f879f400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87a0113c10 0x7f87a019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.648+0000 7f879e000700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0075590 0x7f87a01a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.648+0000 7f879ea00700 1 -- 172.21.15.112:0/1122218169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87a0113c10 msgr2=0x7f87a019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.648+0000 7f879ea00700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87a0113c10 0x7f87a019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.648+0000 7f879ea00700 1 -- 172.21.15.112:0/1122218169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0075590 msgr2=0x7f87a01a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.648+0000 7f879ea00700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0075590 0x7f87a01a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.648+0000 7f879ea00700 1 -- 172.21.15.112:0/1122218169 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a019fcf0 con 0x7f87a0074c70 2024-04-25T18:42:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.648+0000 7f879e000700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0075590 0x7f87a01a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:42:35.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.648+0000 7f879ea00700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87a0074c70 0x7f87a01a53d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f879000e9a0 tx=0x7f879000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.649+0000 7f8797400700 1 -- 172.21.15.112:0/1122218169 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879000cbd0 con 0x7f87a0074c70 2024-04-25T18:42:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.649+0000 7f8797400700 1 -- 172.21.15.112:0/1122218169 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f879000cd70 con 0x7f87a0074c70 2024-04-25T18:42:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.649+0000 7f8797400700 1 -- 172.21.15.112:0/1122218169 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790018680 con 0x7f87a0074c70 2024-04-25T18:42:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.649+0000 7f87a5a00700 1 -- 172.21.15.112:0/1122218169 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a019ffe0 con 0x7f87a0074c70 2024-04-25T18:42:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.649+0000 7f87a5a00700 1 -- 172.21.15.112:0/1122218169 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a01ac2a0 con 0x7f87a0074c70 2024-04-25T18:42:35.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.650+0000 7f87a5a00700 1 -- 172.21.15.112:0/1122218169 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a0069730 con 0x7f87a0074c70 2024-04-25T18:42:35.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.652+0000 7f8797400700 1 -- 172.21.15.112:0/1122218169 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8790010430 con 0x7f87a0074c70 2024-04-25T18:42:35.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.654+0000 7f8797400700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f87840a3cc0 0x7f87840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:35.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.654+0000 7f8797400700 1 -- 172.21.15.112:0/1122218169 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f87900d0670 con 0x7f87a0074c70 2024-04-25T18:42:35.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.654+0000 7f879e000700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f87840a3cc0 0x7f87840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:35.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.654+0000 7f879e000700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f87840a3cc0 0x7f87840a6180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f878c006010 tx=0x7f878c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:35.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.656+0000 7f8797400700 1 -- 172.21.15.112:0/1122218169 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f879009d450 con 0x7f87a0074c70 2024-04-25T18:42:35.682 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 18:42:35 smithi171 ceph-mds[63362]: -- 172.21.15.171:0/3324613472 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd5e33fa40 con 0x55fd5e313000 2024-04-25T18:42:35.682 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 18:42:35 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-f[63358]: starting mds.f at 2024-04-25T18:42:35.682 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 18:42:35 smithi171 bash[63343]: 36603a081b1af301bc4947de505436c212dced7a0744487be60ce1ce2782cac0 2024-04-25T18:42:35.682 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 18:42:35 smithi171 systemd[1]: Started Ceph mds.f for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:35.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.928+0000 7f87a5a00700 1 -- 172.21.15.112:0/1122218169 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87a01a0ac0 con 0x7f87a0074c70 2024-04-25T18:42:35.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.929+0000 7f8797400700 1 -- 172.21.15.112:0/1122218169 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7f879009cd50 con 0x7f87a0074c70 2024-04-25T18:42:35.930 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:42:35.930 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T18:42:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.932+0000 7f87a5a00700 1 -- 172.21.15.112:0/1122218169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f87840a3cc0 msgr2=0x7f87840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.932+0000 7f87a5a00700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f87840a3cc0 0x7f87840a6180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f878c006010 tx=0x7f878c01a040 comp rx=0 tx=0).stop 2024-04-25T18:42:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.932+0000 7f87a5a00700 1 -- 172.21.15.112:0/1122218169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87a0074c70 msgr2=0x7f87a01a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.932+0000 7f87a5a00700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87a0074c70 0x7f87a01a53d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f879000e9a0 tx=0x7f879000ecb0 comp rx=0 tx=0).stop 2024-04-25T18:42:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.932+0000 7f87a5a00700 1 -- 172.21.15.112:0/1122218169 shutdown_connections 2024-04-25T18:42:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.932+0000 7f87a5a00700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f87840a3cc0 0x7f87840a6180 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.932+0000 7f87a5a00700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f87a0113c10 0x7f87a019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.932+0000 7f87a5a00700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87a0074c70 0x7f87a01a53d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.932+0000 7f87a5a00700 1 --2- 172.21.15.112:0/1122218169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a0075590 0x7f87a01a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.932+0000 7f87a5a00700 1 -- 172.21.15.112:0/1122218169 >> 172.21.15.112:0/1122218169 conn(0x7f87a0079ba0 msgr2=0x7f87a010f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.932+0000 7f87a5a00700 1 -- 172.21.15.112:0/1122218169 shutdown_connections 2024-04-25T18:42:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:35.933+0000 7f87a5a00700 1 -- 172.21.15.112:0/1122218169 wait complete. 2024-04-25T18:42:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 6 2024-04-25T18:42:36.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:36 smithi112 ceph-mon[27864]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:36.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:36.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:36.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:36.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:36.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:36.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:36 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:36.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:36 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1122218169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:36 smithi171 ceph-mon[33266]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:36.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:36.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:36.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:36 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:36.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:36 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1122218169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:36 smithi161 ceph-mon[32609]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:36.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:36.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:36 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:36.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:36 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1122218169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:37.275 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:37 smithi171 ceph-mon[33266]: Deploying daemon mds.c on smithi171 2024-04-25T18:42:37.275 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:37 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.171:6834/2423338358,v1:172.21.15.171:6835/2423338358] up:boot 2024-04-25T18:42:37.275 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:37 smithi171 ceph-mon[33266]: fsmap 6 up:standby 2024-04-25T18:42:37.275 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:37 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:42:37.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:37 smithi161 ceph-mon[32609]: Deploying daemon mds.c on smithi171 2024-04-25T18:42:37.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:37 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.171:6834/2423338358,v1:172.21.15.171:6835/2423338358] up:boot 2024-04-25T18:42:37.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:37 smithi161 ceph-mon[32609]: fsmap 6 up:standby 2024-04-25T18:42:37.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:37 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:42:37.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:37 smithi112 ceph-mon[27864]: Deploying daemon mds.c on smithi171 2024-04-25T18:42:37.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:37 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.171:6834/2423338358,v1:172.21.15.171:6835/2423338358] up:boot 2024-04-25T18:42:37.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:37 smithi112 ceph-mon[27864]: fsmap 6 up:standby 2024-04-25T18:42:37.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:37 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:42:37.932 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:42:37 smithi171 systemd[1]: Starting Ceph mds.c for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:42:38.254 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:38 smithi171 ceph-mon[33266]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:38.254 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:42:37 smithi171 podman[63655]: 2024-04-25 18:42:37.87040467 +0000 UTC m=+0.018201425 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:42:38.254 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:42:37 smithi171 podman[63655]: 2024-04-25 18:42:37.993394696 +0000 UTC m=+0.141191422 container create 5e7c956c31a12f78536dd8c34fe13e62097d75e5683045f49778579c342db605 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:42:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:38 smithi161 ceph-mon[32609]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:38 smithi112 ceph-mon[27864]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:38.682 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:42:38 smithi171 podman[63655]: 2024-04-25 18:42:38.253948765 +0000 UTC m=+0.401745496 container init 5e7c956c31a12f78536dd8c34fe13e62097d75e5683045f49778579c342db605 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:42:38.682 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:42:38 smithi171 podman[63655]: 2024-04-25 18:42:38.256771741 +0000 UTC m=+0.404568477 container start 5e7c956c31a12f78536dd8c34fe13e62097d75e5683045f49778579c342db605 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:42:38.682 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:42:38 smithi171 ceph-mds[63674]: -- 172.21.15.171:0/1875797776 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55612a7a9a40 con 0x55612a77cc00 2024-04-25T18:42:38.682 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:42:38 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c[63670]: starting mds.c at 2024-04-25T18:42:38.682 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:42:38 smithi171 bash[63655]: 5e7c956c31a12f78536dd8c34fe13e62097d75e5683045f49778579c342db605 2024-04-25T18:42:38.683 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:42:38 smithi171 systemd[1]: Started Ceph mds.c for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:38.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:42:39.252 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:42:39.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:39.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:39.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:39.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:39.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:39.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:39.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:39 smithi112 ceph-mon[27864]: Deploying daemon mds.h on smithi161 2024-04-25T18:42:39.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:39 smithi112 ceph-mon[27864]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:39.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:39 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.171:6836/3522712423,v1:172.21.15.171:6837/3522712423] up:boot 2024-04-25T18:42:39.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:39 smithi112 ceph-mon[27864]: fsmap 7 up:standby 2024-04-25T18:42:39.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:39 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:42:39.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:39.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:39.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:39.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:39.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:39.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:39.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:39 smithi171 ceph-mon[33266]: Deploying daemon mds.h on smithi161 2024-04-25T18:42:39.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:39 smithi171 ceph-mon[33266]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:39.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:39 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.171:6836/3522712423,v1:172.21.15.171:6837/3522712423] up:boot 2024-04-25T18:42:39.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:39 smithi171 ceph-mon[33266]: fsmap 7 up:standby 2024-04-25T18:42:39.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:39 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:42:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:39.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:39.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:39.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:39.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:39 smithi161 ceph-mon[32609]: Deploying daemon mds.h on smithi161 2024-04-25T18:42:39.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:39 smithi161 ceph-mon[32609]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:39.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:39 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.171:6836/3522712423,v1:172.21.15.171:6837/3522712423] up:boot 2024-04-25T18:42:39.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:39 smithi161 ceph-mon[32609]: fsmap 7 up:standby 2024-04-25T18:42:39.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:39 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:42:40.722 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:42:40 smithi161 systemd[1]: Starting Ceph mds.h for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:42:41.004 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:42:40 smithi161 podman[62494]: 2024-04-25 18:42:40.726794714 +0000 UTC m=+0.110001722 container create 929ece4abbccd3e617fb90e43c560e82362aeea00b07d5ce213698b6dcf9c297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T18:42:41.004 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:42:40 smithi161 podman[62494]: 2024-04-25 18:42:40.634900207 +0000 UTC m=+0.018107231 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:42:41.005 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:42:40 smithi161 podman[62494]: 2024-04-25 18:42:40.920750455 +0000 UTC m=+0.303957466 container init 929ece4abbccd3e617fb90e43c560e82362aeea00b07d5ce213698b6dcf9c297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:42:41.005 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:42:40 smithi161 podman[62494]: 2024-04-25 18:42:40.923284051 +0000 UTC m=+0.306491066 container start 929ece4abbccd3e617fb90e43c560e82362aeea00b07d5ce213698b6dcf9c297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T18:42:41.005 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:42:40 smithi161 ceph-mds[62515]: -- 172.21.15.161:0/2109828342 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d458b77a40 con 0x55d458b4a800 2024-04-25T18:42:41.005 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:42:40 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h[62511]: starting mds.h at 2024-04-25T18:42:41.005 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:42:41 smithi161 bash[62494]: 929ece4abbccd3e617fb90e43c560e82362aeea00b07d5ce213698b6dcf9c297 2024-04-25T18:42:41.471 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:42:41 smithi161 systemd[1]: Started Ceph mds.h for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:41.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.642+0000 7f1e73200700 1 -- 172.21.15.112:0/4215391666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e6c10e3b0 msgr2=0x7f1e6c115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:41.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.642+0000 7f1e73200700 1 --2- 172.21.15.112:0/4215391666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e6c10e3b0 0x7f1e6c115f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400b0d0 tx=0x7f1e6400b3e0 comp rx=0 tx=0).stop 2024-04-25T18:42:41.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.642+0000 7f1e73200700 1 -- 172.21.15.112:0/4215391666 shutdown_connections 2024-04-25T18:42:41.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.642+0000 7f1e73200700 1 --2- 172.21.15.112:0/4215391666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e6c10e3b0 0x7f1e6c115f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:41.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.642+0000 7f1e73200700 1 --2- 172.21.15.112:0/4215391666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6c1092d0 0x7f1e6c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:41.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.642+0000 7f1e73200700 1 --2- 172.21.15.112:0/4215391666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6c109c80 0x7f1e6c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:41.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.642+0000 7f1e73200700 1 -- 172.21.15.112:0/4215391666 >> 172.21.15.112:0/4215391666 conn(0x7f1e6c076210 msgr2=0x7f1e6c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:41.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.642+0000 7f1e73200700 1 -- 172.21.15.112:0/4215391666 shutdown_connections 2024-04-25T18:42:41.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.642+0000 7f1e73200700 1 -- 172.21.15.112:0/4215391666 wait complete. 2024-04-25T18:42:41.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.643+0000 7f1e73200700 1 Processor -- start 2024-04-25T18:42:41.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.643+0000 7f1e73200700 1 -- start start 2024-04-25T18:42:41.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.644+0000 7f1e73200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6c1092d0 0x7f1e6c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:41.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.644+0000 7f1e73200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6c109c80 0x7f1e6c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.644+0000 7f1e73200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e6c10e3b0 0x7f1e6c1a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.644+0000 7f1e73200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c1192a0 con 0x7f1e6c1092d0 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.644+0000 7f1e73200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c119120 con 0x7f1e6c109c80 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.644+0000 7f1e73200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c119420 con 0x7f1e6c10e3b0 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.644+0000 7f1e6be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6c1092d0 0x7f1e6c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.644+0000 7f1e71200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e6c10e3b0 0x7f1e6c1a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.644+0000 7f1e6be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6c1092d0 0x7f1e6c19f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56920/0 (socket says 172.21.15.112:56920) 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.644+0000 7f1e6be00700 1 -- 172.21.15.112:0/1405591082 learned_addr learned my addr 172.21.15.112:0/1405591082 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.644+0000 7f1e63e00700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6c109c80 0x7f1e6c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e6be00700 1 -- 172.21.15.112:0/1405591082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e6c10e3b0 msgr2=0x7f1e6c1a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e6be00700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e6c10e3b0 0x7f1e6c1a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e6be00700 1 -- 172.21.15.112:0/1405591082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6c109c80 msgr2=0x7f1e6c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e6be00700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6c109c80 0x7f1e6c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e6be00700 1 -- 172.21.15.112:0/1405591082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c1a4740 con 0x7f1e6c1092d0 2024-04-25T18:42:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e71200700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e6c10e3b0 0x7f1e6c1a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:42:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e6be00700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6c1092d0 0x7f1e6c19f730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00e9a0 tx=0x7f1e5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e69600700 1 -- 172.21.15.112:0/1405591082 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c00ca10 con 0x7f1e6c1092d0 2024-04-25T18:42:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e73200700 1 -- 172.21.15.112:0/1405591082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c1a4a30 con 0x7f1e6c1092d0 2024-04-25T18:42:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e69600700 1 -- 172.21.15.112:0/1405591082 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e5c00cbb0 con 0x7f1e6c1092d0 2024-04-25T18:42:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e69600700 1 -- 172.21.15.112:0/1405591082 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c018680 con 0x7f1e6c1092d0 2024-04-25T18:42:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.645+0000 7f1e73200700 1 -- 172.21.15.112:0/1405591082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c1ac2b0 con 0x7f1e6c1092d0 2024-04-25T18:42:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.646+0000 7f1e73200700 1 -- 172.21.15.112:0/1405591082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6c069730 con 0x7f1e6c1092d0 2024-04-25T18:42:41.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.648+0000 7f1e69600700 1 -- 172.21.15.112:0/1405591082 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1e5c010430 con 0x7f1e6c1092d0 2024-04-25T18:42:41.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.650+0000 7f1e69600700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e500a3cc0 0x7f1e500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:41.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.650+0000 7f1e69600700 1 -- 172.21.15.112:0/1405591082 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f1e5c0d0410 con 0x7f1e6c1092d0 2024-04-25T18:42:41.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.650+0000 7f1e63e00700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e500a3cc0 0x7f1e500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:41.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.651+0000 7f1e63e00700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e500a3cc0 0x7f1e500a6180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c10f6c0 tx=0x7f1e54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:41.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.653+0000 7f1e69600700 1 -- 172.21.15.112:0/1405591082 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e5c09d1f0 con 0x7f1e6c1092d0 2024-04-25T18:42:41.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.926+0000 7f1e73200700 1 -- 172.21.15.112:0/1405591082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e6c1a0a90 con 0x7f1e6c1092d0 2024-04-25T18:42:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.927+0000 7f1e69600700 1 -- 172.21.15.112:0/1405591082 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7189 (secure 0 0 0) 0x7f1e5c09caf0 con 0x7f1e6c1092d0 2024-04-25T18:42:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:42:41.928 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T18:42:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 -- 172.21.15.112:0/1405591082 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e500a3cc0 msgr2=0x7f1e500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e500a3cc0 0x7f1e500a6180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c10f6c0 tx=0x7f1e54009260 comp rx=0 tx=0).stop 2024-04-25T18:42:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 -- 172.21.15.112:0/1405591082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6c1092d0 msgr2=0x7f1e6c19f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6c1092d0 0x7f1e6c19f730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00e9a0 tx=0x7f1e5c00ecb0 comp rx=0 tx=0).stop 2024-04-25T18:42:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 -- 172.21.15.112:0/1405591082 shutdown_connections 2024-04-25T18:42:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e500a3cc0 0x7f1e500a6180 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e6c10e3b0 0x7f1e6c1a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e6c1092d0 0x7f1e6c19f730 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 --2- 172.21.15.112:0/1405591082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6c109c80 0x7f1e6c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 -- 172.21.15.112:0/1405591082 >> 172.21.15.112:0/1405591082 conn(0x7f1e6c076210 msgr2=0x7f1e6c0767f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:41.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 -- 172.21.15.112:0/1405591082 shutdown_connections 2024-04-25T18:42:41.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:41.931+0000 7f1e73200700 1 -- 172.21.15.112:0/1405591082 wait complete. 2024-04-25T18:42:41.933 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-04-25T18:42:42.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.161:6836/4007863322,v1:172.21.15.161:6837/4007863322] up:boot 2024-04-25T18:42:42.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: fsmap 8 up:standby 2024-04-25T18:42:42.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:42:42.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:42.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:42.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:42.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:42.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:42.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:42.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:42.118 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: Deploying daemon mds.i on smithi171 2024-04-25T18:42:42.118 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:42 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1405591082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:42.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.161:6836/4007863322,v1:172.21.15.161:6837/4007863322] up:boot 2024-04-25T18:42:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: fsmap 8 up:standby 2024-04-25T18:42:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:42:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:42.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:42.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: Deploying daemon mds.i on smithi171 2024-04-25T18:42:42.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:42 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1405591082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:42.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.161:6836/4007863322,v1:172.21.15.161:6837/4007863322] up:boot 2024-04-25T18:42:42.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: fsmap 8 up:standby 2024-04-25T18:42:42.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:42:42.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: Deploying daemon mds.i on smithi171 2024-04-25T18:42:42.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:42 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1405591082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:43.647 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:42:43 smithi171 systemd[1]: Starting Ceph mds.i for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:42:43.932 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:42:43 smithi171 podman[63975]: 2024-04-25 18:42:43.549902953 +0000 UTC m=+0.018176679 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:42:43.933 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:42:43 smithi171 podman[63975]: 2024-04-25 18:42:43.66396748 +0000 UTC m=+0.132241239 container create 87941493cbf977f657c19588b4198a67c31e6063058d79ec257e08585603aeff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:42:43.933 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:42:43 smithi171 podman[63975]: 2024-04-25 18:42:43.857892081 +0000 UTC m=+0.326165798 container init 87941493cbf977f657c19588b4198a67c31e6063058d79ec257e08585603aeff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T18:42:43.933 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:42:43 smithi171 podman[63975]: 2024-04-25 18:42:43.860453866 +0000 UTC m=+0.328727583 container start 87941493cbf977f657c19588b4198a67c31e6063058d79ec257e08585603aeff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:42:43.933 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:42:43 smithi171 ceph-mds[63994]: -- 172.21.15.171:0/453779945 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5601af1c5a40 con 0x5601af199000 2024-04-25T18:42:43.933 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:42:43 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i[63990]: starting mds.i at 2024-04-25T18:42:44.432 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:42:43 smithi171 bash[63975]: 87941493cbf977f657c19588b4198a67c31e6063058d79ec257e08585603aeff 2024-04-25T18:42:44.432 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:42:43 smithi171 systemd[1]: Started Ceph mds.i for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:44 smithi171 ceph-mon[33266]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:44 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:44.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:44 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:44.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:44 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:44.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:44 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:44.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:44 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:44.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:44 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:44 smithi161 ceph-mon[32609]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:44 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:44 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:44 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:44.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:44 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:44.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:44 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:44.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:44 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:44.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:44 smithi112 ceph-mon[27864]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:44 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:44 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:44 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:44 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:44 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:44 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:44.952 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:42:45.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:45 smithi112 ceph-mon[27864]: Deploying daemon mds.g on smithi112 2024-04-25T18:42:45.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:45 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] up:boot 2024-04-25T18:42:45.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:45 smithi112 ceph-mon[27864]: fsmap 9 up:standby 2024-04-25T18:42:45.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:45 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:42:45.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:45 smithi171 ceph-mon[33266]: Deploying daemon mds.g on smithi112 2024-04-25T18:42:45.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:45 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] up:boot 2024-04-25T18:42:45.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:45 smithi171 ceph-mon[33266]: fsmap 9 up:standby 2024-04-25T18:42:45.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:45 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:42:45.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:45 smithi161 ceph-mon[32609]: Deploying daemon mds.g on smithi112 2024-04-25T18:42:45.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:45 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] up:boot 2024-04-25T18:42:45.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:45 smithi161 ceph-mon[32609]: fsmap 9 up:standby 2024-04-25T18:42:45.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:45 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:42:46.404 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:46 smithi112 ceph-mon[27864]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:46 smithi171 ceph-mon[33266]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:46 smithi161 ceph-mon[32609]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:46.519 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:42:47.474 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:42:47 smithi112 systemd[1]: Starting Ceph mds.g for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:42:48.206 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:48 smithi112 ceph-mon[27864]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:48 smithi171 ceph-mon[33266]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:48.460 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:42:48 smithi112 podman[66730]: 2024-04-25 18:42:48.11072702 +0000 UTC m=+0.035383006 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:42:48.460 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:42:48 smithi112 podman[66730]: 2024-04-25 18:42:48.23102539 +0000 UTC m=+0.155681413 container create 0135fb6acc076f5b567cf23eda5f7ac428febbd4c66c8bc3c72f5bd40cf4cb1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T18:42:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:48 smithi161 ceph-mon[32609]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:48.973 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:42:48 smithi112 podman[66730]: 2024-04-25 18:42:48.862291051 +0000 UTC m=+0.786947058 container init 0135fb6acc076f5b567cf23eda5f7ac428febbd4c66c8bc3c72f5bd40cf4cb1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T18:42:48.974 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:42:48 smithi112 podman[66730]: 2024-04-25 18:42:48.872546379 +0000 UTC m=+0.797202409 container start 0135fb6acc076f5b567cf23eda5f7ac428febbd4c66c8bc3c72f5bd40cf4cb1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:42:48.974 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:42:48 smithi112 ceph-mds[66770]: -- 172.21.15.112:0/902204275 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cabba47a40 con 0x55cabba1b000 2024-04-25T18:42:48.974 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:42:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: starting mds.g at 2024-04-25T18:42:48.974 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:42:48 smithi112 bash[66730]: 0135fb6acc076f5b567cf23eda5f7ac428febbd4c66c8bc3c72f5bd40cf4cb1b 2024-04-25T18:42:48.974 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:42:48 smithi112 systemd[1]: Started Ceph mds.g for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:49.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.271+0000 7f4376600700 1 -- 172.21.15.112:0/1232273835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4370108a50 msgr2=0x7f4370108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:49.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.271+0000 7f4376600700 1 --2- 172.21.15.112:0/1232273835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4370108a50 0x7f4370108e30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f43600098f0 tx=0x7f4360009c00 comp rx=0 tx=0).stop 2024-04-25T18:42:49.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.272+0000 7f4376600700 1 -- 172.21.15.112:0/1232273835 shutdown_connections 2024-04-25T18:42:49.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.272+0000 7f4376600700 1 --2- 172.21.15.112:0/1232273835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4370103450 0x7f437010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:49.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.272+0000 7f4376600700 1 --2- 172.21.15.112:0/1232273835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4370108a50 0x7f4370108e30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:49.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.272+0000 7f4376600700 1 --2- 172.21.15.112:0/1232273835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4370102a90 0x7f4370102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:49.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.272+0000 7f4376600700 1 -- 172.21.15.112:0/1232273835 >> 172.21.15.112:0/1232273835 conn(0x7f4370079ba0 msgr2=0x7f4370079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:49.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.272+0000 7f4376600700 1 -- 172.21.15.112:0/1232273835 shutdown_connections 2024-04-25T18:42:49.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.272+0000 7f4376600700 1 -- 172.21.15.112:0/1232273835 wait complete. 2024-04-25T18:42:49.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.273+0000 7f4376600700 1 Processor -- start 2024-04-25T18:42:49.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.273+0000 7f4376600700 1 -- start start 2024-04-25T18:42:49.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.273+0000 7f4376600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4370102a90 0x7f437019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:49.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.273+0000 7f4376600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4370103450 0x7f437019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:49.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.273+0000 7f4376600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4370108a50 0x7f437019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:49.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.273+0000 7f4376600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4370112c30 con 0x7f4370108a50 2024-04-25T18:42:49.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.273+0000 7f4376600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4370112ab0 con 0x7f4370103450 2024-04-25T18:42:49.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.273+0000 7f4376600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4370112db0 con 0x7f4370102a90 2024-04-25T18:42:49.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.274+0000 7f436fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4370108a50 0x7f437019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:49.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.274+0000 7f4367e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4370103450 0x7f437019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:49.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.274+0000 7f436fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4370108a50 0x7f437019fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56960/0 (socket says 172.21.15.112:56960) 2024-04-25T18:42:49.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.274+0000 7f436fe00700 1 -- 172.21.15.112:0/1179418864 learned_addr learned my addr 172.21.15.112:0/1179418864 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:42:49.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.274+0000 7f436f400700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4370102a90 0x7f437019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:49.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.274+0000 7f4367e00700 1 -- 172.21.15.112:0/1179418864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4370102a90 msgr2=0x7f437019b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:49.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.274+0000 7f4367e00700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4370102a90 0x7f437019b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:49.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.274+0000 7f4367e00700 1 -- 172.21.15.112:0/1179418864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4370108a50 msgr2=0x7f437019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:49.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.274+0000 7f4367e00700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4370108a50 0x7f437019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:49.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.274+0000 7f4367e00700 1 -- 172.21.15.112:0/1179418864 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43701a03f0 con 0x7f4370103450 2024-04-25T18:42:49.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.275+0000 7f4367e00700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4370103450 0x7f437019b920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f435800e9c0 tx=0x7f435800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:49.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.275+0000 7f436cc00700 1 -- 172.21.15.112:0/1179418864 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435800c9c0 con 0x7f4370103450 2024-04-25T18:42:49.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.275+0000 7f436cc00700 1 -- 172.21.15.112:0/1179418864 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f435800cb60 con 0x7f4370103450 2024-04-25T18:42:49.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.275+0000 7f4376600700 1 -- 172.21.15.112:0/1179418864 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43701a06e0 con 0x7f4370103450 2024-04-25T18:42:49.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.275+0000 7f436cc00700 1 -- 172.21.15.112:0/1179418864 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358018420 con 0x7f4370103450 2024-04-25T18:42:49.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.275+0000 7f4376600700 1 -- 172.21.15.112:0/1179418864 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43701a7f60 con 0x7f4370103450 2024-04-25T18:42:49.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.276+0000 7f4376600700 1 -- 172.21.15.112:0/1179418864 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4370069730 con 0x7f4370103450 2024-04-25T18:42:49.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.278+0000 7f436cc00700 1 -- 172.21.15.112:0/1179418864 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4358010430 con 0x7f4370103450 2024-04-25T18:42:49.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.279+0000 7f436cc00700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f43540a3cc0 0x7f43540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:49.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.280+0000 7f436cc00700 1 -- 172.21.15.112:0/1179418864 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f43580d0090 con 0x7f4370103450 2024-04-25T18:42:49.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.280+0000 7f436f400700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f43540a3cc0 0x7f43540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:49.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.280+0000 7f436f400700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f43540a3cc0 0x7f43540a6180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f43600098c0 tx=0x7f4360019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:49.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.283+0000 7f436cc00700 1 -- 172.21.15.112:0/1179418864 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f435809ce70 con 0x7f4370103450 2024-04-25T18:42:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.560+0000 7f4376600700 1 -- 172.21.15.112:0/1179418864 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f437019c740 con 0x7f4370103450 2024-04-25T18:42:49.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.561+0000 7f436cc00700 1 -- 172.21.15.112:0/1179418864 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8848 (secure 0 0 0) 0x7f435809c770 con 0x7f4370103450 2024-04-25T18:42:49.563 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:42:49.563 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T18:42:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 -- 172.21.15.112:0/1179418864 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f43540a3cc0 msgr2=0x7f43540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f43540a3cc0 0x7f43540a6180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f43600098c0 tx=0x7f4360019040 comp rx=0 tx=0).stop 2024-04-25T18:42:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 -- 172.21.15.112:0/1179418864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4370103450 msgr2=0x7f437019b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4370103450 0x7f437019b920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f435800e9c0 tx=0x7f435800ecd0 comp rx=0 tx=0).stop 2024-04-25T18:42:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 -- 172.21.15.112:0/1179418864 shutdown_connections 2024-04-25T18:42:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f43540a3cc0 0x7f43540a6180 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4370108a50 0x7f437019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4370102a90 0x7f437019b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 --2- 172.21.15.112:0/1179418864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4370103450 0x7f437019b920 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 -- 172.21.15.112:0/1179418864 >> 172.21.15.112:0/1179418864 conn(0x7f4370079ba0 msgr2=0x7f437010df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 -- 172.21.15.112:0/1179418864 shutdown_connections 2024-04-25T18:42:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:49.564+0000 7f4376600700 1 -- 172.21.15.112:0/1179418864 wait complete. 2024-04-25T18:42:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T18:42:50.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:50.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:50.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:50.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:50.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:50.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:50.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:50.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: Deploying daemon mds.a on smithi112 2024-04-25T18:42:50.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.112:6838/3241331981,v1:172.21.15.112:6839/3241331981] up:boot 2024-04-25T18:42:50.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: fsmap 10 up:standby 2024-04-25T18:42:50.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:42:50.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:50 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1179418864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:50.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:50.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:50.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:50.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: Deploying daemon mds.a on smithi112 2024-04-25T18:42:50.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.112:6838/3241331981,v1:172.21.15.112:6839/3241331981] up:boot 2024-04-25T18:42:50.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: fsmap 10 up:standby 2024-04-25T18:42:50.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:42:50.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:50 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1179418864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:50.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: Deploying daemon mds.a on smithi112 2024-04-25T18:42:50.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.112:6838/3241331981,v1:172.21.15.112:6839/3241331981] up:boot 2024-04-25T18:42:50.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: fsmap 10 up:standby 2024-04-25T18:42:50.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:42:50.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:50 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1179418864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:52.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:52 smithi112 ceph-mon[27864]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:52.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:52 smithi171 ceph-mon[33266]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:52 smithi161 ceph-mon[32609]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:52.544 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:42:53.194 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:42:52 smithi112 systemd[1]: Starting Ceph mds.a for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:42:53.474 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:42:53 smithi112 podman[67119]: 2024-04-25 18:42:53.097110762 +0000 UTC m=+0.034198973 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:42:53.474 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:42:53 smithi112 podman[67119]: 2024-04-25 18:42:53.251197741 +0000 UTC m=+0.188285992 container create d1f85f3692b806b87e0a5bc12b10997449c9bb65c785436058eefb2169fb6bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:42:53.704 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:42:53.857 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:42:53 smithi112 podman[67119]: 2024-04-25 18:42:53.561428875 +0000 UTC m=+0.498517156 container init d1f85f3692b806b87e0a5bc12b10997449c9bb65c785436058eefb2169fb6bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-25T18:42:53.857 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:42:53 smithi112 podman[67119]: 2024-04-25 18:42:53.572144711 +0000 UTC m=+0.509232965 container start d1f85f3692b806b87e0a5bc12b10997449c9bb65c785436058eefb2169fb6bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T18:42:53.858 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:42:53 smithi112 ceph-mds[67164]: -- 172.21.15.112:0/2821217274 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5121e5a40 con 0x55b5121b9000 2024-04-25T18:42:53.858 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:42:53 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a[67160]: starting mds.a at 2024-04-25T18:42:53.858 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:42:53 smithi112 bash[67119]: d1f85f3692b806b87e0a5bc12b10997449c9bb65c785436058eefb2169fb6bc2 2024-04-25T18:42:53.858 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:42:53 smithi112 systemd[1]: Started Ceph mds.a for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:54 smithi171 ceph-mon[33266]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:54 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:54 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:54.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:54 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:54.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:54 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:54.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:54 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:54.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:54 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:54 smithi161 ceph-mon[32609]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:54 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:54 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:54 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:54 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:54 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:54.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:54 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:54.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:54 smithi112 ceph-mon[27864]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:54.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:54 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:54.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:54 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:54.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:54 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:54.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:54 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:42:54.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:54 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:42:54.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:54 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:42:55.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:55 smithi171 ceph-mon[33266]: Deploying daemon mds.b on smithi161 2024-04-25T18:42:55.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:55 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] up:boot 2024-04-25T18:42:55.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:55 smithi171 ceph-mon[33266]: fsmap 11 up:standby 2024-04-25T18:42:55.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:55 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:42:55.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:55 smithi161 ceph-mon[32609]: Deploying daemon mds.b on smithi161 2024-04-25T18:42:55.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:55 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] up:boot 2024-04-25T18:42:55.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:55 smithi161 ceph-mon[32609]: fsmap 11 up:standby 2024-04-25T18:42:55.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:55 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:42:55.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:55 smithi112 ceph-mon[27864]: Deploying daemon mds.b on smithi161 2024-04-25T18:42:55.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:55 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] up:boot 2024-04-25T18:42:55.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:55 smithi112 ceph-mon[27864]: fsmap 11 up:standby 2024-04-25T18:42:55.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:42:55.906 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:42:55 smithi161 systemd[1]: Starting Ceph mds.b for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:42:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.063+0000 7f320ae00700 1 -- 172.21.15.112:0/3559297923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3204109c80 msgr2=0x7f320410dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.063+0000 7f320ae00700 1 --2- 172.21.15.112:0/3559297923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3204109c80 0x7f320410dcd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009990 tx=0x7f31f0009ca0 comp rx=0 tx=0).stop 2024-04-25T18:42:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.063+0000 7f320ae00700 1 -- 172.21.15.112:0/3559297923 shutdown_connections 2024-04-25T18:42:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.063+0000 7f320ae00700 1 --2- 172.21.15.112:0/3559297923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410e3b0 0x7f3204115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.063+0000 7f320ae00700 1 --2- 172.21.15.112:0/3559297923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f32041092d0 0x7f32041096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.063+0000 7f320ae00700 1 --2- 172.21.15.112:0/3559297923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3204109c80 0x7f320410dcd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.063+0000 7f320ae00700 1 -- 172.21.15.112:0/3559297923 >> 172.21.15.112:0/3559297923 conn(0x7f3204076210 msgr2=0x7f3204107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.064+0000 7f320ae00700 1 -- 172.21.15.112:0/3559297923 shutdown_connections 2024-04-25T18:42:56.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.064+0000 7f320ae00700 1 -- 172.21.15.112:0/3559297923 wait complete. 2024-04-25T18:42:56.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.065+0000 7f320ae00700 1 Processor -- start 2024-04-25T18:42:56.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.065+0000 7f320ae00700 1 -- start start 2024-04-25T18:42:56.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.065+0000 7f320ae00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32041092d0 0x7f32041a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.065+0000 7f320ae00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3204109c80 0x7f32041a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.065+0000 7f320ae00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410e3b0 0x7f320419f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.065+0000 7f320ae00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3204119290 con 0x7f320410e3b0 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.065+0000 7f320ae00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3204119110 con 0x7f32041092d0 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.065+0000 7f320ae00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3204119410 con 0x7f3204109c80 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3203e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32041092d0 0x7f32041a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3203e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32041092d0 0x7f32041a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:58094/0 (socket says 172.21.15.112:58094) 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3203400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3204109c80 0x7f32041a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3203e00700 1 -- 172.21.15.112:0/3355330969 learned_addr learned my addr 172.21.15.112:0/3355330969 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3208e00700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410e3b0 0x7f320419f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3203e00700 1 -- 172.21.15.112:0/3355330969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3204109c80 msgr2=0x7f32041a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3203e00700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3204109c80 0x7f32041a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:56.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3203e00700 1 -- 172.21.15.112:0/3355330969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410e3b0 msgr2=0x7f320419f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3203e00700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410e3b0 0x7f320419f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3203e00700 1 -- 172.21.15.112:0/3355330969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320419fd30 con 0x7f32041092d0 2024-04-25T18:42:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3203e00700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32041092d0 0x7f32041a5410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f31f400e9c0 tx=0x7f31f400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.066+0000 7f3200c00700 1 -- 172.21.15.112:0/3355330969 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f400cbd0 con 0x7f32041092d0 2024-04-25T18:42:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.067+0000 7f3200c00700 1 -- 172.21.15.112:0/3355330969 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31f400ee50 con 0x7f32041092d0 2024-04-25T18:42:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.067+0000 7f320ae00700 1 -- 172.21.15.112:0/3355330969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32041a0020 con 0x7f32041092d0 2024-04-25T18:42:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.067+0000 7f3200c00700 1 -- 172.21.15.112:0/3355330969 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4005550 con 0x7f32041092d0 2024-04-25T18:42:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.067+0000 7f320ae00700 1 -- 172.21.15.112:0/3355330969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32041ac2e0 con 0x7f32041092d0 2024-04-25T18:42:56.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.068+0000 7f320ae00700 1 -- 172.21.15.112:0/3355330969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3204069730 con 0x7f32041092d0 2024-04-25T18:42:56.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.070+0000 7f3200c00700 1 -- 172.21.15.112:0/3355330969 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f31f4004750 con 0x7f32041092d0 2024-04-25T18:42:56.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.071+0000 7f3200c00700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f31e80a3cc0 0x7f31e80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:42:56.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.071+0000 7f3200c00700 1 -- 172.21.15.112:0/3355330969 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f31f40d0200 con 0x7f32041092d0 2024-04-25T18:42:56.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.071+0000 7f3203400700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f31e80a3cc0 0x7f31e80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:42:56.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.072+0000 7f3203400700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f31e80a3cc0 0x7f31e80a6180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f32040771e0 tx=0x7f31f000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:42:56.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.075+0000 7f3200c00700 1 -- 172.21.15.112:0/3355330969 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31f409cfe0 con 0x7f32041092d0 2024-04-25T18:42:56.170 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:42:56 smithi161 podman[62815]: 2024-04-25 18:42:56.025206666 +0000 UTC m=+0.017435198 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:42:56.170 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:42:56 smithi161 podman[62815]: 2024-04-25 18:42:56.13729256 +0000 UTC m=+0.129521081 container create 742d4f112ada94b3d143b172b774ff3ef71567edc561a188d9e6a8ae4f49a99c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T18:42:56.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.351+0000 7f320ae00700 1 -- 172.21.15.112:0/3355330969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32041a0b00 con 0x7f32041092d0 2024-04-25T18:42:56.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.352+0000 7f3200c00700 1 -- 172.21.15.112:0/3355330969 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9677 (secure 0 0 0) 0x7f31f409c8e0 con 0x7f32041092d0 2024-04-25T18:42:56.354 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:42:56.354 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-04-25T18:42:56.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.355+0000 7f320ae00700 1 -- 172.21.15.112:0/3355330969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f31e80a3cc0 msgr2=0x7f31e80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:56.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.355+0000 7f320ae00700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f31e80a3cc0 0x7f31e80a6180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f32040771e0 tx=0x7f31f000b260 comp rx=0 tx=0).stop 2024-04-25T18:42:56.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.355+0000 7f320ae00700 1 -- 172.21.15.112:0/3355330969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32041092d0 msgr2=0x7f32041a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:42:56.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.355+0000 7f320ae00700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32041092d0 0x7f32041a5410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f31f400e9c0 tx=0x7f31f400ecd0 comp rx=0 tx=0).stop 2024-04-25T18:42:56.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.355+0000 7f320ae00700 1 -- 172.21.15.112:0/3355330969 shutdown_connections 2024-04-25T18:42:56.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.355+0000 7f320ae00700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f31e80a3cc0 0x7f31e80a6180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:56.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.355+0000 7f320ae00700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f320410e3b0 0x7f320419f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:56.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.355+0000 7f320ae00700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32041092d0 0x7f32041a5410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:56.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.355+0000 7f320ae00700 1 --2- 172.21.15.112:0/3355330969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3204109c80 0x7f32041a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:42:56.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.355+0000 7f320ae00700 1 -- 172.21.15.112:0/3355330969 >> 172.21.15.112:0/3355330969 conn(0x7f3204076210 msgr2=0x7f320410f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:42:56.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.355+0000 7f320ae00700 1 -- 172.21.15.112:0/3355330969 shutdown_connections 2024-04-25T18:42:56.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:42:56.356+0000 7f320ae00700 1 -- 172.21.15.112:0/3355330969 wait complete. 2024-04-25T18:42:56.358 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 12 2024-04-25T18:42:56.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:56 smithi171 ceph-mon[33266]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:56.449 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:56 smithi161 ceph-mon[32609]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:56.449 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:42:56 smithi161 podman[62815]: 2024-04-25 18:42:56.373674263 +0000 UTC m=+0.365902779 container init 742d4f112ada94b3d143b172b774ff3ef71567edc561a188d9e6a8ae4f49a99c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T18:42:56.449 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:42:56 smithi161 podman[62815]: 2024-04-25 18:42:56.37620057 +0000 UTC m=+0.368429087 container start 742d4f112ada94b3d143b172b774ff3ef71567edc561a188d9e6a8ae4f49a99c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:42:56.449 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:42:56 smithi161 ceph-mds[62834]: -- 172.21.15.161:0/1483862537 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd71321a40 con 0x55bd712f4800 2024-04-25T18:42:56.449 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:42:56 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b[62830]: starting mds.b at 2024-04-25T18:42:56.449 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:42:56 smithi161 bash[62815]: 742d4f112ada94b3d143b172b774ff3ef71567edc561a188d9e6a8ae4f49a99c 2024-04-25T18:42:56.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:56 smithi112 ceph-mon[27864]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:56.721 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:42:56 smithi161 systemd[1]: Started Ceph mds.b for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:42:57.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:57 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3355330969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:57.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:57 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:42:57.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:57 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3355330969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:57.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:57 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:42:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:57 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3355330969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:42:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:57.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:57 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:42:58.376 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:58 smithi112 ceph-mon[27864]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:58.376 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:58 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] up:boot 2024-04-25T18:42:58.376 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:58 smithi112 ceph-mon[27864]: fsmap 12 up:standby 2024-04-25T18:42:58.376 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:58 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:42:58.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:58 smithi161 ceph-mon[32609]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:58.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:58 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] up:boot 2024-04-25T18:42:58.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:58 smithi161 ceph-mon[32609]: fsmap 12 up:standby 2024-04-25T18:42:58.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:58 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:42:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:58 smithi171 ceph-mon[33266]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:58 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] up:boot 2024-04-25T18:42:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:58 smithi171 ceph-mon[33266]: fsmap 12 up:standby 2024-04-25T18:42:58.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:58 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:42:59.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:42:59.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:59 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:59 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:59 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:59 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:42:59 smithi112 ceph-mon[27864]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:59.848 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:42:59.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:59 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:59 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:59 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:59 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:42:59 smithi171 ceph-mon[33266]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:42:59.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:59 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:59 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:59 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:59 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:42:59.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:42:59 smithi161 ceph-mon[32609]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:00.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:00 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:00.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:00 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:00.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:00 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:43:00.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:00 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:43:00.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:00 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:00.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:00 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:43:00.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:00 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:00.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:00 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:00.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:00 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:00.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:00 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:43:00.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:00 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:43:00.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:00 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:00.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:00 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:43:00.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:00 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:00.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:00 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:00.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:00 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:00.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:00 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:43:00.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:00 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:43:00.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:00 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:00.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:00 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:43:00.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:00 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.906 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:01 smithi112 ceph-mon[27864]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:01.907 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.907 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.907 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.907 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:01 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:01 smithi171 ceph-mon[33266]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:01.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:01 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:01 smithi161 ceph-mon[32609]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:01.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:01.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:01 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.181+0000 7f52f8200700 1 -- 172.21.15.112:0/4210847677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f0103450 msgr2=0x7f52f010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.181+0000 7f52f8200700 1 --2- 172.21.15.112:0/4210847677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f0103450 0x7f52f010f8b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f52e800b0a0 tx=0x7f52e800b3b0 comp rx=0 tx=0).stop 2024-04-25T18:43:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.181+0000 7f52f8200700 1 -- 172.21.15.112:0/4210847677 shutdown_connections 2024-04-25T18:43:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.181+0000 7f52f8200700 1 --2- 172.21.15.112:0/4210847677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f0103450 0x7f52f010f8b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.181+0000 7f52f8200700 1 --2- 172.21.15.112:0/4210847677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f0108a50 0x7f52f0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.181+0000 7f52f8200700 1 --2- 172.21.15.112:0/4210847677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f0102a90 0x7f52f0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.181+0000 7f52f8200700 1 -- 172.21.15.112:0/4210847677 >> 172.21.15.112:0/4210847677 conn(0x7f52f0079ba0 msgr2=0x7f52f0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.181+0000 7f52f8200700 1 -- 172.21.15.112:0/4210847677 shutdown_connections 2024-04-25T18:43:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.181+0000 7f52f8200700 1 -- 172.21.15.112:0/4210847677 wait complete. 2024-04-25T18:43:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.182+0000 7f52f8200700 1 Processor -- start 2024-04-25T18:43:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.182+0000 7f52f8200700 1 -- start start 2024-04-25T18:43:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.183+0000 7f52f8200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f0102a90 0x7f52f0074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.183+0000 7f52f8200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f0103450 0x7f52f0075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.183+0000 7f52f8200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f0108a50 0x7f52f00757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.183+0000 7f52f8200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0112c50 con 0x7f52f0108a50 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.183+0000 7f52f8200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0112ad0 con 0x7f52f0103450 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.183+0000 7f52f8200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0112dd0 con 0x7f52f0102a90 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.183+0000 7f52f5800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f0102a90 0x7f52f0074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.183+0000 7f52f5800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f0102a90 0x7f52f0074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:37066/0 (socket says 172.21.15.112:37066) 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.183+0000 7f52f5800700 1 -- 172.21.15.112:0/664366154 learned_addr learned my addr 172.21.15.112:0/664366154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.183+0000 7f52f4e00700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f0103450 0x7f52f0075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.183+0000 7f52f6200700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f0108a50 0x7f52f00757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.184+0000 7f52f5800700 1 -- 172.21.15.112:0/664366154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f0103450 msgr2=0x7f52f0075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.184+0000 7f52f5800700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f0103450 0x7f52f0075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.184+0000 7f52f5800700 1 -- 172.21.15.112:0/664366154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f0108a50 msgr2=0x7f52f00757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.184+0000 7f52f5800700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f0108a50 0x7f52f00757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.184+0000 7f52f5800700 1 -- 172.21.15.112:0/664366154 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f01a7640 con 0x7f52f0102a90 2024-04-25T18:43:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.184+0000 7f52f5800700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f0102a90 0x7f52f0074d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f52e400d980 tx=0x7f52e400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.184+0000 7f52ee000700 1 -- 172.21.15.112:0/664366154 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4016070 con 0x7f52f0102a90 2024-04-25T18:43:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.184+0000 7f52f8200700 1 -- 172.21.15.112:0/664366154 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f01a7930 con 0x7f52f0102a90 2024-04-25T18:43:02.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.184+0000 7f52ee000700 1 -- 172.21.15.112:0/664366154 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52e4004830 con 0x7f52f0102a90 2024-04-25T18:43:02.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.184+0000 7f52ee000700 1 -- 172.21.15.112:0/664366154 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4005540 con 0x7f52f0102a90 2024-04-25T18:43:02.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.184+0000 7f52f8200700 1 -- 172.21.15.112:0/664366154 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f01a7e70 con 0x7f52f0102a90 2024-04-25T18:43:02.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.185+0000 7f52f8200700 1 -- 172.21.15.112:0/664366154 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f0069730 con 0x7f52f0102a90 2024-04-25T18:43:02.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.187+0000 7f52ee000700 1 -- 172.21.15.112:0/664366154 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f52e4004220 con 0x7f52f0102a90 2024-04-25T18:43:02.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.189+0000 7f52ee000700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f52d80a3c40 0x7f52d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:02.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.189+0000 7f52ee000700 1 -- 172.21.15.112:0/664366154 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f52e40cfd00 con 0x7f52f0102a90 2024-04-25T18:43:02.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.189+0000 7f52f4e00700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f52d80a3c40 0x7f52d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:02.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.190+0000 7f52f4e00700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f52d80a3c40 0x7f52d80a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f52f007a1a0 tx=0x7f52dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:02.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.193+0000 7f52ee000700 1 -- 172.21.15.112:0/664366154 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52e401f020 con 0x7f52f0102a90 2024-04-25T18:43:02.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.469+0000 7f52f8200700 1 -- 172.21.15.112:0/664366154 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52f002e140 con 0x7f52f0102a90 2024-04-25T18:43:02.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.471+0000 7f52ee000700 1 -- 172.21.15.112:0/664366154 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10506 (secure 0 0 0) 0x7f52e4022350 con 0x7f52f0102a90 2024-04-25T18:43:02.473 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:43:02.473 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24511,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T18:43:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 -- 172.21.15.112:0/664366154 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f52d80a3c40 msgr2=0x7f52d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f52d80a3c40 0x7f52d80a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f52f007a1a0 tx=0x7f52dc009260 comp rx=0 tx=0).stop 2024-04-25T18:43:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 -- 172.21.15.112:0/664366154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f0102a90 msgr2=0x7f52f0074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f0102a90 0x7f52f0074d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f52e400d980 tx=0x7f52e400dc90 comp rx=0 tx=0).stop 2024-04-25T18:43:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 -- 172.21.15.112:0/664366154 shutdown_connections 2024-04-25T18:43:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f52d80a3c40 0x7f52d80a6100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52f0108a50 0x7f52f00757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52f0102a90 0x7f52f0074d40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 --2- 172.21.15.112:0/664366154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f0103450 0x7f52f0075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 -- 172.21.15.112:0/664366154 >> 172.21.15.112:0/664366154 conn(0x7f52f0079ba0 msgr2=0x7f52f01017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 -- 172.21.15.112:0/664366154 shutdown_connections 2024-04-25T18:43:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:02.474+0000 7f52f8200700 1 -- 172.21.15.112:0/664366154 wait complete. 2024-04-25T18:43:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 13 2024-04-25T18:43:02.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:02 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/664366154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:43:02.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:02 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/664366154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:43:02.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:02 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/664366154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:43:03.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:03 smithi171 ceph-mon[33266]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:03.949 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:03 smithi112 ceph-mon[27864]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:03.954 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-25T18:43:03.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:03 smithi161 ceph-mon[32609]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:04.413 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:06.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:06 smithi112 ceph-mon[27864]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:06.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:06.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:06.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:43:06.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:43:06.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:06 smithi171 ceph-mon[33266]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:43:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:43:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:06 smithi161 ceph-mon[32609]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:43:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:43:06.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:43:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.836+0000 7fb6e6200700 1 -- 172.21.15.112:0/3092728402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e0109cc0 msgr2=0x7fb6e010dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.836+0000 7fb6e6200700 1 --2- 172.21.15.112:0/3092728402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e0109cc0 0x7fb6e010dcc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0099c0 tx=0x7fb6cc009cd0 comp rx=0 tx=0).stop 2024-04-25T18:43:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.836+0000 7fb6e6200700 1 -- 172.21.15.112:0/3092728402 shutdown_connections 2024-04-25T18:43:06.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.836+0000 7fb6e6200700 1 --2- 172.21.15.112:0/3092728402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6e010e3a0 0x7fb6e0115f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.836+0000 7fb6e6200700 1 --2- 172.21.15.112:0/3092728402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e0109310 0x7fb6e01096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.836+0000 7fb6e6200700 1 --2- 172.21.15.112:0/3092728402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e0109cc0 0x7fb6e010dcc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.836+0000 7fb6e6200700 1 -- 172.21.15.112:0/3092728402 >> 172.21.15.112:0/3092728402 conn(0x7fb6e0076210 msgr2=0x7fb6e0078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.836+0000 7fb6e6200700 1 -- 172.21.15.112:0/3092728402 shutdown_connections 2024-04-25T18:43:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.836+0000 7fb6e6200700 1 -- 172.21.15.112:0/3092728402 wait complete. 2024-04-25T18:43:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.837+0000 7fb6e6200700 1 Processor -- start 2024-04-25T18:43:06.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.838+0000 7fb6e6200700 1 -- start start 2024-04-25T18:43:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.838+0000 7fb6e6200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e0109310 0x7fb6e01a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.838+0000 7fb6e6200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6e0109cc0 0x7fb6e01a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.838+0000 7fb6e6200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e010e3a0 0x7fb6e019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.838+0000 7fb6e6200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e0119230 con 0x7fb6e010e3a0 2024-04-25T18:43:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.838+0000 7fb6e6200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e01190b0 con 0x7fb6e0109310 2024-04-25T18:43:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.838+0000 7fb6e6200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e01193b0 con 0x7fb6e0109cc0 2024-04-25T18:43:06.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.838+0000 7fb6df400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e0109310 0x7fb6e01a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.838+0000 7fb6dea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6e0109cc0 0x7fb6e01a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.839+0000 7fb6df400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e0109310 0x7fb6e01a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:57976/0 (socket says 172.21.15.112:57976) 2024-04-25T18:43:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.839+0000 7fb6df400700 1 -- 172.21.15.112:0/2359841927 learned_addr learned my addr 172.21.15.112:0/2359841927 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.839+0000 7fb6dea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6e0109cc0 0x7fb6e01a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:37078/0 (socket says 172.21.15.112:37078) 2024-04-25T18:43:06.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.839+0000 7fb6dfe00700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e010e3a0 0x7fb6e019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.839+0000 7fb6dea00700 1 -- 172.21.15.112:0/2359841927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e0109310 msgr2=0x7fb6e01a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.839+0000 7fb6dea00700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e0109310 0x7fb6e01a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.839+0000 7fb6dea00700 1 -- 172.21.15.112:0/2359841927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e010e3a0 msgr2=0x7fb6e019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.839+0000 7fb6dea00700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e010e3a0 0x7fb6e019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.839+0000 7fb6dea00700 1 -- 172.21.15.112:0/2359841927 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e019fd50 con 0x7fb6e0109cc0 2024-04-25T18:43:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.839+0000 7fb6dea00700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6e0109cc0 0x7fb6e01a5920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0098e0 tx=0x7fb6cc0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.840+0000 7fb6d7e00700 1 -- 172.21.15.112:0/2359841927 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc026070 con 0x7fb6e0109cc0 2024-04-25T18:43:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.840+0000 7fb6d7e00700 1 -- 172.21.15.112:0/2359841927 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6cc00f460 con 0x7fb6e0109cc0 2024-04-25T18:43:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.840+0000 7fb6d7e00700 1 -- 172.21.15.112:0/2359841927 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc009e60 con 0x7fb6e0109cc0 2024-04-25T18:43:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.840+0000 7fb6e6200700 1 -- 172.21.15.112:0/2359841927 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e019ffe0 con 0x7fb6e0109cc0 2024-04-25T18:43:06.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.840+0000 7fb6e6200700 1 -- 172.21.15.112:0/2359841927 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e01ac2b0 con 0x7fb6e0109cc0 2024-04-25T18:43:06.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.841+0000 7fb6e6200700 1 -- 172.21.15.112:0/2359841927 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e0069730 con 0x7fb6e0109cc0 2024-04-25T18:43:06.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.843+0000 7fb6d7e00700 1 -- 172.21.15.112:0/2359841927 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fb6cc029070 con 0x7fb6e0109cc0 2024-04-25T18:43:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.844+0000 7fb6d7e00700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fb6c40a3cc0 0x7fb6c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.844+0000 7fb6d7e00700 1 -- 172.21.15.112:0/2359841927 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fb6cc0d08d0 con 0x7fb6e0109cc0 2024-04-25T18:43:06.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.845+0000 7fb6df400700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fb6c40a3cc0 0x7fb6c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:06.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.845+0000 7fb6df400700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fb6c40a3cc0 0x7fb6c40a6180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005950 tx=0x7fb6d000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:06.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:06.847+0000 7fb6d7e00700 1 -- 172.21.15.112:0/2359841927 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6cc013070 con 0x7fb6e0109cc0 2024-04-25T18:43:07.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.074+0000 7fb6e6200700 1 -- 172.21.15.112:0/2359841927 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb6e002df80 con 0x7fb6e0109cc0 2024-04-25T18:43:07.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.075+0000 7fb6d7e00700 1 -- 172.21.15.112:0/2359841927 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) ==== 84+0+65 (secure 0 0 0) 0x7fb6cc09d740 con 0x7fb6e0109cc0 2024-04-25T18:43:07.077 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:43:07.077 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-04-25T18:43:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:43:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-04-25T18:43:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-04-25T18:43:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:43:07.078 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-04-25T18:43:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 -- 172.21.15.112:0/2359841927 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fb6c40a3cc0 msgr2=0x7fb6c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fb6c40a3cc0 0x7fb6c40a6180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005950 tx=0x7fb6d000b500 comp rx=0 tx=0).stop 2024-04-25T18:43:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 -- 172.21.15.112:0/2359841927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6e0109cc0 msgr2=0x7fb6e01a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6e0109cc0 0x7fb6e01a5920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0098e0 tx=0x7fb6cc0045e0 comp rx=0 tx=0).stop 2024-04-25T18:43:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 -- 172.21.15.112:0/2359841927 shutdown_connections 2024-04-25T18:43:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fb6c40a3cc0 0x7fb6c40a6180 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6e010e3a0 0x7fb6e019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e0109310 0x7fb6e01a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 --2- 172.21.15.112:0/2359841927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6e0109cc0 0x7fb6e01a5920 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:07.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 -- 172.21.15.112:0/2359841927 >> 172.21.15.112:0/2359841927 conn(0x7fb6e0076210 msgr2=0x7fb6e0078560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 -- 172.21.15.112:0/2359841927 shutdown_connections 2024-04-25T18:43:07.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:07.079+0000 7fb6e6200700 1 -- 172.21.15.112:0/2359841927 wait complete. 2024-04-25T18:43:07.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:07 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2359841927' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T18:43:07.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:07 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2359841927' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T18:43:07.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:07 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2359841927' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T18:43:08.003 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T18:43:08.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T18:43:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:08 smithi161 ceph-mon[32609]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:08.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:08 smithi112 ceph-mon[27864]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:08.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:08 smithi171 ceph-mon[33266]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:08.446 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:10 smithi112 ceph-mon[27864]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:10.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:10 smithi171 ceph-mon[33266]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:10.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:10 smithi161 ceph-mon[32609]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:10.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.767+0000 7ffbcc800700 1 -- 172.21.15.112:0/4151546823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbc410f4a0 msgr2=0x7ffbc4111890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:10.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.767+0000 7ffbcc800700 1 --2- 172.21.15.112:0/4151546823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbc410f4a0 0x7ffbc4111890 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000b0a0 tx=0x7ffbc000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:43:10.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.767+0000 7ffbcc800700 1 -- 172.21.15.112:0/4151546823 shutdown_connections 2024-04-25T18:43:10.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.767+0000 7ffbcc800700 1 --2- 172.21.15.112:0/4151546823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbc410f4a0 0x7ffbc4111890 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:10.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.767+0000 7ffbcc800700 1 --2- 172.21.15.112:0/4151546823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffbc4074c70 0x7ffbc4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:10.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.767+0000 7ffbcc800700 1 --2- 172.21.15.112:0/4151546823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbc4075590 0x7ffbc410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:10.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.767+0000 7ffbcc800700 1 -- 172.21.15.112:0/4151546823 >> 172.21.15.112:0/4151546823 conn(0x7ffbc4079ba0 msgr2=0x7ffbc4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:10.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.768+0000 7ffbcc800700 1 -- 172.21.15.112:0/4151546823 shutdown_connections 2024-04-25T18:43:10.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.768+0000 7ffbcc800700 1 -- 172.21.15.112:0/4151546823 wait complete. 2024-04-25T18:43:10.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.769+0000 7ffbcc800700 1 Processor -- start 2024-04-25T18:43:10.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.769+0000 7ffbcc800700 1 -- start start 2024-04-25T18:43:10.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.769+0000 7ffbcc800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffbc4074c70 0x7ffbc419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:10.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.769+0000 7ffbcc800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbc4075590 0x7ffbc419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:10.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.769+0000 7ffbcc800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbc410f4a0 0x7ffbc419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.769+0000 7ffbcc800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc4114e30 con 0x7ffbc410f4a0 2024-04-25T18:43:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.769+0000 7ffbcc800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc4114cb0 con 0x7ffbc4075590 2024-04-25T18:43:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.769+0000 7ffbcc800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc4114fb0 con 0x7ffbc4074c70 2024-04-25T18:43:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbca800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbc410f4a0 0x7ffbc419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbc9e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffbc4074c70 0x7ffbc419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbca800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbc410f4a0 0x7ffbc419fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54190/0 (socket says 172.21.15.112:54190) 2024-04-25T18:43:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbca800700 1 -- 172.21.15.112:0/1623151914 learned_addr learned my addr 172.21.15.112:0/1623151914 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:10.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbc9400700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbc4075590 0x7ffbc419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:10.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbca800700 1 -- 172.21.15.112:0/1623151914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffbc4074c70 msgr2=0x7ffbc419b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:10.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbca800700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffbc4074c70 0x7ffbc419b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:10.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbca800700 1 -- 172.21.15.112:0/1623151914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbc4075590 msgr2=0x7ffbc419b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:10.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbca800700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbc4075590 0x7ffbc419b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:10.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbca800700 1 -- 172.21.15.112:0/1623151914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc41a0270 con 0x7ffbc410f4a0 2024-04-25T18:43:10.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbc9400700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbc4075590 0x7ffbc419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:43:10.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.770+0000 7ffbca800700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbc410f4a0 0x7ffbc419fb30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000b780 tx=0x7ffbc0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:10.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.771+0000 7ffbbea00700 1 -- 172.21.15.112:0/1623151914 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0021070 con 0x7ffbc410f4a0 2024-04-25T18:43:10.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.771+0000 7ffbbea00700 1 -- 172.21.15.112:0/1623151914 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbc0007e30 con 0x7ffbc410f4a0 2024-04-25T18:43:10.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.771+0000 7ffbcc800700 1 -- 172.21.15.112:0/1623151914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc41a0500 con 0x7ffbc410f4a0 2024-04-25T18:43:10.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.771+0000 7ffbbea00700 1 -- 172.21.15.112:0/1623151914 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0004700 con 0x7ffbc410f4a0 2024-04-25T18:43:10.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.771+0000 7ffbcc800700 1 -- 172.21.15.112:0/1623151914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc41a7de0 con 0x7ffbc410f4a0 2024-04-25T18:43:10.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.772+0000 7ffbcc800700 1 -- 172.21.15.112:0/1623151914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc4069730 con 0x7ffbc410f4a0 2024-04-25T18:43:10.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.774+0000 7ffbbea00700 1 -- 172.21.15.112:0/1623151914 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ffbc0007980 con 0x7ffbc410f4a0 2024-04-25T18:43:10.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.775+0000 7ffbbea00700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ffbac0a3c40 0x7ffbac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:10.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.776+0000 7ffbbea00700 1 -- 172.21.15.112:0/1623151914 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7ffbc00d0e20 con 0x7ffbc410f4a0 2024-04-25T18:43:10.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.776+0000 7ffbc9e00700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ffbac0a3c40 0x7ffbac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:10.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.777+0000 7ffbc9e00700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ffbac0a3c40 0x7ffbac0a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009c60 tx=0x7ffbb8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:10.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:10.778+0000 7ffbbea00700 1 -- 172.21.15.112:0/1623151914 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffbc009dc90 con 0x7ffbc410f4a0 2024-04-25T18:43:11.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.004+0000 7ffbcc800700 1 -- 172.21.15.112:0/1623151914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ffbc41a7fb0 con 0x7ffbc410f4a0 2024-04-25T18:43:11.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.177+0000 7ffbbea00700 1 -- 172.21.15.112:0/1623151914 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) ==== 138+0+0 (secure 0 0 0) 0x7ffbc009d590 con 0x7ffbc410f4a0 2024-04-25T18:43:11.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.182+0000 7ffbcc800700 1 -- 172.21.15.112:0/1623151914 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ffbac0a3c40 msgr2=0x7ffbac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:11.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.182+0000 7ffbcc800700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ffbac0a3c40 0x7ffbac0a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009c60 tx=0x7ffbb8009260 comp rx=0 tx=0).stop 2024-04-25T18:43:11.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.182+0000 7ffbcc800700 1 -- 172.21.15.112:0/1623151914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbc410f4a0 msgr2=0x7ffbc419fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:11.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.182+0000 7ffbcc800700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbc410f4a0 0x7ffbc419fb30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000b780 tx=0x7ffbc0004d00 comp rx=0 tx=0).stop 2024-04-25T18:43:11.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.184+0000 7ffbcc800700 1 -- 172.21.15.112:0/1623151914 shutdown_connections 2024-04-25T18:43:11.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.184+0000 7ffbcc800700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ffbac0a3c40 0x7ffbac0a6100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:11.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.184+0000 7ffbcc800700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbc410f4a0 0x7ffbc419fb30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:11.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.184+0000 7ffbcc800700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffbc4074c70 0x7ffbc419b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:11.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.184+0000 7ffbcc800700 1 --2- 172.21.15.112:0/1623151914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbc4075590 0x7ffbc419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:11.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.184+0000 7ffbcc800700 1 -- 172.21.15.112:0/1623151914 >> 172.21.15.112:0/1623151914 conn(0x7ffbc4079ba0 msgr2=0x7ffbc40ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:11.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.185+0000 7ffbcc800700 1 -- 172.21.15.112:0/1623151914 shutdown_connections 2024-04-25T18:43:11.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:11.185+0000 7ffbcc800700 1 -- 172.21.15.112:0/1623151914 wait complete. 2024-04-25T18:43:11.190 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-04-25T18:43:11.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:11 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1623151914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T18:43:11.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:11 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1623151914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T18:43:11.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:11 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1623151914' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T18:43:12.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T18:43:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:12 smithi161 ceph-mon[32609]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:12.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:12 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1623151914' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T18:43:12.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:12 smithi161 ceph-mon[32609]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T18:43:12.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:12 smithi112 ceph-mon[27864]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:12.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:12 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1623151914' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T18:43:12.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:12 smithi112 ceph-mon[27864]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T18:43:12.618 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:12.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:12 smithi171 ceph-mon[33266]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:12.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:12 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1623151914' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T18:43:12.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:12 smithi171 ceph-mon[33266]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T18:43:13.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:13 smithi112 ceph-mon[27864]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T18:43:13.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:13 smithi161 ceph-mon[32609]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T18:43:13.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:13 smithi171 ceph-mon[33266]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T18:43:14.349 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:14 smithi112 ceph-mon[27864]: pgmap v326: 33 pgs: 24 unknown, 9 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:14.349 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:14 smithi112 ceph-mon[27864]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:43:14.349 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:14 smithi112 ceph-mon[27864]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T18:43:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:14 smithi161 ceph-mon[32609]: pgmap v326: 33 pgs: 24 unknown, 9 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:14 smithi161 ceph-mon[32609]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:43:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:14 smithi161 ceph-mon[32609]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T18:43:14.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:14 smithi171 ceph-mon[33266]: pgmap v326: 33 pgs: 24 unknown, 9 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:14.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:14 smithi171 ceph-mon[33266]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:43:14.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:14 smithi171 ceph-mon[33266]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T18:43:14.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.860+0000 7f3a1b400700 1 -- 172.21.15.112:0/2830455422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a1406b980 msgr2=0x7f3a1410f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:14.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.860+0000 7f3a1b400700 1 --2- 172.21.15.112:0/2830455422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a1406b980 0x7f3a1410f8f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00b0a0 tx=0x7f3a0c00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:43:14.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.860+0000 7f3a1b400700 1 -- 172.21.15.112:0/2830455422 shutdown_connections 2024-04-25T18:43:14.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.860+0000 7f3a1b400700 1 --2- 172.21.15.112:0/2830455422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a1406b980 0x7f3a1410f8f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:14.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.860+0000 7f3a1b400700 1 --2- 172.21.15.112:0/2830455422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a14108a20 0x7f3a14108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:14.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.860+0000 7f3a1b400700 1 --2- 172.21.15.112:0/2830455422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a1406afe0 0x7f3a1406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:14.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.861+0000 7f3a1b400700 1 -- 172.21.15.112:0/2830455422 >> 172.21.15.112:0/2830455422 conn(0x7f3a140788b0 msgr2=0x7f3a14078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:14.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.861+0000 7f3a1b400700 1 -- 172.21.15.112:0/2830455422 shutdown_connections 2024-04-25T18:43:14.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.861+0000 7f3a1b400700 1 -- 172.21.15.112:0/2830455422 wait complete. 2024-04-25T18:43:14.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.862+0000 7f3a1b400700 1 Processor -- start 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.862+0000 7f3a1b400700 1 -- start start 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.863+0000 7f3a1b400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a1406afe0 0x7f3a1419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.863+0000 7f3a1b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1406b980 0x7f3a1419b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.863+0000 7f3a1b400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a14108a20 0x7f3a1419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.863+0000 7f3a1b400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14112c50 con 0x7f3a1406b980 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.863+0000 7f3a1b400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14112ad0 con 0x7f3a14108a20 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.863+0000 7f3a1b400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14112dd0 con 0x7f3a1406afe0 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.863+0000 7f3a13e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1406b980 0x7f3a1419b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.863+0000 7f3a13e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1406b980 0x7f3a1419b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54202/0 (socket says 172.21.15.112:54202) 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.863+0000 7f3a13e00700 1 -- 172.21.15.112:0/3790113019 learned_addr learned my addr 172.21.15.112:0/3790113019 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.863+0000 7f3a18a00700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a1406afe0 0x7f3a1419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:14.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.863+0000 7f3a19400700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a14108a20 0x7f3a1419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.864+0000 7f3a13e00700 1 -- 172.21.15.112:0/3790113019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a1406afe0 msgr2=0x7f3a1419b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.864+0000 7f3a13e00700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a1406afe0 0x7f3a1419b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.864+0000 7f3a13e00700 1 -- 172.21.15.112:0/3790113019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a14108a20 msgr2=0x7f3a1419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.864+0000 7f3a13e00700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a14108a20 0x7f3a1419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.864+0000 7f3a13e00700 1 -- 172.21.15.112:0/3790113019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a141a02d0 con 0x7f3a1406b980 2024-04-25T18:43:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.864+0000 7f3a13e00700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1406b980 0x7f3a1419b800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000e9c0 tx=0x7f3a0000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:14.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.864+0000 7f3a11600700 1 -- 172.21.15.112:0/3790113019 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0000cbd0 con 0x7f3a1406b980 2024-04-25T18:43:14.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.864+0000 7f3a11600700 1 -- 172.21.15.112:0/3790113019 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0000ee60 con 0x7f3a1406b980 2024-04-25T18:43:14.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.864+0000 7f3a1b400700 1 -- 172.21.15.112:0/3790113019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a141a05c0 con 0x7f3a1406b980 2024-04-25T18:43:14.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.864+0000 7f3a11600700 1 -- 172.21.15.112:0/3790113019 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a000055c0 con 0x7f3a1406b980 2024-04-25T18:43:14.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.865+0000 7f3a1b400700 1 -- 172.21.15.112:0/3790113019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a141a7e40 con 0x7f3a1406b980 2024-04-25T18:43:14.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.866+0000 7f3a1b400700 1 -- 172.21.15.112:0/3790113019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a14069730 con 0x7f3a1406b980 2024-04-25T18:43:14.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.868+0000 7f3a11600700 1 -- 172.21.15.112:0/3790113019 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3a00004750 con 0x7f3a1406b980 2024-04-25T18:43:14.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.869+0000 7f3a11600700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f39fc0a3c40 0x7f39fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:14.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.870+0000 7f3a11600700 1 -- 172.21.15.112:0/3790113019 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7497+0+0 (secure 0 0 0) 0x7f3a000d0600 con 0x7f3a1406b980 2024-04-25T18:43:14.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.870+0000 7f3a18a00700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f39fc0a3c40 0x7f39fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:14.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.870+0000 7f3a18a00700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f39fc0a3c40 0x7f39fc0a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009680 tx=0x7f3a08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:14.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:14.872+0000 7f3a11600700 1 -- 172.21.15.112:0/3790113019 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a00053ff0 con 0x7f3a1406b980 2024-04-25T18:43:15.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.100+0000 7f3a1b400700 1 -- 172.21.15.112:0/3790113019 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3a1402df80 con 0x7f3a1406b980 2024-04-25T18:43:15.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.220+0000 7f3a11600700 1 -- 172.21.15.112:0/3790113019 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v89) ==== 170+0+0 (secure 0 0 0) 0x7f3a0009d270 con 0x7f3a1406b980 2024-04-25T18:43:15.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.225+0000 7f3a1b400700 1 -- 172.21.15.112:0/3790113019 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f39fc0a3c40 msgr2=0x7f39fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:15.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.225+0000 7f3a1b400700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f39fc0a3c40 0x7f39fc0a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009680 tx=0x7f3a08009260 comp rx=0 tx=0).stop 2024-04-25T18:43:15.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.225+0000 7f3a1b400700 1 -- 172.21.15.112:0/3790113019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1406b980 msgr2=0x7f3a1419b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:15.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.225+0000 7f3a1b400700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1406b980 0x7f3a1419b800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000e9c0 tx=0x7f3a0000ecd0 comp rx=0 tx=0).stop 2024-04-25T18:43:15.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.226+0000 7f3a1b400700 1 -- 172.21.15.112:0/3790113019 shutdown_connections 2024-04-25T18:43:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.226+0000 7f3a1b400700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f39fc0a3c40 0x7f39fc0a6100 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.226+0000 7f3a1b400700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a14108a20 0x7f3a1419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.226+0000 7f3a1b400700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a1406afe0 0x7f3a1419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.226+0000 7f3a1b400700 1 --2- 172.21.15.112:0/3790113019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a1406b980 0x7f3a1419b800 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:15.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.226+0000 7f3a1b400700 1 -- 172.21.15.112:0/3790113019 >> 172.21.15.112:0/3790113019 conn(0x7f3a140788b0 msgr2=0x7f3a1410def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:15.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.228+0000 7f3a1b400700 1 -- 172.21.15.112:0/3790113019 shutdown_connections 2024-04-25T18:43:15.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:15.228+0000 7f3a1b400700 1 -- 172.21.15.112:0/3790113019 wait complete. 2024-04-25T18:43:15.233 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-04-25T18:43:15.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:15 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3790113019' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:43:15.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:15 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3790113019' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:43:15.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:15 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3790113019' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:43:16.143 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T18:43:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:16 smithi112 ceph-mon[27864]: pgmap v328: 33 pgs: 3 unknown, 30 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:16 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3790113019' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T18:43:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:16 smithi112 ceph-mon[27864]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T18:43:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:16.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:16 smithi171 ceph-mon[33266]: pgmap v328: 33 pgs: 3 unknown, 30 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:16.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:16 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3790113019' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T18:43:16.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:16 smithi171 ceph-mon[33266]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T18:43:16.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:16 smithi161 ceph-mon[32609]: pgmap v328: 33 pgs: 3 unknown, 30 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:16.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:16 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3790113019' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T18:43:16.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:16 smithi161 ceph-mon[32609]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T18:43:17.300 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:17 smithi112 ceph-mon[27864]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T18:43:17.300 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:17 smithi112 ceph-mon[27864]: pgmap v331: 97 pgs: 8 creating+peering, 40 active+clean, 49 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:17.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:17 smithi171 ceph-mon[33266]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T18:43:17.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:17 smithi171 ceph-mon[33266]: pgmap v331: 97 pgs: 8 creating+peering, 40 active+clean, 49 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:17.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:17 smithi161 ceph-mon[32609]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T18:43:17.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:17 smithi161 ceph-mon[32609]: pgmap v331: 97 pgs: 8 creating+peering, 40 active+clean, 49 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:18 smithi112 ceph-mon[27864]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T18:43:18.622 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:18 smithi161 ceph-mon[32609]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T18:43:18.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:18 smithi171 ceph-mon[33266]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T18:43:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.069+0000 7f16d5200700 1 -- 172.21.15.112:0/1009316792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16d0107940 msgr2=0x7f16d0107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.069+0000 7f16d5200700 1 --2- 172.21.15.112:0/1009316792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16d0107940 0x7f16d0107d20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16c00099c0 tx=0x7f16c0009cd0 comp rx=0 tx=0).stop 2024-04-25T18:43:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.069+0000 7f16d5200700 1 -- 172.21.15.112:0/1009316792 shutdown_connections 2024-04-25T18:43:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.069+0000 7f16d5200700 1 --2- 172.21.15.112:0/1009316792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d01034b0 0x7f16d010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.069+0000 7f16d5200700 1 --2- 172.21.15.112:0/1009316792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16d0107940 0x7f16d0107d20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.069+0000 7f16d5200700 1 --2- 172.21.15.112:0/1009316792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d0102af0 0x7f16d0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.069+0000 7f16d5200700 1 -- 172.21.15.112:0/1009316792 >> 172.21.15.112:0/1009316792 conn(0x7f16d0079ba0 msgr2=0x7f16d0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.070+0000 7f16d5200700 1 -- 172.21.15.112:0/1009316792 shutdown_connections 2024-04-25T18:43:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.070+0000 7f16d5200700 1 -- 172.21.15.112:0/1009316792 wait complete. 2024-04-25T18:43:19.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.071+0000 7f16d5200700 1 Processor -- start 2024-04-25T18:43:19.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.071+0000 7f16d5200700 1 -- start start 2024-04-25T18:43:19.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.071+0000 7f16d5200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16d0102af0 0x7f16d0199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:19.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.071+0000 7f16d5200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d01034b0 0x7f16d0199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.071+0000 7f16d5200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d0107940 0x7f16d019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.071+0000 7f16d5200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0112c60 con 0x7f16d01034b0 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.071+0000 7f16d5200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0112ae0 con 0x7f16d0102af0 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.071+0000 7f16d5200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0112de0 con 0x7f16d0107940 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.071+0000 7f16cd400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d01034b0 0x7f16d0199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.072+0000 7f16ce800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d0107940 0x7f16d019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.072+0000 7f16cd400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d01034b0 0x7f16d0199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54216/0 (socket says 172.21.15.112:54216) 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.072+0000 7f16cde00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16d0102af0 0x7f16d0199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.072+0000 7f16cd400700 1 -- 172.21.15.112:0/3038229347 learned_addr learned my addr 172.21.15.112:0/3038229347 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.072+0000 7f16cde00700 1 -- 172.21.15.112:0/3038229347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d0107940 msgr2=0x7f16d019d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.072+0000 7f16cde00700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d0107940 0x7f16d019d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:19.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.072+0000 7f16cde00700 1 -- 172.21.15.112:0/3038229347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d01034b0 msgr2=0x7f16d0199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:19.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.072+0000 7f16cde00700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d01034b0 0x7f16d0199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:19.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.072+0000 7f16cde00700 1 -- 172.21.15.112:0/3038229347 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d019e120 con 0x7f16d0102af0 2024-04-25T18:43:19.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.072+0000 7f16cde00700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16d0102af0 0x7f16d0199110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16c00098e0 tx=0x7f16c0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:19.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.072+0000 7f16c6a00700 1 -- 172.21.15.112:0/3038229347 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0026070 con 0x7f16d0102af0 2024-04-25T18:43:19.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.073+0000 7f16d5200700 1 -- 172.21.15.112:0/3038229347 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d019e3b0 con 0x7f16d0102af0 2024-04-25T18:43:19.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.073+0000 7f16c6a00700 1 -- 172.21.15.112:0/3038229347 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16c0003c70 con 0x7f16d0102af0 2024-04-25T18:43:19.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.073+0000 7f16c6a00700 1 -- 172.21.15.112:0/3038229347 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c000f9c0 con 0x7f16d0102af0 2024-04-25T18:43:19.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.073+0000 7f16d5200700 1 -- 172.21.15.112:0/3038229347 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d01a5c90 con 0x7f16d0102af0 2024-04-25T18:43:19.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.074+0000 7f16d5200700 1 -- 172.21.15.112:0/3038229347 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d01041b0 con 0x7f16d0102af0 2024-04-25T18:43:19.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.076+0000 7f16c6a00700 1 -- 172.21.15.112:0/3038229347 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f16c0029070 con 0x7f16d0102af0 2024-04-25T18:43:19.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.077+0000 7f16c6a00700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f16b40a3c40 0x7f16b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:19.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.077+0000 7f16c6a00700 1 -- 172.21.15.112:0/3038229347 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7879+0+0 (secure 0 0 0) 0x7f16c00d0dd0 con 0x7f16d0102af0 2024-04-25T18:43:19.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.077+0000 7f16cd400700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f16b40a3c40 0x7f16b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:19.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.078+0000 7f16cd400700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f16b40a3c40 0x7f16b40a6100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f16d007a210 tx=0x7f16bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:19.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.080+0000 7f16c6a00700 1 -- 172.21.15.112:0/3038229347 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16c0030020 con 0x7f16d0102af0 2024-04-25T18:43:19.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:19.345+0000 7f16d5200700 1 -- 172.21.15.112:0/3038229347 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f16d002df80 con 0x7f16d0102af0 2024-04-25T18:43:19.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:19 smithi112 ceph-mon[27864]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:43:19.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:19 smithi112 ceph-mon[27864]: pgmap v333: 97 pgs: 8 creating+peering, 69 active+clean, 20 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:19.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:19 smithi171 ceph-mon[33266]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:43:19.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:19 smithi171 ceph-mon[33266]: pgmap v333: 97 pgs: 8 creating+peering, 69 active+clean, 20 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:19.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:19 smithi161 ceph-mon[32609]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:43:19.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:19 smithi161 ceph-mon[32609]: pgmap v333: 97 pgs: 8 creating+peering, 69 active+clean, 20 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:20.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.287+0000 7f16c6a00700 1 -- 172.21.15.112:0/3038229347 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f16c0013070 con 0x7f16d0102af0 2024-04-25T18:43:20.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.293+0000 7f16d5200700 1 -- 172.21.15.112:0/3038229347 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f16b40a3c40 msgr2=0x7f16b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:20.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.293+0000 7f16d5200700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f16b40a3c40 0x7f16b40a6100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f16d007a210 tx=0x7f16bc009260 comp rx=0 tx=0).stop 2024-04-25T18:43:20.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.293+0000 7f16d5200700 1 -- 172.21.15.112:0/3038229347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16d0102af0 msgr2=0x7f16d0199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:20.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.293+0000 7f16d5200700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16d0102af0 0x7f16d0199110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16c00098e0 tx=0x7f16c0004270 comp rx=0 tx=0).stop 2024-04-25T18:43:20.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.295+0000 7f16d5200700 1 -- 172.21.15.112:0/3038229347 shutdown_connections 2024-04-25T18:43:20.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.295+0000 7f16d5200700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f16b40a3c40 0x7f16b40a6100 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:20.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.295+0000 7f16d5200700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d0107940 0x7f16d019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:20.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.295+0000 7f16d5200700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16d0102af0 0x7f16d0199110 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:20.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.295+0000 7f16d5200700 1 --2- 172.21.15.112:0/3038229347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d01034b0 0x7f16d0199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:20.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.295+0000 7f16d5200700 1 -- 172.21.15.112:0/3038229347 >> 172.21.15.112:0/3038229347 conn(0x7f16d0079ba0 msgr2=0x7f16d0100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.298+0000 7f16d5200700 1 -- 172.21.15.112:0/3038229347 shutdown_connections 2024-04-25T18:43:20.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:20.298+0000 7f16d5200700 1 -- 172.21.15.112:0/3038229347 wait complete. 2024-04-25T18:43:20.302 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T18:43:20.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:20 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3038229347' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:43:20.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:20 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:43:20.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:20 smithi171 ceph-mon[33266]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T18:43:20.683 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:20 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3038229347' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:43:20.683 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:20 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:43:20.683 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:20 smithi112 ceph-mon[27864]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T18:43:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:20 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3038229347' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:43:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:20 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:43:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:20 smithi161 ceph-mon[32609]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T18:43:21.178 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:43:21.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:21 smithi112 ceph-mon[27864]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T18:43:21.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:21 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T18:43:21.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:21 smithi112 ceph-mon[27864]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T18:43:21.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:21 smithi112 ceph-mon[27864]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T18:43:21.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:21 smithi112 ceph-mon[27864]: pgmap v335: 97 pgs: 97 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:21.618 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:21.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:21 smithi171 ceph-mon[33266]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T18:43:21.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:21 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T18:43:21.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:21 smithi171 ceph-mon[33266]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T18:43:21.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:21 smithi171 ceph-mon[33266]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T18:43:21.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:21 smithi171 ceph-mon[33266]: pgmap v335: 97 pgs: 97 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:21.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:21 smithi161 ceph-mon[32609]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T18:43:21.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:21 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T18:43:21.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:21 smithi161 ceph-mon[32609]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T18:43:21.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:21 smithi161 ceph-mon[32609]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T18:43:21.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:21 smithi161 ceph-mon[32609]: pgmap v335: 97 pgs: 97 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:22.439 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:22 smithi112 ceph-mon[27864]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T18:43:22.440 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:22 smithi112 ceph-mon[27864]: Cluster is now healthy 2024-04-25T18:43:22.440 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:22 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] up:active 2024-04-25T18:43:22.440 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:22 smithi112 ceph-mon[27864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:22.440 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:22 smithi112 ceph-mon[27864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:22 smithi171 ceph-mon[33266]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T18:43:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:22 smithi171 ceph-mon[33266]: Cluster is now healthy 2024-04-25T18:43:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:22 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] up:active 2024-04-25T18:43:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:22 smithi171 ceph-mon[33266]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:22 smithi171 ceph-mon[33266]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:22 smithi161 ceph-mon[32609]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T18:43:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:22 smithi161 ceph-mon[32609]: Cluster is now healthy 2024-04-25T18:43:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:22 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] up:active 2024-04-25T18:43:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:22 smithi161 ceph-mon[32609]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:22 smithi161 ceph-mon[32609]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:23.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:23 smithi161 ceph-mon[32609]: pgmap v336: 97 pgs: 97 active+clean; 578 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 745 B/s wr, 1 op/s 2024-04-25T18:43:23.471 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:43:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b[62830]: 2024-04-25T18:43:23.189+0000 7fd6eb3ac700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T18:43:23.659 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:23 smithi112 ceph-mon[27864]: pgmap v336: 97 pgs: 97 active+clean; 578 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 745 B/s wr, 1 op/s 2024-04-25T18:43:23.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:23 smithi171 ceph-mon[33266]: pgmap v336: 97 pgs: 97 active+clean; 578 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 745 B/s wr, 1 op/s 2024-04-25T18:43:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.196+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3624391783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a28109f90 msgr2=0x7f7a28111b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.196+0000 7f7a2ea00700 1 --2- 172.21.15.112:0/3624391783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a28109f90 0x7f7a28111b60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000b0d0 tx=0x7f7a2000b3e0 comp rx=0 tx=0).stop 2024-04-25T18:43:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.197+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3624391783 shutdown_connections 2024-04-25T18:43:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.197+0000 7f7a2ea00700 1 --2- 172.21.15.112:0/3624391783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a28109f90 0x7f7a28111b60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.197+0000 7f7a2ea00700 1 --2- 172.21.15.112:0/3624391783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a28104eb0 0x7f7a28105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.197+0000 7f7a2ea00700 1 --2- 172.21.15.112:0/3624391783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a28105860 0x7f7a281098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.197+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3624391783 >> 172.21.15.112:0/3624391783 conn(0x7f7a28100740 msgr2=0x7f7a28102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.197+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3624391783 shutdown_connections 2024-04-25T18:43:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.197+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3624391783 wait complete. 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.198+0000 7f7a2ea00700 1 Processor -- start 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.198+0000 7f7a2ea00700 1 -- start start 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.198+0000 7f7a2ea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a28104eb0 0x7f7a2819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.198+0000 7f7a2ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a28105860 0x7f7a2819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.198+0000 7f7a2ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a28109f90 0x7f7a2819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.198+0000 7f7a2ea00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28114ea0 con 0x7f7a28105860 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.198+0000 7f7a2ea00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28114d20 con 0x7f7a28109f90 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.198+0000 7f7a2ea00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28115020 con 0x7f7a28104eb0 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a27e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a28104eb0 0x7f7a2819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a27e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a28104eb0 0x7f7a2819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:55138/0 (socket says 172.21.15.112:55138) 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a27e00700 1 -- 172.21.15.112:0/3300183611 learned_addr learned my addr 172.21.15.112:0/3300183611 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:24.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a1fe00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a28105860 0x7f7a2819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a2ca00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a28109f90 0x7f7a2819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a27e00700 1 -- 172.21.15.112:0/3300183611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a28109f90 msgr2=0x7f7a2819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a27e00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a28109f90 0x7f7a2819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a27e00700 1 -- 172.21.15.112:0/3300183611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a28105860 msgr2=0x7f7a2819b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a27e00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a28105860 0x7f7a2819b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a27e00700 1 -- 172.21.15.112:0/3300183611 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a281a0360 con 0x7f7a28104eb0 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a2ca00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a28109f90 0x7f7a2819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a1fe00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a28105860 0x7f7a2819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.199+0000 7f7a27e00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a28104eb0 0x7f7a2819b350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800e9c0 tx=0x7f7a1800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.200+0000 7f7a25600700 1 -- 172.21.15.112:0/3300183611 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1800cbd0 con 0x7f7a28104eb0 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.200+0000 7f7a25600700 1 -- 172.21.15.112:0/3300183611 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a1800cd70 con 0x7f7a28104eb0 2024-04-25T18:43:24.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.200+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3300183611 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a281a0650 con 0x7f7a28104eb0 2024-04-25T18:43:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.200+0000 7f7a25600700 1 -- 172.21.15.112:0/3300183611 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18005610 con 0x7f7a28104eb0 2024-04-25T18:43:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.200+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3300183611 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a281a7ed0 con 0x7f7a28104eb0 2024-04-25T18:43:24.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.201+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3300183611 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a28069730 con 0x7f7a28104eb0 2024-04-25T18:43:24.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.203+0000 7f7a25600700 1 -- 172.21.15.112:0/3300183611 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7a18004750 con 0x7f7a28104eb0 2024-04-25T18:43:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.204+0000 7f7a25600700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7a0c0a3cc0 0x7f7a0c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.205+0000 7f7a25600700 1 -- 172.21.15.112:0/3300183611 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f7a180d07c0 con 0x7f7a28104eb0 2024-04-25T18:43:24.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.205+0000 7f7a1fe00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7a0c0a3cc0 0x7f7a0c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:24.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.205+0000 7f7a1fe00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7a0c0a3cc0 0x7f7a0c0a6180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7a2810b5a0 tx=0x7f7a10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:24.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.207+0000 7f7a25600700 1 -- 172.21.15.112:0/3300183611 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a18053ff0 con 0x7f7a28104eb0 2024-04-25T18:43:24.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.442+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3300183611 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a2802df80 con 0x7f7a28104eb0 2024-04-25T18:43:24.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.444+0000 7f7a25600700 1 -- 172.21.15.112:0/3300183611 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22239 (secure 0 0 0) 0x7f7a1809d250 con 0x7f7a28104eb0 2024-04-25T18:43:24.447 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:43:24.447 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":92,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","created":"2024-04-25T18:33:37.379089+0000","modified":"2024-04-25T18:43:19.347501+0000","last_up_change":"2024-04-25T18:42:00.961646+0000","last_in_change":"2024-04-25T18:41:40.647162+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:37:45.106199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:43:11.017975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:43:15.116917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efb4bc01-e29b-4402-8a7a-3387d1731ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6803","nonce":232254535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6805","nonce":232254535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6809","nonce":232254535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6807","nonce":232254535}]},"public_addr":"172.21.15.112:6803/232254535","cluster_addr":"172.21.15.112:6805/232254535","heartbeat_back_addr":"172.21.15.112:6809/232254535","heartbeat_front_addr":"172.21.15.112:6807/232254535","state":["exists","up"]},{"osd":1,"uuid":"65a83fa2-96a6-4743-8c17-21531f5e885e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6801","nonce":664224744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6803","nonce":664224744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6807","nonce":664224744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6805","nonce":664224744}]},"public_addr":"172.21.15.161:6801/664224744","cluster_addr":"172.21.15.161:6803/664224744","heartbeat_back_addr":"172.21.15.161:6807/664224744","heartbeat_front_addr":"172.21.15.161:6805/664224744","state":["exists","up"]},{"osd":2,"uuid":"7562ec4c-c126-48ec-a51c-56fa461a2e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6801","nonce":345172073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6803","nonce":345172073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6807","nonce":345172073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6805","nonce":345172073}]},"public_addr":"172.21.15.171:6801/345172073","cluster_addr":"172.21.15.171:6803/345172073","heartbeat_back_addr":"172.21.15.171:6807/345172073","heartbeat_front_addr":"172.21.15.171:6805/345172073","state":["exists","up"]},{"osd":3,"uuid":"d7aae8cd-fec7-46d9-8773-25eaaca3cc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6811","nonce":1472407156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6813","nonce":1472407156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6817","nonce":1472407156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6815","nonce":1472407156}]},"public_addr":"172.21.15.112:6811/1472407156","cluster_addr":"172.21.15.112:6813/1472407156","heartbeat_back_addr":"172.21.15.112:6817/1472407156","heartbeat_front_addr":"172.21.15.112:6815/1472407156","state":["exists","up"]},{"osd":4,"uuid":"b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6809","nonce":373204306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6811","nonce":373204306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6815","nonce":373204306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6813","nonce":373204306}]},"public_addr":"172.21.15.161:6809/373204306","cluster_addr":"172.21.15.161:6811/373204306","heartbeat_back_addr":"172.21.15.161:6815/373204306","heartbeat_front_addr":"172.21.15.161:6813/373204306","state":["exists","up"]},{"osd":5,"uuid":"56124d12-bc1f-4cba-8430-938248851776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6809","nonce":2643512313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6811","nonce":2643512313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6815","nonce":2643512313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6813","nonce":2643512313}]},"public_addr":"172.21.15.171:6809/2643512313","cluster_addr":"172.21.15.171:6811/2643512313","heartbeat_back_addr":"172.21.15.171:6815/2643512313","heartbeat_front_addr":"172.21.15.171:6813/2643512313","state":["exists","up"]},{"osd":6,"uuid":"9563a9d3-352b-4686-975c-282ec167248b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6819","nonce":1933654933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6821","nonce":1933654933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6825","nonce":1933654933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6823","nonce":1933654933}]},"public_addr":"172.21.15.112:6819/1933654933","cluster_addr":"172.21.15.112:6821/1933654933","heartbeat_back_addr":"172.21.15.112:6825/1933654933","heartbeat_front_addr":"172.21.15.112:6823/1933654933","state":["exists","up"]},{"osd":7,"uuid":"f2d06e86-6aac-40f4-956e-18fe0b0e3543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6817","nonce":3060201138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6819","nonce":3060201138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6823","nonce":3060201138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6821","nonce":3060201138}]},"public_addr":"172.21.15.161:6817/3060201138","cluster_addr":"172.21.15.161:6819/3060201138","heartbeat_back_addr":"172.21.15.161:6823/3060201138","heartbeat_front_addr":"172.21.15.161:6821/3060201138","state":["exists","up"]},{"osd":8,"uuid":"f006e45f-2bb9-45d0-9020-be7c33c6a825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6817","nonce":835756571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6819","nonce":835756571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6823","nonce":835756571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6821","nonce":835756571}]},"public_addr":"172.21.15.171:6817/835756571","cluster_addr":"172.21.15.171:6819/835756571","heartbeat_back_addr":"172.21.15.171:6823/835756571","heartbeat_front_addr":"172.21.15.171:6821/835756571","state":["exists","up"]},{"osd":9,"uuid":"c5a96491-4151-4633-b3bc-2c214fe51c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6827","nonce":349758528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6829","nonce":349758528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6833","nonce":349758528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6831","nonce":349758528}]},"public_addr":"172.21.15.112:6827/349758528","cluster_addr":"172.21.15.112:6829/349758528","heartbeat_back_addr":"172.21.15.112:6833/349758528","heartbeat_front_addr":"172.21.15.112:6831/349758528","state":["exists","up"]},{"osd":10,"uuid":"244eda3e-25a0-490e-879f-c1e2726c1be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6825","nonce":3013292765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6827","nonce":3013292765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6831","nonce":3013292765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6829","nonce":3013292765}]},"public_addr":"172.21.15.161:6825/3013292765","cluster_addr":"172.21.15.161:6827/3013292765","heartbeat_back_addr":"172.21.15.161:6831/3013292765","heartbeat_front_addr":"172.21.15.161:6829/3013292765","state":["exists","up"]},{"osd":11,"uuid":"356e5c54-bbc3-45ee-bda2-85164ce3cb92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6825","nonce":1010336458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6827","nonce":1010336458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6831","nonce":1010336458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6829","nonce":1010336458}]},"public_addr":"172.21.15.171:6825/1010336458","cluster_addr":"172.21.15.171:6827/1010336458","heartbeat_back_addr":"172.21.15.171:6831/1010336458","heartbeat_front_addr":"172.21.15.171:6829/1010336458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:36:51.024650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:17.026638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:41.239726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:15.148968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:41.029118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:06.225858+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:41.639444+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:07.753400+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:32.292085+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:07.159926+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:33.749782+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:58.594447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/181862246":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/3393454453":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3195850527":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6801/3468460974":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3392048544":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/1329430933":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/735578423":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/3468460974":"2024-04-26T18:34:20.605775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.447+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3300183611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7a0c0a3cc0 msgr2=0x7f7a0c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.447+0000 7f7a2ea00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7a0c0a3cc0 0x7f7a0c0a6180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7a2810b5a0 tx=0x7f7a10009260 comp rx=0 tx=0).stop 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.447+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3300183611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a28104eb0 msgr2=0x7f7a2819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.447+0000 7f7a2ea00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a28104eb0 0x7f7a2819b350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800e9c0 tx=0x7f7a1800ecd0 comp rx=0 tx=0).stop 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.448+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3300183611 shutdown_connections 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.448+0000 7f7a2ea00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7a0c0a3cc0 0x7f7a0c0a6180 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.448+0000 7f7a2ea00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a28109f90 0x7f7a2819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.448+0000 7f7a2ea00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a28104eb0 0x7f7a2819b350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.448+0000 7f7a2ea00700 1 --2- 172.21.15.112:0/3300183611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a28105860 0x7f7a2819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.448+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3300183611 >> 172.21.15.112:0/3300183611 conn(0x7f7a28100740 msgr2=0x7f7a28102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.448+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3300183611 shutdown_connections 2024-04-25T18:43:24.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:24.448+0000 7f7a2ea00700 1 -- 172.21.15.112:0/3300183611 wait complete. 2024-04-25T18:43:24.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:24 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3300183611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:43:24.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:24 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3300183611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:43:24.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:24 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3300183611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:43:25.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:43:25.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:25 smithi112 ceph-mon[27864]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T18:43:25.799 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:25 smithi171 ceph-mon[33266]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T18:43:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:25 smithi161 ceph-mon[32609]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T18:43:28.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.003+0000 7f1a3be00700 1 -- 172.21.15.112:0/2008870812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c1044f0 msgr2=0x7f1a3c10c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.003+0000 7f1a3be00700 1 --2- 172.21.15.112:0/2008870812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c1044f0 0x7f1a3c10c9e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009990 tx=0x7f1a28009ca0 comp rx=0 tx=0).stop 2024-04-25T18:43:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.003+0000 7f1a3be00700 1 -- 172.21.15.112:0/2008870812 shutdown_connections 2024-04-25T18:43:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.003+0000 7f1a3be00700 1 --2- 172.21.15.112:0/2008870812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a3c10cfb0 0x7f1a3c10f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.003+0000 7f1a3be00700 1 --2- 172.21.15.112:0/2008870812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a3c103bd0 0x7f1a3c103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.003+0000 7f1a3be00700 1 --2- 172.21.15.112:0/2008870812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c1044f0 0x7f1a3c10c9e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.003+0000 7f1a3be00700 1 -- 172.21.15.112:0/2008870812 >> 172.21.15.112:0/2008870812 conn(0x7f1a3c0fd360 msgr2=0x7f1a3c0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.003+0000 7f1a3be00700 1 -- 172.21.15.112:0/2008870812 shutdown_connections 2024-04-25T18:43:28.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.003+0000 7f1a3be00700 1 -- 172.21.15.112:0/2008870812 wait complete. 2024-04-25T18:43:28.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.004+0000 7f1a3be00700 1 Processor -- start 2024-04-25T18:43:28.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.004+0000 7f1a3be00700 1 -- start start 2024-04-25T18:43:28.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.005+0000 7f1a3be00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a3c103bd0 0x7f1a3c103420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:28.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.005+0000 7f1a3be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a3c1044f0 0x7f1a3c101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:28.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.005+0000 7f1a3be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c10cfb0 0x7f1a3c101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.005+0000 7f1a3be00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c112b40 con 0x7f1a3c10cfb0 2024-04-25T18:43:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.005+0000 7f1a3be00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1129c0 con 0x7f1a3c1044f0 2024-04-25T18:43:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.005+0000 7f1a3be00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c112cc0 con 0x7f1a3c103bd0 2024-04-25T18:43:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.005+0000 7f1a3b400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c10cfb0 0x7f1a3c101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.005+0000 7f1a3a000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a3c1044f0 0x7f1a3c101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.005+0000 7f1a3a000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a3c1044f0 0x7f1a3c101830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:37504/0 (socket says 172.21.15.112:37504) 2024-04-25T18:43:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.005+0000 7f1a3a000700 1 -- 172.21.15.112:0/1847310038 learned_addr learned my addr 172.21.15.112:0/1847310038 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:28.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.005+0000 7f1a3aa00700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a3c103bd0 0x7f1a3c103420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.006+0000 7f1a3a000700 1 -- 172.21.15.112:0/1847310038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a3c103bd0 msgr2=0x7f1a3c103420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.006+0000 7f1a3a000700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a3c103bd0 0x7f1a3c103420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.006+0000 7f1a3a000700 1 -- 172.21.15.112:0/1847310038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c10cfb0 msgr2=0x7f1a3c101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.006+0000 7f1a3a000700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c10cfb0 0x7f1a3c101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.006+0000 7f1a3a000700 1 -- 172.21.15.112:0/1847310038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c1aba40 con 0x7f1a3c1044f0 2024-04-25T18:43:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.006+0000 7f1a3b400700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c10cfb0 0x7f1a3c101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:43:28.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.006+0000 7f1a3a000700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a3c1044f0 0x7f1a3c101830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a28000c00 tx=0x7f1a28009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.007+0000 7f1a33400700 1 -- 172.21.15.112:0/1847310038 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28026070 con 0x7f1a3c1044f0 2024-04-25T18:43:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.007+0000 7f1a3be00700 1 -- 172.21.15.112:0/1847310038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c1abc70 con 0x7f1a3c1044f0 2024-04-25T18:43:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.007+0000 7f1a33400700 1 -- 172.21.15.112:0/1847310038 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a28021410 con 0x7f1a3c1044f0 2024-04-25T18:43:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.007+0000 7f1a33400700 1 -- 172.21.15.112:0/1847310038 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28017650 con 0x7f1a3c1044f0 2024-04-25T18:43:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.007+0000 7f1a3be00700 1 -- 172.21.15.112:0/1847310038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3c1ac150 con 0x7f1a3c1044f0 2024-04-25T18:43:28.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.008+0000 7f1a3be00700 1 -- 172.21.15.112:0/1847310038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3c06af60 con 0x7f1a3c1044f0 2024-04-25T18:43:28.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.010+0000 7f1a33400700 1 -- 172.21.15.112:0/1847310038 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1a28029070 con 0x7f1a3c1044f0 2024-04-25T18:43:28.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.011+0000 7f1a33400700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1a200a3c40 0x7f1a200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:28.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.011+0000 7f1a33400700 1 -- 172.21.15.112:0/1847310038 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f1a280d1240 con 0x7f1a3c1044f0 2024-04-25T18:43:28.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.011+0000 7f1a3aa00700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1a200a3c40 0x7f1a200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:28.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.012+0000 7f1a3aa00700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1a200a3c40 0x7f1a200a6100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009c60 tx=0x7f1a2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:28.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.013+0000 7f1a33400700 1 -- 172.21.15.112:0/1847310038 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a28030080 con 0x7f1a3c1044f0 2024-04-25T18:43:28.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.247+0000 7f1a3be00700 1 -- 172.21.15.112:0/1847310038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a3c02df80 con 0x7f1a3c1044f0 2024-04-25T18:43:28.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.249+0000 7f1a33400700 1 -- 172.21.15.112:0/1847310038 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22239 (secure 0 0 0) 0x7f1a28013070 con 0x7f1a3c1044f0 2024-04-25T18:43:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:43:28.252 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":92,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","created":"2024-04-25T18:33:37.379089+0000","modified":"2024-04-25T18:43:19.347501+0000","last_up_change":"2024-04-25T18:42:00.961646+0000","last_in_change":"2024-04-25T18:41:40.647162+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:37:45.106199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:43:11.017975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:43:15.116917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efb4bc01-e29b-4402-8a7a-3387d1731ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6803","nonce":232254535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6805","nonce":232254535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6809","nonce":232254535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6807","nonce":232254535}]},"public_addr":"172.21.15.112:6803/232254535","cluster_addr":"172.21.15.112:6805/232254535","heartbeat_back_addr":"172.21.15.112:6809/232254535","heartbeat_front_addr":"172.21.15.112:6807/232254535","state":["exists","up"]},{"osd":1,"uuid":"65a83fa2-96a6-4743-8c17-21531f5e885e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6801","nonce":664224744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6803","nonce":664224744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6807","nonce":664224744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6805","nonce":664224744}]},"public_addr":"172.21.15.161:6801/664224744","cluster_addr":"172.21.15.161:6803/664224744","heartbeat_back_addr":"172.21.15.161:6807/664224744","heartbeat_front_addr":"172.21.15.161:6805/664224744","state":["exists","up"]},{"osd":2,"uuid":"7562ec4c-c126-48ec-a51c-56fa461a2e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6801","nonce":345172073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6803","nonce":345172073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6807","nonce":345172073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6805","nonce":345172073}]},"public_addr":"172.21.15.171:6801/345172073","cluster_addr":"172.21.15.171:6803/345172073","heartbeat_back_addr":"172.21.15.171:6807/345172073","heartbeat_front_addr":"172.21.15.171:6805/345172073","state":["exists","up"]},{"osd":3,"uuid":"d7aae8cd-fec7-46d9-8773-25eaaca3cc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6811","nonce":1472407156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6813","nonce":1472407156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6817","nonce":1472407156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6815","nonce":1472407156}]},"public_addr":"172.21.15.112:6811/1472407156","cluster_addr":"172.21.15.112:6813/1472407156","heartbeat_back_addr":"172.21.15.112:6817/1472407156","heartbeat_front_addr":"172.21.15.112:6815/1472407156","state":["exists","up"]},{"osd":4,"uuid":"b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6809","nonce":373204306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6811","nonce":373204306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6815","nonce":373204306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6813","nonce":373204306}]},"public_addr":"172.21.15.161:6809/373204306","cluster_addr":"172.21.15.161:6811/373204306","heartbeat_back_addr":"172.21.15.161:6815/373204306","heartbeat_front_addr":"172.21.15.161:6813/373204306","state":["exists","up"]},{"osd":5,"uuid":"56124d12-bc1f-4cba-8430-938248851776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6809","nonce":2643512313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6811","nonce":2643512313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6815","nonce":2643512313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6813","nonce":2643512313}]},"public_addr":"172.21.15.171:6809/2643512313","cluster_addr":"172.21.15.171:6811/2643512313","heartbeat_back_addr":"172.21.15.171:6815/2643512313","heartbeat_front_addr":"172.21.15.171:6813/2643512313","state":["exists","up"]},{"osd":6,"uuid":"9563a9d3-352b-4686-975c-282ec167248b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6819","nonce":1933654933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6821","nonce":1933654933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6825","nonce":1933654933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6823","nonce":1933654933}]},"public_addr":"172.21.15.112:6819/1933654933","cluster_addr":"172.21.15.112:6821/1933654933","heartbeat_back_addr":"172.21.15.112:6825/1933654933","heartbeat_front_addr":"172.21.15.112:6823/1933654933","state":["exists","up"]},{"osd":7,"uuid":"f2d06e86-6aac-40f4-956e-18fe0b0e3543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6817","nonce":3060201138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6819","nonce":3060201138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6823","nonce":3060201138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6821","nonce":3060201138}]},"public_addr":"172.21.15.161:6817/3060201138","cluster_addr":"172.21.15.161:6819/3060201138","heartbeat_back_addr":"172.21.15.161:6823/3060201138","heartbeat_front_addr":"172.21.15.161:6821/3060201138","state":["exists","up"]},{"osd":8,"uuid":"f006e45f-2bb9-45d0-9020-be7c33c6a825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6817","nonce":835756571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6819","nonce":835756571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6823","nonce":835756571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6821","nonce":835756571}]},"public_addr":"172.21.15.171:6817/835756571","cluster_addr":"172.21.15.171:6819/835756571","heartbeat_back_addr":"172.21.15.171:6823/835756571","heartbeat_front_addr":"172.21.15.171:6821/835756571","state":["exists","up"]},{"osd":9,"uuid":"c5a96491-4151-4633-b3bc-2c214fe51c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6827","nonce":349758528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6829","nonce":349758528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6833","nonce":349758528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6831","nonce":349758528}]},"public_addr":"172.21.15.112:6827/349758528","cluster_addr":"172.21.15.112:6829/349758528","heartbeat_back_addr":"172.21.15.112:6833/349758528","heartbeat_front_addr":"172.21.15.112:6831/349758528","state":["exists","up"]},{"osd":10,"uuid":"244eda3e-25a0-490e-879f-c1e2726c1be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6825","nonce":3013292765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6827","nonce":3013292765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6831","nonce":3013292765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6829","nonce":3013292765}]},"public_addr":"172.21.15.161:6825/3013292765","cluster_addr":"172.21.15.161:6827/3013292765","heartbeat_back_addr":"172.21.15.161:6831/3013292765","heartbeat_front_addr":"172.21.15.161:6829/3013292765","state":["exists","up"]},{"osd":11,"uuid":"356e5c54-bbc3-45ee-bda2-85164ce3cb92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6825","nonce":1010336458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6827","nonce":1010336458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6831","nonce":1010336458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6829","nonce":1010336458}]},"public_addr":"172.21.15.171:6825/1010336458","cluster_addr":"172.21.15.171:6827/1010336458","heartbeat_back_addr":"172.21.15.171:6831/1010336458","heartbeat_front_addr":"172.21.15.171:6829/1010336458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:36:51.024650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:17.026638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:41.239726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:15.148968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:41.029118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:06.225858+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:41.639444+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:07.753400+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:32.292085+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:07.159926+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:33.749782+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:58.594447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/181862246":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/3393454453":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3195850527":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6801/3468460974":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3392048544":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/1329430933":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/735578423":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/3468460974":"2024-04-26T18:34:20.605775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:43:28.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.252+0000 7f1a3be00700 1 -- 172.21.15.112:0/1847310038 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1a200a3c40 msgr2=0x7f1a200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:28.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.252+0000 7f1a3be00700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1a200a3c40 0x7f1a200a6100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009c60 tx=0x7f1a2c009260 comp rx=0 tx=0).stop 2024-04-25T18:43:28.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.252+0000 7f1a3be00700 1 -- 172.21.15.112:0/1847310038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a3c1044f0 msgr2=0x7f1a3c101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:28.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.252+0000 7f1a3be00700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a3c1044f0 0x7f1a3c101830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a28000c00 tx=0x7f1a28009510 comp rx=0 tx=0).stop 2024-04-25T18:43:28.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.253+0000 7f1a3be00700 1 -- 172.21.15.112:0/1847310038 shutdown_connections 2024-04-25T18:43:28.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.253+0000 7f1a3be00700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1a200a3c40 0x7f1a200a6100 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:28.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.253+0000 7f1a3be00700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c10cfb0 0x7f1a3c101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:28.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.253+0000 7f1a3be00700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a3c103bd0 0x7f1a3c103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:28.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.253+0000 7f1a3be00700 1 --2- 172.21.15.112:0/1847310038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a3c1044f0 0x7f1a3c101830 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:28.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.253+0000 7f1a3be00700 1 -- 172.21.15.112:0/1847310038 >> 172.21.15.112:0/1847310038 conn(0x7f1a3c0fd360 msgr2=0x7f1a3c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:28.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.253+0000 7f1a3be00700 1 -- 172.21.15.112:0/1847310038 shutdown_connections 2024-04-25T18:43:28.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:28.253+0000 7f1a3be00700 1 -- 172.21.15.112:0/1847310038 wait complete. 2024-04-25T18:43:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:28 smithi171 ceph-mon[33266]: pgmap v338: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:43:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:28 smithi161 ceph-mon[32609]: pgmap v338: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:43:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:28 smithi112 ceph-mon[27864]: pgmap v338: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:43:29.134 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-25T18:43:29.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:29 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1847310038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:43:29.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:29 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1847310038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:43:29.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:29 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1847310038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:43:29.583 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:30.270 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:30 smithi112 ceph-mon[27864]: pgmap v339: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:43:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:30 smithi171 ceph-mon[33266]: pgmap v339: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:43:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:30 smithi161 ceph-mon[32609]: pgmap v339: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:43:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.910+0000 7fc6ae800700 1 -- 172.21.15.112:0/755754674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6a8105870 msgr2=0x7fc6a81098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.910+0000 7fc6ae800700 1 --2- 172.21.15.112:0/755754674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6a8105870 0x7fc6a81098c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc6900099c0 tx=0x7fc690009cd0 comp rx=0 tx=0).stop 2024-04-25T18:43:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.910+0000 7fc6ae800700 1 -- 172.21.15.112:0/755754674 shutdown_connections 2024-04-25T18:43:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.910+0000 7fc6ae800700 1 --2- 172.21.15.112:0/755754674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a8109fa0 0x7fc6a8111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.910+0000 7fc6ae800700 1 --2- 172.21.15.112:0/755754674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a8104ec0 0x7fc6a81052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.910+0000 7fc6ae800700 1 --2- 172.21.15.112:0/755754674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6a8105870 0x7fc6a81098c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.910+0000 7fc6ae800700 1 -- 172.21.15.112:0/755754674 >> 172.21.15.112:0/755754674 conn(0x7fc6a8100750 msgr2=0x7fc6a8102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.911+0000 7fc6ae800700 1 -- 172.21.15.112:0/755754674 shutdown_connections 2024-04-25T18:43:31.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.911+0000 7fc6ae800700 1 -- 172.21.15.112:0/755754674 wait complete. 2024-04-25T18:43:31.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.912+0000 7fc6ae800700 1 Processor -- start 2024-04-25T18:43:31.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.912+0000 7fc6ae800700 1 -- start start 2024-04-25T18:43:31.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.913+0000 7fc6ae800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6a8104ec0 0x7fc6a819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:31.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.913+0000 7fc6ae800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a8105870 0x7fc6a819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:31.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.913+0000 7fc6ae800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a8109fa0 0x7fc6a819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.913+0000 7fc6ae800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8114e10 con 0x7fc6a8105870 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.913+0000 7fc6ae800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8114c90 con 0x7fc6a8109fa0 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.913+0000 7fc6ae800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8114f90 con 0x7fc6a8104ec0 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.913+0000 7fc6a7400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6a8104ec0 0x7fc6a819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.913+0000 7fc69fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a8105870 0x7fc6a819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.913+0000 7fc6a7400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6a8104ec0 0x7fc6a819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:53394/0 (socket says 172.21.15.112:53394) 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.913+0000 7fc6a7400700 1 -- 172.21.15.112:0/2989552964 learned_addr learned my addr 172.21.15.112:0/2989552964 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.913+0000 7fc6a7e00700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a8109fa0 0x7fc6a819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.914+0000 7fc6a7e00700 1 -- 172.21.15.112:0/2989552964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6a8104ec0 msgr2=0x7fc6a819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.914+0000 7fc6a7e00700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6a8104ec0 0x7fc6a819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.914+0000 7fc6a7e00700 1 -- 172.21.15.112:0/2989552964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a8105870 msgr2=0x7fc6a819b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:31.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.914+0000 7fc6a7e00700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a8105870 0x7fc6a819b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:31.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.914+0000 7fc6a7e00700 1 -- 172.21.15.112:0/2989552964 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a81a0330 con 0x7fc6a8109fa0 2024-04-25T18:43:31.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.914+0000 7fc6a7e00700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a8109fa0 0x7fc6a819fbf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0007f10 tx=0x7fc6a000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:31.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.914+0000 7fc6a4c00700 1 -- 172.21.15.112:0/2989552964 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0004db0 con 0x7fc6a8109fa0 2024-04-25T18:43:31.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.914+0000 7fc6a4c00700 1 -- 172.21.15.112:0/2989552964 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a0003a00 con 0x7fc6a8109fa0 2024-04-25T18:43:31.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.914+0000 7fc6ae800700 1 -- 172.21.15.112:0/2989552964 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a81a0620 con 0x7fc6a8109fa0 2024-04-25T18:43:31.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.915+0000 7fc6a4c00700 1 -- 172.21.15.112:0/2989552964 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0004580 con 0x7fc6a8109fa0 2024-04-25T18:43:31.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.915+0000 7fc6ae800700 1 -- 172.21.15.112:0/2989552964 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a81a7ea0 con 0x7fc6a8109fa0 2024-04-25T18:43:31.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.916+0000 7fc6ae800700 1 -- 172.21.15.112:0/2989552964 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a8069730 con 0x7fc6a8109fa0 2024-04-25T18:43:31.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.918+0000 7fc6a4c00700 1 -- 172.21.15.112:0/2989552964 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc6a001c020 con 0x7fc6a8109fa0 2024-04-25T18:43:31.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.919+0000 7fc6a4c00700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc68c0a3cc0 0x7fc68c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:31.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.919+0000 7fc6a4c00700 1 -- 172.21.15.112:0/2989552964 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7fc6a00cfc70 con 0x7fc6a8109fa0 2024-04-25T18:43:31.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.919+0000 7fc6a7400700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc68c0a3cc0 0x7fc68c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:31.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.920+0000 7fc6a7400700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc68c0a3cc0 0x7fc68c0a6180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc698009d20 tx=0x7fc698009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:31.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:31.922+0000 7fc6a4c00700 1 -- 172.21.15.112:0/2989552964 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6a0010070 con 0x7fc6a8109fa0 2024-04-25T18:43:32.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:32.195+0000 7fc6ae800700 1 -- 172.21.15.112:0/2989552964 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc6a802e140 con 0x7fc6a8109fa0 2024-04-25T18:43:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:32 smithi171 ceph-mon[33266]: pgmap v340: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-25T18:43:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:32 smithi161 ceph-mon[32609]: pgmap v340: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-25T18:43:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:32 smithi112 ceph-mon[27864]: pgmap v340: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-25T18:43:33.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.189+0000 7fc6a4c00700 1 -- 172.21.15.112:0/2989552964 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fc6a009c700 con 0x7fc6a8109fa0 2024-04-25T18:43:33.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 -- 172.21.15.112:0/2989552964 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc68c0a3cc0 msgr2=0x7fc68c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:33.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc68c0a3cc0 0x7fc68c0a6180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc698009d20 tx=0x7fc698009260 comp rx=0 tx=0).stop 2024-04-25T18:43:33.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 -- 172.21.15.112:0/2989552964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a8109fa0 msgr2=0x7fc6a819fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:33.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a8109fa0 0x7fc6a819fbf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0007f10 tx=0x7fc6a000d3b0 comp rx=0 tx=0).stop 2024-04-25T18:43:33.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 -- 172.21.15.112:0/2989552964 shutdown_connections 2024-04-25T18:43:33.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc68c0a3cc0 0x7fc68c0a6180 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:33.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a8109fa0 0x7fc6a819fbf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:33.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6a8104ec0 0x7fc6a819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:33.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 --2- 172.21.15.112:0/2989552964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6a8105870 0x7fc6a819b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:33.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 -- 172.21.15.112:0/2989552964 >> 172.21.15.112:0/2989552964 conn(0x7fc6a8100750 msgr2=0x7fc6a8102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:33.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 -- 172.21.15.112:0/2989552964 shutdown_connections 2024-04-25T18:43:33.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:33.193+0000 7fc6ae800700 1 -- 172.21.15.112:0/2989552964 wait complete. 2024-04-25T18:43:33.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:33 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2989552964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:43:33.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:33 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:43:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:33 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2989552964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:43:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:33 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:43:33.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:33 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2989552964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:43:33.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:33 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:43:34.078 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T18:43:34.078 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-04-25T18:43:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:34 smithi161 ceph-mon[32609]: pgmap v341: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T18:43:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:34 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T18:43:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:34 smithi161 ceph-mon[32609]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:34.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:34 smithi112 ceph-mon[27864]: pgmap v341: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T18:43:34.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:34 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T18:43:34.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:34 smithi112 ceph-mon[27864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:34.518 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:34.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:34 smithi171 ceph-mon[33266]: pgmap v341: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T18:43:34.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:34 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T18:43:34.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:34 smithi171 ceph-mon[33266]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:36.378 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:36 smithi112 ceph-mon[27864]: pgmap v342: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-04-25T18:43:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:36 smithi161 ceph-mon[32609]: pgmap v342: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-04-25T18:43:36.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:36 smithi171 ceph-mon[33266]: pgmap v342: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-04-25T18:43:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.894+0000 7f659e000700 1 -- 172.21.15.112:0/3305513818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598104ec0 msgr2=0x7f65981052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.894+0000 7f659e000700 1 --2- 172.21.15.112:0/3305513818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598104ec0 0x7f65981052a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f65880099c0 tx=0x7f6588009cd0 comp rx=0 tx=0).stop 2024-04-25T18:43:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.894+0000 7f659e000700 1 -- 172.21.15.112:0/3305513818 shutdown_connections 2024-04-25T18:43:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.894+0000 7f659e000700 1 --2- 172.21.15.112:0/3305513818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6598109fa0 0x7f6598111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.894+0000 7f659e000700 1 --2- 172.21.15.112:0/3305513818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598104ec0 0x7f65981052a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.894+0000 7f659e000700 1 --2- 172.21.15.112:0/3305513818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6598105870 0x7f65981098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.894+0000 7f659e000700 1 -- 172.21.15.112:0/3305513818 >> 172.21.15.112:0/3305513818 conn(0x7f6598100750 msgr2=0x7f6598102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:36.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.894+0000 7f659e000700 1 -- 172.21.15.112:0/3305513818 shutdown_connections 2024-04-25T18:43:36.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.894+0000 7f659e000700 1 -- 172.21.15.112:0/3305513818 wait complete. 2024-04-25T18:43:36.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.895+0000 7f659e000700 1 Processor -- start 2024-04-25T18:43:36.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.895+0000 7f659e000700 1 -- start start 2024-04-25T18:43:36.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.895+0000 7f659e000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6598104ec0 0x7f6598076f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:36.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.895+0000 7f659e000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6598105870 0x7f6598077490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:36.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f659e000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598109fa0 0x7f65980779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:36.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f659e000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6598114e30 con 0x7f6598109fa0 2024-04-25T18:43:36.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f659e000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6598114cb0 con 0x7f6598105870 2024-04-25T18:43:36.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f659e000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6598114fb0 con 0x7f6598104ec0 2024-04-25T18:43:36.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f6597400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6598104ec0 0x7f6598076f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:36.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f6597e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598109fa0 0x7f65980779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:36.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f6597e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598109fa0 0x7f65980779d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56614/0 (socket says 172.21.15.112:56614) 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f6597400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6598104ec0 0x7f6598076f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:53410/0 (socket says 172.21.15.112:53410) 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f6597e00700 1 -- 172.21.15.112:0/340088146 learned_addr learned my addr 172.21.15.112:0/340088146 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f6596a00700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6598105870 0x7f6598077490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f6596a00700 1 -- 172.21.15.112:0/340088146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6598104ec0 msgr2=0x7f6598076f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f6596a00700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6598104ec0 0x7f6598076f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f6596a00700 1 -- 172.21.15.112:0/340088146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598109fa0 msgr2=0x7f65980779d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f6596a00700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598109fa0 0x7f65980779d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.896+0000 7f6596a00700 1 -- 172.21.15.112:0/340088146 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65981ab9d0 con 0x7f6598105870 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.897+0000 7f6596a00700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6598105870 0x7f6598077490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f658400e9f0 tx=0x7f658400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.897+0000 7f658fe00700 1 -- 172.21.15.112:0/340088146 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658400c9c0 con 0x7f6598105870 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.897+0000 7f659e000700 1 -- 172.21.15.112:0/340088146 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65981abcc0 con 0x7f6598105870 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.897+0000 7f658fe00700 1 -- 172.21.15.112:0/340088146 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6584010430 con 0x7f6598105870 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.897+0000 7f658fe00700 1 -- 172.21.15.112:0/340088146 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584018680 con 0x7f6598105870 2024-04-25T18:43:36.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.897+0000 7f659e000700 1 -- 172.21.15.112:0/340088146 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65981ac1b0 con 0x7f6598105870 2024-04-25T18:43:36.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.898+0000 7f659e000700 1 -- 172.21.15.112:0/340088146 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65981063a0 con 0x7f6598105870 2024-04-25T18:43:36.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.900+0000 7f658fe00700 1 -- 172.21.15.112:0/340088146 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f65840108e0 con 0x7f6598105870 2024-04-25T18:43:36.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.901+0000 7f658fe00700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f657c0a3c40 0x7f657c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:36.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.902+0000 7f658fe00700 1 -- 172.21.15.112:0/340088146 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f65840d08c0 con 0x7f6598105870 2024-04-25T18:43:36.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.902+0000 7f6597400700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f657c0a3c40 0x7f657c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:36.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.902+0000 7f6597400700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f657c0a3c40 0x7f657c0a6100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6588000c00 tx=0x7f6588005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:36.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:36.904+0000 7f658fe00700 1 -- 172.21.15.112:0/340088146 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f658409d350 con 0x7f6598105870 2024-04-25T18:43:37.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.175+0000 7f659e000700 1 -- 172.21.15.112:0/340088146 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f659802df80 con 0x7f6598105870 2024-04-25T18:43:37.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.199+0000 7f658fe00700 1 -- 172.21.15.112:0/340088146 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f658409cc50 con 0x7f6598105870 2024-04-25T18:43:37.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.202+0000 7f659e000700 1 -- 172.21.15.112:0/340088146 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f657c0a3c40 msgr2=0x7f657c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:37.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.202+0000 7f659e000700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f657c0a3c40 0x7f657c0a6100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6588000c00 tx=0x7f6588005fb0 comp rx=0 tx=0).stop 2024-04-25T18:43:37.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.202+0000 7f659e000700 1 -- 172.21.15.112:0/340088146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6598105870 msgr2=0x7f6598077490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:37.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.202+0000 7f659e000700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6598105870 0x7f6598077490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f658400e9f0 tx=0x7f658400ed00 comp rx=0 tx=0).stop 2024-04-25T18:43:37.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.202+0000 7f659e000700 1 -- 172.21.15.112:0/340088146 shutdown_connections 2024-04-25T18:43:37.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.202+0000 7f659e000700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f657c0a3c40 0x7f657c0a6100 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:37.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.202+0000 7f659e000700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6598109fa0 0x7f65980779d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:37.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.202+0000 7f659e000700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6598104ec0 0x7f6598076f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:37.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.202+0000 7f659e000700 1 --2- 172.21.15.112:0/340088146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6598105870 0x7f6598077490 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:37.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.202+0000 7f659e000700 1 -- 172.21.15.112:0/340088146 >> 172.21.15.112:0/340088146 conn(0x7f6598100750 msgr2=0x7f6598109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:37.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.202+0000 7f659e000700 1 -- 172.21.15.112:0/340088146 shutdown_connections 2024-04-25T18:43:37.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:37.203+0000 7f659e000700 1 -- 172.21.15.112:0/340088146 wait complete. 2024-04-25T18:43:37.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:37 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/340088146' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:43:37.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:37 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:43:37.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:37 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/340088146' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:43:37.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:37 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:43:37.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:37 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/340088146' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:43:37.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:37 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:43:38.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-25T18:43:38.457 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:38 smithi112 ceph-mon[27864]: pgmap v343: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:38 smithi112 ceph-mon[27864]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T18:43:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:38 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T18:43:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:38 smithi112 ceph-mon[27864]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T18:43:38.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:38 smithi112 ceph-mon[27864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:38.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:38 smithi112 ceph-mon[27864]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-25T18:43:38.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:38 smithi112 ceph-mon[27864]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-25T18:43:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:38 smithi171 ceph-mon[33266]: pgmap v343: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:38 smithi171 ceph-mon[33266]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T18:43:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:38 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T18:43:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:38 smithi171 ceph-mon[33266]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T18:43:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:38 smithi171 ceph-mon[33266]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:38 smithi171 ceph-mon[33266]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-25T18:43:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:38 smithi171 ceph-mon[33266]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-25T18:43:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:38 smithi161 ceph-mon[32609]: pgmap v343: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:43:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:38 smithi161 ceph-mon[32609]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T18:43:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:38 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T18:43:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:38 smithi161 ceph-mon[32609]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T18:43:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:38 smithi161 ceph-mon[32609]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:43:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:38 smithi161 ceph-mon[32609]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-25T18:43:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:38 smithi161 ceph-mon[32609]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-25T18:43:39.522 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:39 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] up:active 2024-04-25T18:43:39.522 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:39 smithi112 ceph-mon[27864]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T18:43:39.522 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:39 smithi112 ceph-mon[27864]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T18:43:39.522 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:39 smithi112 ceph-mon[27864]: Cluster is now healthy 2024-04-25T18:43:39.522 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:39 smithi112 ceph-mon[27864]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-25T18:43:39.522 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:39 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-25T18:43:39.523 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:39 smithi112 ceph-mon[27864]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T18:43:39.523 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:39 smithi112 ceph-mon[27864]: pgmap v344: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 2 op/s 2024-04-25T18:43:39.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:39 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] up:active 2024-04-25T18:43:39.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:39 smithi171 ceph-mon[33266]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T18:43:39.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:39 smithi171 ceph-mon[33266]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T18:43:39.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:39 smithi171 ceph-mon[33266]: Cluster is now healthy 2024-04-25T18:43:39.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:39 smithi171 ceph-mon[33266]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-25T18:43:39.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:39 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-25T18:43:39.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:39 smithi171 ceph-mon[33266]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T18:43:39.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:39 smithi171 ceph-mon[33266]: pgmap v344: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 2 op/s 2024-04-25T18:43:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:39 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] up:active 2024-04-25T18:43:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:39 smithi161 ceph-mon[32609]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T18:43:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:39 smithi161 ceph-mon[32609]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T18:43:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:39 smithi161 ceph-mon[32609]: Cluster is now healthy 2024-04-25T18:43:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:39 smithi161 ceph-mon[32609]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-25T18:43:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:39 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-25T18:43:39.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:39 smithi161 ceph-mon[32609]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T18:43:39.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:39 smithi161 ceph-mon[32609]: pgmap v344: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 2 op/s 2024-04-25T18:43:40.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:40 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] up:active 2024-04-25T18:43:40.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:40 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:43:40.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:40 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] up:active 2024-04-25T18:43:40.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:40 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:43:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.681+0000 7f9b1c600700 1 -- 172.21.15.112:0/4007407117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b14102a90 msgr2=0x7f9b14102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.681+0000 7f9b1c600700 1 --2- 172.21.15.112:0/4007407117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b14102a90 0x7f9b14102f10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009990 tx=0x7f9b00009ca0 comp rx=0 tx=0).stop 2024-04-25T18:43:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.681+0000 7f9b1c600700 1 -- 172.21.15.112:0/4007407117 shutdown_connections 2024-04-25T18:43:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.681+0000 7f9b1c600700 1 --2- 172.21.15.112:0/4007407117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b14103450 0x7f9b1410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.681+0000 7f9b1c600700 1 --2- 172.21.15.112:0/4007407117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b14108a50 0x7f9b14108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.681+0000 7f9b1c600700 1 --2- 172.21.15.112:0/4007407117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b14102a90 0x7f9b14102f10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.681+0000 7f9b1c600700 1 -- 172.21.15.112:0/4007407117 >> 172.21.15.112:0/4007407117 conn(0x7f9b14079ba0 msgr2=0x7f9b14079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.681+0000 7f9b1c600700 1 -- 172.21.15.112:0/4007407117 shutdown_connections 2024-04-25T18:43:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.681+0000 7f9b1c600700 1 -- 172.21.15.112:0/4007407117 wait complete. 2024-04-25T18:43:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.683+0000 7f9b1c600700 1 Processor -- start 2024-04-25T18:43:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.683+0000 7f9b1c600700 1 -- start start 2024-04-25T18:43:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.683+0000 7f9b1c600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b14102a90 0x7f9b1419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.683+0000 7f9b1c600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b14103450 0x7f9b1419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.683+0000 7f9b1c600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b14108a50 0x7f9b1419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.683+0000 7f9b1c600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b14112be0 con 0x7f9b14102a90 2024-04-25T18:43:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.683+0000 7f9b1c600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9b14112a60 con 0x7f9b14103450 2024-04-25T18:43:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.683+0000 7f9b1c600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9b14112d60 con 0x7f9b14108a50 2024-04-25T18:43:40.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.684+0000 7f9b19c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b14102a90 0x7f9b1419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:40.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.684+0000 7f9b19200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b14103450 0x7f9b1419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:40.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.684+0000 7f9b19200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b14103450 0x7f9b1419b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:59550/0 (socket says 172.21.15.112:59550) 2024-04-25T18:43:40.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.684+0000 7f9b19200700 1 -- 172.21.15.112:0/3260346103 learned_addr learned my addr 172.21.15.112:0/3260346103 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:40.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.684+0000 7f9b1a600700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b14108a50 0x7f9b1419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:40.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.684+0000 7f9b19200700 1 -- 172.21.15.112:0/3260346103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b14108a50 msgr2=0x7f9b1419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:40.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.684+0000 7f9b19200700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b14108a50 0x7f9b1419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:40.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.684+0000 7f9b19200700 1 -- 172.21.15.112:0/3260346103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b14102a90 msgr2=0x7f9b1419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:40.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.684+0000 7f9b19200700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b14102a90 0x7f9b1419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:40.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.684+0000 7f9b19200700 1 -- 172.21.15.112:0/3260346103 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b141a0360 con 0x7f9b14103450 2024-04-25T18:43:40.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.684+0000 7f9b19c00700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b14102a90 0x7f9b1419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:43:40.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.685+0000 7f9b19200700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b14103450 0x7f9b1419b890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9b000057c0 tx=0x7f9b00009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:40.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.685+0000 7f9b12000700 1 -- 172.21.15.112:0/3260346103 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00026070 con 0x7f9b14103450 2024-04-25T18:43:40.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.685+0000 7f9b12000700 1 -- 172.21.15.112:0/3260346103 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b00021410 con 0x7f9b14103450 2024-04-25T18:43:40.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.685+0000 7f9b12000700 1 -- 172.21.15.112:0/3260346103 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00017650 con 0x7f9b14103450 2024-04-25T18:43:40.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.685+0000 7f9b1c600700 1 -- 172.21.15.112:0/3260346103 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b141a05f0 con 0x7f9b14103450 2024-04-25T18:43:40.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.685+0000 7f9b1c600700 1 -- 172.21.15.112:0/3260346103 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b141a7ed0 con 0x7f9b14103450 2024-04-25T18:43:40.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.686+0000 7f9b1c600700 1 -- 172.21.15.112:0/3260346103 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b14069730 con 0x7f9b14103450 2024-04-25T18:43:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.688+0000 7f9b12000700 1 -- 172.21.15.112:0/3260346103 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9b00029070 con 0x7f9b14103450 2024-04-25T18:43:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.689+0000 7f9b12000700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9af80a3cc0 0x7f9af80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.690+0000 7f9b12000700 1 -- 172.21.15.112:0/3260346103 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f9b000d1140 con 0x7f9b14103450 2024-04-25T18:43:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.690+0000 7f9b19c00700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9af80a3cc0 0x7f9af80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.690+0000 7f9b19c00700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9af80a3cc0 0x7f9af80a6180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009480 tx=0x7f9b0400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:40.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.692+0000 7f9b12000700 1 -- 172.21.15.112:0/3260346103 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b00013070 con 0x7f9b14103450 2024-04-25T18:43:40.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:40 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] up:active 2024-04-25T18:43:40.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:40 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:43:40.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:40.967+0000 7f9b1c600700 1 -- 172.21.15.112:0/3260346103 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f9b1402df80 con 0x7f9b14103450 2024-04-25T18:43:41.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.250+0000 7f9b12000700 1 -- 172.21.15.112:0/3260346103 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f9b0009dcf0 con 0x7f9b14103450 2024-04-25T18:43:41.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.253+0000 7f9b1c600700 1 -- 172.21.15.112:0/3260346103 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9af80a3cc0 msgr2=0x7f9af80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:41.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.253+0000 7f9b1c600700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9af80a3cc0 0x7f9af80a6180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009480 tx=0x7f9b0400b040 comp rx=0 tx=0).stop 2024-04-25T18:43:41.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.253+0000 7f9b1c600700 1 -- 172.21.15.112:0/3260346103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b14103450 msgr2=0x7f9b1419b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:41.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.253+0000 7f9b1c600700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b14103450 0x7f9b1419b890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9b000057c0 tx=0x7f9b00009580 comp rx=0 tx=0).stop 2024-04-25T18:43:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.254+0000 7f9b1c600700 1 -- 172.21.15.112:0/3260346103 shutdown_connections 2024-04-25T18:43:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.254+0000 7f9b1c600700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9af80a3cc0 0x7f9af80a6180 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.254+0000 7f9b1c600700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b14108a50 0x7f9b1419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.254+0000 7f9b1c600700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b14102a90 0x7f9b1419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.254+0000 7f9b1c600700 1 --2- 172.21.15.112:0/3260346103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b14103450 0x7f9b1419b890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.254+0000 7f9b1c600700 1 -- 172.21.15.112:0/3260346103 >> 172.21.15.112:0/3260346103 conn(0x7f9b14079ba0 msgr2=0x7f9b1410df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:41.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.254+0000 7f9b1c600700 1 -- 172.21.15.112:0/3260346103 shutdown_connections 2024-04-25T18:43:41.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:41.254+0000 7f9b1c600700 1 -- 172.21.15.112:0/3260346103 wait complete. 2024-04-25T18:43:41.660 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:41 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3260346103' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:43:41.660 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:41 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:43:41.660 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:41 smithi112 ceph-mon[27864]: pgmap v345: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-04-25T18:43:41.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:41 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3260346103' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:43:41.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:41 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:43:41.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:41 smithi171 ceph-mon[33266]: pgmap v345: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-04-25T18:43:41.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:41 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3260346103' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:43:41.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:41 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:43:41.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:41 smithi161 ceph-mon[32609]: pgmap v345: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-04-25T18:43:42.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-25T18:43:42.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:42 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T18:43:42.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:42 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:43:42.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:42 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:43:42.560 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:42.583 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:42 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T18:43:42.583 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:42 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:43:42.583 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:42 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:43:42.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:42 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T18:43:42.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:42 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:43:42.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:42 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:43:43.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:43 smithi112 ceph-mon[27864]: pgmap v346: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T18:43:43.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:43 smithi171 ceph-mon[33266]: pgmap v346: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T18:43:43.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:43 smithi161 ceph-mon[32609]: pgmap v346: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T18:43:44.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.924+0000 7f2775200700 1 -- 172.21.15.112:0/1277778919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27701092d0 msgr2=0x7f27701096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:44.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.924+0000 7f2775200700 1 --2- 172.21.15.112:0/1277778919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27701092d0 0x7f27701096b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f27600099c0 tx=0x7f2760009cd0 comp rx=0 tx=0).stop 2024-04-25T18:43:44.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.924+0000 7f2775200700 1 -- 172.21.15.112:0/1277778919 shutdown_connections 2024-04-25T18:43:44.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.924+0000 7f2775200700 1 --2- 172.21.15.112:0/1277778919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f277010e3b0 0x7f2770115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:44.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.924+0000 7f2775200700 1 --2- 172.21.15.112:0/1277778919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27701092d0 0x7f27701096b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:44.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.924+0000 7f2775200700 1 --2- 172.21.15.112:0/1277778919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2770109c80 0x7f277010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:44.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.924+0000 7f2775200700 1 -- 172.21.15.112:0/1277778919 >> 172.21.15.112:0/1277778919 conn(0x7f2770076210 msgr2=0x7f2770107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:44.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.924+0000 7f2775200700 1 -- 172.21.15.112:0/1277778919 shutdown_connections 2024-04-25T18:43:44.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.924+0000 7f2775200700 1 -- 172.21.15.112:0/1277778919 wait complete. 2024-04-25T18:43:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.925+0000 7f2775200700 1 Processor -- start 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f2775200700 1 -- start start 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f2775200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27701092d0 0x7f277019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f2775200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770109c80 0x7f277019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f2775200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f277010e3b0 0x7f27701a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f2775200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2770119240 con 0x7f277010e3b0 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f2775200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f27701190c0 con 0x7f2770109c80 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f2775200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f27701193c0 con 0x7f27701092d0 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f276de00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27701092d0 0x7f277019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f276e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f277010e3b0 0x7f27701a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f276d400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770109c80 0x7f277019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f276de00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27701092d0 0x7f277019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:53172/0 (socket says 172.21.15.112:53172) 2024-04-25T18:43:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.926+0000 7f276de00700 1 -- 172.21.15.112:0/15010172 learned_addr learned my addr 172.21.15.112:0/15010172 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.927+0000 7f276de00700 1 -- 172.21.15.112:0/15010172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770109c80 msgr2=0x7f277019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.927+0000 7f276de00700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770109c80 0x7f277019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.927+0000 7f276de00700 1 -- 172.21.15.112:0/15010172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f277010e3b0 msgr2=0x7f27701a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.927+0000 7f276de00700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f277010e3b0 0x7f27701a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.927+0000 7f276de00700 1 -- 172.21.15.112:0/15010172 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27701a4720 con 0x7f27701092d0 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.927+0000 7f276d400700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770109c80 0x7f277019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.927+0000 7f276de00700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27701092d0 0x7f277019f710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2760006010 tx=0x7f2760005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.928+0000 7f2766a00700 1 -- 172.21.15.112:0/15010172 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760026070 con 0x7f27701092d0 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.928+0000 7f2766a00700 1 -- 172.21.15.112:0/15010172 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2760005680 con 0x7f27701092d0 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.928+0000 7f2775200700 1 -- 172.21.15.112:0/15010172 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27701a49b0 con 0x7f27701092d0 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.928+0000 7f2766a00700 1 -- 172.21.15.112:0/15010172 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276000fb20 con 0x7f27701092d0 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.928+0000 7f2775200700 1 -- 172.21.15.112:0/15010172 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27701ac290 con 0x7f27701092d0 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.929+0000 7f2775200700 1 -- 172.21.15.112:0/15010172 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2770069730 con 0x7f27701092d0 2024-04-25T18:43:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.931+0000 7f2766a00700 1 -- 172.21.15.112:0/15010172 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2760029070 con 0x7f27701092d0 2024-04-25T18:43:44.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.933+0000 7f2766a00700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27540a3d40 0x7f27540a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:44.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.933+0000 7f2766a00700 1 -- 172.21.15.112:0/15010172 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f27600d10f0 con 0x7f27701092d0 2024-04-25T18:43:44.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.933+0000 7f276d400700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27540a3d40 0x7f27540a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:44.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.934+0000 7f276d400700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27540a3d40 0x7f27540a6200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f275c009680 tx=0x7f275c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:44.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:44.935+0000 7f2766a00700 1 -- 172.21.15.112:0/15010172 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2760030020 con 0x7f27701092d0 2024-04-25T18:43:45.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:45.209+0000 7f2775200700 1 -- 172.21.15.112:0/15010172 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f27701a0a70 con 0x7f27701092d0 2024-04-25T18:43:46.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.176+0000 7f2766a00700 1 -- 172.21.15.112:0/15010172 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f2760013070 con 0x7f27701092d0 2024-04-25T18:43:46.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 -- 172.21.15.112:0/15010172 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27540a3d40 msgr2=0x7f27540a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:46.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27540a3d40 0x7f27540a6200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f275c009680 tx=0x7f275c009260 comp rx=0 tx=0).stop 2024-04-25T18:43:46.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 -- 172.21.15.112:0/15010172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27701092d0 msgr2=0x7f277019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:46.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27701092d0 0x7f277019f710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2760006010 tx=0x7f2760005280 comp rx=0 tx=0).stop 2024-04-25T18:43:46.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 -- 172.21.15.112:0/15010172 shutdown_connections 2024-04-25T18:43:46.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27540a3d40 0x7f27540a6200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:46.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f277010e3b0 0x7f27701a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:46.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27701092d0 0x7f277019f710 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:46.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 --2- 172.21.15.112:0/15010172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2770109c80 0x7f277019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:46.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 -- 172.21.15.112:0/15010172 >> 172.21.15.112:0/15010172 conn(0x7f2770076210 msgr2=0x7f277010cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:46.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 -- 172.21.15.112:0/15010172 shutdown_connections 2024-04-25T18:43:46.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:46.180+0000 7f2775200700 1 -- 172.21.15.112:0/15010172 wait complete. 2024-04-25T18:43:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:46 smithi171 ceph-mon[33266]: pgmap v347: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T18:43:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:46 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/15010172' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:43:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:46 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:43:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:46 smithi161 ceph-mon[32609]: pgmap v347: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T18:43:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:46 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/15010172' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:43:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:46 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:43:46.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:46 smithi112 ceph-mon[27864]: pgmap v347: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T18:43:46.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:46 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/15010172' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:43:46.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:46 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:43:47.036 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T18:43:47.036 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-25T18:43:47.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:47 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T18:43:47.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:47 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:43:47.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:47 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T18:43:47.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:47 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:43:47.479 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:47.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:47 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T18:43:47.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:47 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:43:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:48 smithi161 ceph-mon[32609]: pgmap v348: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T18:43:48.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:48 smithi112 ceph-mon[27864]: pgmap v348: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T18:43:48.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:48 smithi171 ceph-mon[33266]: pgmap v348: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T18:43:49.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.830+0000 7fc015400700 1 -- 172.21.15.112:0/716544796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0101092d0 msgr2=0x7fc0101096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:49.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.830+0000 7fc015400700 1 --2- 172.21.15.112:0/716544796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0101092d0 0x7fc0101096b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc0000099c0 tx=0x7fc000009cd0 comp rx=0 tx=0).stop 2024-04-25T18:43:49.833 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:49 smithi112 ceph-mon[27864]: pgmap v349: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T18:43:49.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.831+0000 7fc015400700 1 -- 172.21.15.112:0/716544796 shutdown_connections 2024-04-25T18:43:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.831+0000 7fc015400700 1 --2- 172.21.15.112:0/716544796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc01010e3b0 0x7fc010115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.831+0000 7fc015400700 1 --2- 172.21.15.112:0/716544796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0101092d0 0x7fc0101096b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.831+0000 7fc015400700 1 --2- 172.21.15.112:0/716544796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc010109c80 0x7fc01010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.831+0000 7fc015400700 1 -- 172.21.15.112:0/716544796 >> 172.21.15.112:0/716544796 conn(0x7fc010076210 msgr2=0x7fc010107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.831+0000 7fc015400700 1 -- 172.21.15.112:0/716544796 shutdown_connections 2024-04-25T18:43:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.831+0000 7fc015400700 1 -- 172.21.15.112:0/716544796 wait complete. 2024-04-25T18:43:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.832+0000 7fc015400700 1 Processor -- start 2024-04-25T18:43:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.832+0000 7fc015400700 1 -- start start 2024-04-25T18:43:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.833+0000 7fc015400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0101092d0 0x7fc0101a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:49.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.833+0000 7fc015400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc010109c80 0x7fc0101a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.833+0000 7fc015400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc01010e3b0 0x7fc01019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.833+0000 7fc015400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc010119290 con 0x7fc0101092d0 2024-04-25T18:43:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.833+0000 7fc015400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc010119110 con 0x7fc01010e3b0 2024-04-25T18:43:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.833+0000 7fc015400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc010119410 con 0x7fc010109c80 2024-04-25T18:43:49.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.833+0000 7fc00de00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc010109c80 0x7fc0101a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:49.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.833+0000 7fc00e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0101092d0 0x7fc0101a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.833+0000 7fc00de00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc010109c80 0x7fc0101a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:60476/0 (socket says 172.21.15.112:60476) 2024-04-25T18:43:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.833+0000 7fc00de00700 1 -- 172.21.15.112:0/4064544745 learned_addr learned my addr 172.21.15.112:0/4064544745 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.834+0000 7fc00f200700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc01010e3b0 0x7fc01019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.834+0000 7fc00de00700 1 -- 172.21.15.112:0/4064544745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc01010e3b0 msgr2=0x7fc01019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.834+0000 7fc00de00700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc01010e3b0 0x7fc01019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.834+0000 7fc00de00700 1 -- 172.21.15.112:0/4064544745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0101092d0 msgr2=0x7fc0101a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.834+0000 7fc00de00700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0101092d0 0x7fc0101a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.834+0000 7fc00de00700 1 -- 172.21.15.112:0/4064544745 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01019fd50 con 0x7fc010109c80 2024-04-25T18:43:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.834+0000 7fc00e800700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0101092d0 0x7fc0101a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:43:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.834+0000 7fc00de00700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc010109c80 0x7fc0101a5920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00e970 tx=0x7fbffc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.835+0000 7fc007400700 1 -- 172.21.15.112:0/4064544745 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc004a30 con 0x7fc010109c80 2024-04-25T18:43:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.835+0000 7fc007400700 1 -- 172.21.15.112:0/4064544745 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbffc00cc90 con 0x7fc010109c80 2024-04-25T18:43:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.835+0000 7fc015400700 1 -- 172.21.15.112:0/4064544745 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0101a0040 con 0x7fc010109c80 2024-04-25T18:43:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.835+0000 7fc015400700 1 -- 172.21.15.112:0/4064544745 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0101ac2b0 con 0x7fc010109c80 2024-04-25T18:43:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.835+0000 7fc007400700 1 -- 172.21.15.112:0/4064544745 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc005610 con 0x7fc010109c80 2024-04-25T18:43:49.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.836+0000 7fc015400700 1 -- 172.21.15.112:0/4064544745 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff0005330 con 0x7fc010109c80 2024-04-25T18:43:49.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.838+0000 7fc007400700 1 -- 172.21.15.112:0/4064544745 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fbffc00c7e0 con 0x7fc010109c80 2024-04-25T18:43:49.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.839+0000 7fc007400700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbff40a3cc0 0x7fbff40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:49.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.840+0000 7fc007400700 1 -- 172.21.15.112:0/4064544745 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7fbffc0d04f0 con 0x7fc010109c80 2024-04-25T18:43:49.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.840+0000 7fc00e800700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbff40a3cc0 0x7fbff40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:49.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.840+0000 7fc00e800700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbff40a3cc0 0x7fbff40a6180 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc000003760 tx=0x7fc000005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:49.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:49.843+0000 7fc007400700 1 -- 172.21.15.112:0/4064544745 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbffc09cf80 con 0x7fc010109c80 2024-04-25T18:43:49.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:49 smithi171 ceph-mon[33266]: pgmap v349: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T18:43:49.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:49 smithi161 ceph-mon[32609]: pgmap v349: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T18:43:50.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.100+0000 7fc015400700 1 -- 172.21.15.112:0/4064544745 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbff0000bc0 con 0x7fbff40a3cc0 2024-04-25T18:43:50.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.134+0000 7fc007400700 1 -- 172.21.15.112:0/4064544745 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbff0000bc0 con 0x7fbff40a3cc0 2024-04-25T18:43:50.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 -- 172.21.15.112:0/4064544745 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbff40a3cc0 msgr2=0x7fbff40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:50.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbff40a3cc0 0x7fbff40a6180 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc000003760 tx=0x7fc000005fb0 comp rx=0 tx=0).stop 2024-04-25T18:43:50.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 -- 172.21.15.112:0/4064544745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc010109c80 msgr2=0x7fc0101a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:50.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc010109c80 0x7fc0101a5920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00e970 tx=0x7fbffc00ec80 comp rx=0 tx=0).stop 2024-04-25T18:43:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 -- 172.21.15.112:0/4064544745 shutdown_connections 2024-04-25T18:43:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbff40a3cc0 0x7fbff40a6180 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc01010e3b0 0x7fc01019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0101092d0 0x7fc0101a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 --2- 172.21.15.112:0/4064544745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc010109c80 0x7fc0101a5920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 -- 172.21.15.112:0/4064544745 >> 172.21.15.112:0/4064544745 conn(0x7fc010076210 msgr2=0x7fc010077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 -- 172.21.15.112:0/4064544745 shutdown_connections 2024-04-25T18:43:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:50.138+0000 7fc015400700 1 -- 172.21.15.112:0/4064544745 wait complete. 2024-04-25T18:43:50.474 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:43:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:43:50.110+0000 7f6dd2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:43:50.474 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:43:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:43:50.110+0000 7f6dd2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:43:50.474 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:43:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:43:50.110+0000 7f6dd2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:43:50.474 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:43:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:43:50.110+0000 7f6dd2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:43:50.474 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:43:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:43:50.110+0000 7f6dd2800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:43:50.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:50 smithi171 ceph-mon[33266]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:43:50.969 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:50 smithi112 ceph-mon[27864]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:43:50.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:50 smithi161 ceph-mon[32609]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:43:51.019 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T18:43:51.456 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:51.914 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:51 smithi112 ceph-mon[27864]: pgmap v350: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-25T18:43:51.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:51 smithi171 ceph-mon[33266]: pgmap v350: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-25T18:43:51.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:51 smithi161 ceph-mon[32609]: pgmap v350: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-25T18:43:52.640 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:52 smithi112 ceph-mon[27864]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T18:43:52.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:52 smithi171 ceph-mon[33266]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T18:43:52.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:52 smithi161 ceph-mon[32609]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T18:43:53.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.770+0000 7f660de00700 1 -- 172.21.15.112:0/3579424226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6608104ec0 msgr2=0x7f66081052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:53.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.770+0000 7f660de00700 1 --2- 172.21.15.112:0/3579424226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6608104ec0 0x7f66081052a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65f80099c0 tx=0x7f65f8009cd0 comp rx=0 tx=0).stop 2024-04-25T18:43:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.770+0000 7f660de00700 1 -- 172.21.15.112:0/3579424226 shutdown_connections 2024-04-25T18:43:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.770+0000 7f660de00700 1 --2- 172.21.15.112:0/3579424226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6608109fa0 0x7f6608111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.770+0000 7f660de00700 1 --2- 172.21.15.112:0/3579424226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6608104ec0 0x7f66081052a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.770+0000 7f660de00700 1 --2- 172.21.15.112:0/3579424226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6608105870 0x7f66081098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.770+0000 7f660de00700 1 -- 172.21.15.112:0/3579424226 >> 172.21.15.112:0/3579424226 conn(0x7f6608100750 msgr2=0x7f6608102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.770+0000 7f660de00700 1 -- 172.21.15.112:0/3579424226 shutdown_connections 2024-04-25T18:43:53.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.770+0000 7f660de00700 1 -- 172.21.15.112:0/3579424226 wait complete. 2024-04-25T18:43:53.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.771+0000 7f660de00700 1 Processor -- start 2024-04-25T18:43:53.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.771+0000 7f660de00700 1 -- start start 2024-04-25T18:43:53.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.772+0000 7f660de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6608104ec0 0x7f660819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:53.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.772+0000 7f660de00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6608105870 0x7f660819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:53.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.772+0000 7f660de00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6608109fa0 0x7f660819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:53.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.772+0000 7f660de00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6608114eb0 con 0x7f6608104ec0 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.772+0000 7f660de00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6608114d30 con 0x7f6608109fa0 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.772+0000 7f660de00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6608115030 con 0x7f6608105870 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.772+0000 7f6606000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6608105870 0x7f660819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.772+0000 7f6606000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6608105870 0x7f660819b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:60508/0 (socket says 172.21.15.112:60508) 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.772+0000 7f6606000700 1 -- 172.21.15.112:0/628399817 learned_addr learned my addr 172.21.15.112:0/628399817 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.772+0000 7f6606a00700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6608104ec0 0x7f660819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.772+0000 7f6607400700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6608109fa0 0x7f660819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.773+0000 7f6606000700 1 -- 172.21.15.112:0/628399817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6608109fa0 msgr2=0x7f660819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.773+0000 7f6606000700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6608109fa0 0x7f660819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.773+0000 7f6606000700 1 -- 172.21.15.112:0/628399817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6608104ec0 msgr2=0x7f660819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.773+0000 7f6606000700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6608104ec0 0x7f660819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.773+0000 7f6606000700 1 -- 172.21.15.112:0/628399817 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66081a0350 con 0x7f6608105870 2024-04-25T18:43:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.773+0000 7f6606000700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6608105870 0x7f660819b880 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65f400d950 tx=0x7f65f400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.773+0000 7f660ca00700 1 -- 172.21.15.112:0/628399817 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4016070 con 0x7f6608105870 2024-04-25T18:43:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.773+0000 7f660de00700 1 -- 172.21.15.112:0/628399817 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66081a0640 con 0x7f6608105870 2024-04-25T18:43:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.773+0000 7f660ca00700 1 -- 172.21.15.112:0/628399817 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f4004830 con 0x7f6608105870 2024-04-25T18:43:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.773+0000 7f660ca00700 1 -- 172.21.15.112:0/628399817 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4005420 con 0x7f6608105870 2024-04-25T18:43:53.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.773+0000 7f660de00700 1 -- 172.21.15.112:0/628399817 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66081a7ec0 con 0x7f6608105870 2024-04-25T18:43:53.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.775+0000 7f660de00700 1 -- 172.21.15.112:0/628399817 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6608069730 con 0x7f6608105870 2024-04-25T18:43:53.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.776+0000 7f660ca00700 1 -- 172.21.15.112:0/628399817 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f65f4004220 con 0x7f6608105870 2024-04-25T18:43:53.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.778+0000 7f660ca00700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f65ec0a3cd0 0x7f65ec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.778+0000 7f660ca00700 1 -- 172.21.15.112:0/628399817 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f65f40d0230 con 0x7f6608105870 2024-04-25T18:43:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.778+0000 7f6606a00700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f65ec0a3cd0 0x7f65ec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.779+0000 7f6606a00700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f65ec0a3cd0 0x7f65ec0a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009990 tx=0x7f65f8005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:53.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:53.782+0000 7f660ca00700 1 -- 172.21.15.112:0/628399817 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65f401f020 con 0x7f6608105870 2024-04-25T18:43:53.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:53 smithi171 ceph-mon[33266]: pgmap v351: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-25T18:43:53.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:53 smithi161 ceph-mon[32609]: pgmap v351: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-25T18:43:53.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:53 smithi112 ceph-mon[27864]: pgmap v351: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-25T18:43:54.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.046+0000 7f660de00700 1 -- 172.21.15.112:0/628399817 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6608062e80 con 0x7f65ec0a3cd0 2024-04-25T18:43:54.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.089+0000 7f660ca00700 1 -- 172.21.15.112:0/628399817 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6608062e80 con 0x7f65ec0a3cd0 2024-04-25T18:43:54.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.092+0000 7f660de00700 1 -- 172.21.15.112:0/628399817 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f65ec0a3cd0 msgr2=0x7f65ec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:54.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.092+0000 7f660de00700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f65ec0a3cd0 0x7f65ec0a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009990 tx=0x7f65f8005c40 comp rx=0 tx=0).stop 2024-04-25T18:43:54.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.092+0000 7f660de00700 1 -- 172.21.15.112:0/628399817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6608105870 msgr2=0x7f660819b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:54.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.092+0000 7f660de00700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6608105870 0x7f660819b880 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65f400d950 tx=0x7f65f400dc60 comp rx=0 tx=0).stop 2024-04-25T18:43:54.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.092+0000 7f660de00700 1 -- 172.21.15.112:0/628399817 shutdown_connections 2024-04-25T18:43:54.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.092+0000 7f660de00700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f65ec0a3cd0 0x7f65ec0a6190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:54.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.092+0000 7f660de00700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6608109fa0 0x7f660819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:54.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.092+0000 7f660de00700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6608104ec0 0x7f660819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:54.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.092+0000 7f660de00700 1 --2- 172.21.15.112:0/628399817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6608105870 0x7f660819b880 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:54.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.092+0000 7f660de00700 1 -- 172.21.15.112:0/628399817 >> 172.21.15.112:0/628399817 conn(0x7f6608100750 msgr2=0x7f6608102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:54.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.092+0000 7f660de00700 1 -- 172.21.15.112:0/628399817 shutdown_connections 2024-04-25T18:43:54.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:54.093+0000 7f660de00700 1 -- 172.21.15.112:0/628399817 wait complete. 2024-04-25T18:43:54.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:54 smithi112 ceph-mon[27864]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:43:54.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:54 smithi171 ceph-mon[33266]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:43:54.945 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T18:43:54.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:54 smithi161 ceph-mon[32609]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:43:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:55.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T18:43:55.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T18:43:55.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-25T18:43:55.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-04-25T18:43:55.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-04-25T18:43:55.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-25T18:43:55.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T18:43:55.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-04-25T18:43:55.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-25T18:43:55.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:55 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T18:43:55.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:55 smithi112 ceph-mon[27864]: pgmap v352: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T18:43:55.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:55 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T18:43:55.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:55 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T18:43:55.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:55 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-25T18:43:55.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:55 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-04-25T18:43:55.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:55 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-04-25T18:43:55.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:55 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-25T18:43:55.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:55 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T18:43:55.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:55 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-04-25T18:43:55.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:55 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-25T18:43:55.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:55 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T18:43:55.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:55 smithi171 ceph-mon[33266]: pgmap v352: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T18:43:55.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:55 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T18:43:55.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:55 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T18:43:55.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:55 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-25T18:43:55.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:55 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-04-25T18:43:55.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:55 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-04-25T18:43:55.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:55 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-25T18:43:55.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:55 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T18:43:55.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:55 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-04-25T18:43:55.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:55 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-25T18:43:55.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:55 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T18:43:55.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:55 smithi161 ceph-mon[32609]: pgmap v352: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T18:43:56.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T18:43:56.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T18:43:56.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-25T18:43:56.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-04-25T18:43:56.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-04-25T18:43:56.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-25T18:43:56.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T18:43:56.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-04-25T18:43:56.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-25T18:43:56.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:56 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T18:43:56.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:56 smithi112 ceph-mon[27864]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T18:43:56.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T18:43:56.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T18:43:56.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-25T18:43:56.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-04-25T18:43:56.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-04-25T18:43:56.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-25T18:43:56.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T18:43:56.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-04-25T18:43:56.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-25T18:43:56.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:56 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T18:43:56.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:56 smithi171 ceph-mon[33266]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T18:43:56.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T18:43:56.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T18:43:56.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-25T18:43:56.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-04-25T18:43:56.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-04-25T18:43:56.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-25T18:43:56.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T18:43:56.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-04-25T18:43:56.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-25T18:43:56.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:56 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T18:43:56.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:56 smithi161 ceph-mon[32609]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T18:43:57.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.727+0000 7f18d2600700 1 -- 172.21.15.112:0/1512376929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18cc109310 msgr2=0x7f18cc1096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.727+0000 7f18d2600700 1 --2- 172.21.15.112:0/1512376929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18cc109310 0x7f18cc1096f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0099c0 tx=0x7f18bc009cd0 comp rx=0 tx=0).stop 2024-04-25T18:43:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.727+0000 7f18d2600700 1 -- 172.21.15.112:0/1512376929 shutdown_connections 2024-04-25T18:43:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.727+0000 7f18d2600700 1 --2- 172.21.15.112:0/1512376929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18cc10e3a0 0x7f18cc115f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.727+0000 7f18d2600700 1 --2- 172.21.15.112:0/1512376929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18cc109310 0x7f18cc1096f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.727+0000 7f18d2600700 1 --2- 172.21.15.112:0/1512376929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18cc109cc0 0x7f18cc10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.727+0000 7f18d2600700 1 -- 172.21.15.112:0/1512376929 >> 172.21.15.112:0/1512376929 conn(0x7f18cc076210 msgr2=0x7f18cc078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.727+0000 7f18d2600700 1 -- 172.21.15.112:0/1512376929 shutdown_connections 2024-04-25T18:43:57.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.727+0000 7f18d2600700 1 -- 172.21.15.112:0/1512376929 wait complete. 2024-04-25T18:43:57.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.729+0000 7f18d2600700 1 Processor -- start 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.729+0000 7f18d2600700 1 -- start start 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.729+0000 7f18d2600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18cc109310 0x7f18cc1a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.729+0000 7f18d2600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18cc109cc0 0x7f18cc1a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.729+0000 7f18d2600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18cc10e3a0 0x7f18cc19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.729+0000 7f18d2600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc119230 con 0x7f18cc10e3a0 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.729+0000 7f18d2600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1190b0 con 0x7f18cc109cc0 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.729+0000 7f18d2600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1193b0 con 0x7f18cc109310 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.729+0000 7f18cb400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18cc109310 0x7f18cc1a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.730+0000 7f18cb400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18cc109310 0x7f18cc1a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:60530/0 (socket says 172.21.15.112:60530) 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.730+0000 7f18cb400700 1 -- 172.21.15.112:0/2493930043 learned_addr learned my addr 172.21.15.112:0/2493930043 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.730+0000 7f18cbe00700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18cc10e3a0 0x7f18cc19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:57.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.730+0000 7f18caa00700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18cc109cc0 0x7f18cc1a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.730+0000 7f18cb400700 1 -- 172.21.15.112:0/2493930043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18cc109cc0 msgr2=0x7f18cc1a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.730+0000 7f18cb400700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18cc109cc0 0x7f18cc1a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.730+0000 7f18cb400700 1 -- 172.21.15.112:0/2493930043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18cc10e3a0 msgr2=0x7f18cc19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.730+0000 7f18cb400700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18cc10e3a0 0x7f18cc19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.730+0000 7f18cb400700 1 -- 172.21.15.112:0/2493930043 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc19fd10 con 0x7f18cc109310 2024-04-25T18:43:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.730+0000 7f18cbe00700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18cc10e3a0 0x7f18cc19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:43:57.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.730+0000 7f18cb400700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18cc109310 0x7f18cc1a53f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f18bc006010 tx=0x7f18bc005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:57.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.731+0000 7f18c3e00700 1 -- 172.21.15.112:0/2493930043 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc026070 con 0x7f18cc109310 2024-04-25T18:43:57.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.731+0000 7f18c3e00700 1 -- 172.21.15.112:0/2493930043 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18bc003c70 con 0x7f18cc109310 2024-04-25T18:43:57.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.731+0000 7f18d2600700 1 -- 172.21.15.112:0/2493930043 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18cc19ffa0 con 0x7f18cc109310 2024-04-25T18:43:57.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.731+0000 7f18c3e00700 1 -- 172.21.15.112:0/2493930043 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00fc20 con 0x7f18cc109310 2024-04-25T18:43:57.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.731+0000 7f18d2600700 1 -- 172.21.15.112:0/2493930043 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18cc1ac2c0 con 0x7f18cc109310 2024-04-25T18:43:57.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.732+0000 7f18d2600700 1 -- 172.21.15.112:0/2493930043 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18cc069730 con 0x7f18cc109310 2024-04-25T18:43:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.734+0000 7f18c3e00700 1 -- 172.21.15.112:0/2493930043 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f18bc029070 con 0x7f18cc109310 2024-04-25T18:43:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.735+0000 7f18c3e00700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b00a3cd0 0x7f18b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:43:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.735+0000 7f18c3e00700 1 -- 172.21.15.112:0/2493930043 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f18bc0d1440 con 0x7f18cc109310 2024-04-25T18:43:57.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.736+0000 7f18caa00700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b00a3cd0 0x7f18b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:43:57.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.736+0000 7f18caa00700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b00a3cd0 0x7f18b00a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009680 tx=0x7f18b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:43:57.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:57.738+0000 7f18c3e00700 1 -- 172.21.15.112:0/2493930043 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18bc030020 con 0x7f18cc109310 2024-04-25T18:43:57.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:57 smithi171 ceph-mon[33266]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T18:43:57.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:57 smithi171 ceph-mon[33266]: pgmap v355: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T18:43:57.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:57 smithi161 ceph-mon[32609]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T18:43:57.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:57 smithi161 ceph-mon[32609]: pgmap v355: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T18:43:57.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:57 smithi112 ceph-mon[27864]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T18:43:57.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:57 smithi112 ceph-mon[27864]: pgmap v355: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T18:43:58.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.000+0000 7f18d2600700 1 -- 172.21.15.112:0/2493930043 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f18cc1a0ae0 con 0x7f18b00a3cd0 2024-04-25T18:43:58.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.013+0000 7f18c3e00700 1 -- 172.21.15.112:0/2493930043 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f18cc1a0ae0 con 0x7f18b00a3cd0 2024-04-25T18:43:58.015 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_0/88adb84a-34e4-4b4a-8169-cd8b6cc94b56 2024-04-25T18:43:58.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 -- 172.21.15.112:0/2493930043 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b00a3cd0 msgr2=0x7f18b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:58.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b00a3cd0 0x7f18b00a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009680 tx=0x7f18b8009260 comp rx=0 tx=0).stop 2024-04-25T18:43:58.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 -- 172.21.15.112:0/2493930043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18cc109310 msgr2=0x7f18cc1a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:43:58.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18cc109310 0x7f18cc1a53f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f18bc006010 tx=0x7f18bc005280 comp rx=0 tx=0).stop 2024-04-25T18:43:58.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 -- 172.21.15.112:0/2493930043 shutdown_connections 2024-04-25T18:43:58.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b00a3cd0 0x7f18b00a6190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:58.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18cc10e3a0 0x7f18cc19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:58.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18cc109310 0x7f18cc1a53f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:58.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 --2- 172.21.15.112:0/2493930043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18cc109cc0 0x7f18cc1a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:43:58.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 -- 172.21.15.112:0/2493930043 >> 172.21.15.112:0/2493930043 conn(0x7f18cc076210 msgr2=0x7f18cc10d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:43:58.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 -- 172.21.15.112:0/2493930043 shutdown_connections 2024-04-25T18:43:58.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:43:58.017+0000 7f18d2600700 1 -- 172.21.15.112:0/2493930043 wait complete. 2024-04-25T18:43:58.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:58 smithi112 ceph-mon[27864]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:43:58.897 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T18:43:58.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:58 smithi171 ceph-mon[33266]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:43:58.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:58 smithi161 ceph-mon[32609]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:43:59.340 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:43:59.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:43:59 smithi112 ceph-mon[27864]: pgmap v356: 97 pgs: 97 active+clean; 608 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:43:59.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:43:59 smithi171 ceph-mon[33266]: pgmap v356: 97 pgs: 97 active+clean; 608 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:43:59.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:43:59 smithi161 ceph-mon[32609]: pgmap v356: 97 pgs: 97 active+clean; 608 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:44:01.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.659+0000 7eff47400700 1 -- 172.21.15.112:0/1074402021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff40107990 msgr2=0x7eff40107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.659+0000 7eff47400700 1 --2- 172.21.15.112:0/1074402021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff40107990 0x7eff40107d70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7eff34009990 tx=0x7eff34009ca0 comp rx=0 tx=0).stop 2024-04-25T18:44:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.659+0000 7eff47400700 1 -- 172.21.15.112:0/1074402021 shutdown_connections 2024-04-25T18:44:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.659+0000 7eff47400700 1 --2- 172.21.15.112:0/1074402021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff40102300 0x7eff4010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.659+0000 7eff47400700 1 --2- 172.21.15.112:0/1074402021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff40107990 0x7eff40107d70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.659+0000 7eff47400700 1 --2- 172.21.15.112:0/1074402021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff40101940 0x7eff40101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.659+0000 7eff47400700 1 -- 172.21.15.112:0/1074402021 >> 172.21.15.112:0/1074402021 conn(0x7eff400788f0 msgr2=0x7eff40078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.659+0000 7eff47400700 1 -- 172.21.15.112:0/1074402021 shutdown_connections 2024-04-25T18:44:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.659+0000 7eff47400700 1 -- 172.21.15.112:0/1074402021 wait complete. 2024-04-25T18:44:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.661+0000 7eff47400700 1 Processor -- start 2024-04-25T18:44:01.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.661+0000 7eff47400700 1 -- start start 2024-04-25T18:44:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.661+0000 7eff47400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff40101940 0x7eff401034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.661+0000 7eff47400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff40102300 0x7eff40103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.661+0000 7eff47400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff40107990 0x7eff40103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.661+0000 7eff47400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff40111ae0 con 0x7eff40101940 2024-04-25T18:44:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.661+0000 7eff47400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff40111960 con 0x7eff40107990 2024-04-25T18:44:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.661+0000 7eff47400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7eff40111c60 con 0x7eff40102300 2024-04-25T18:44:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff44a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff40101940 0x7eff401034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff3fe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff40102300 0x7eff40103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:01.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff3fe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff40102300 0x7eff40103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:56764/0 (socket says 172.21.15.112:56764) 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff3fe00700 1 -- 172.21.15.112:0/1527006741 learned_addr learned my addr 172.21.15.112:0/1527006741 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff44a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff40101940 0x7eff401034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58282/0 (socket says 172.21.15.112:58282) 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff45400700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff40107990 0x7eff40103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff3fe00700 1 -- 172.21.15.112:0/1527006741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff40107990 msgr2=0x7eff40103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff3fe00700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff40107990 0x7eff40103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff3fe00700 1 -- 172.21.15.112:0/1527006741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff40101940 msgr2=0x7eff401034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff3fe00700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff40101940 0x7eff401034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff3fe00700 1 -- 172.21.15.112:0/1527006741 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff401a7610 con 0x7eff40102300 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.662+0000 7eff3fe00700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff40102300 0x7eff40103a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00eb30 tx=0x7eff2c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.663+0000 7eff3d600700 1 -- 172.21.15.112:0/1527006741 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c00cbd0 con 0x7eff40102300 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.663+0000 7eff47400700 1 -- 172.21.15.112:0/1527006741 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff401a7900 con 0x7eff40102300 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.663+0000 7eff3d600700 1 -- 172.21.15.112:0/1527006741 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff2c0044d0 con 0x7eff40102300 2024-04-25T18:44:01.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.663+0000 7eff47400700 1 -- 172.21.15.112:0/1527006741 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff401a7e90 con 0x7eff40102300 2024-04-25T18:44:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.663+0000 7eff3d600700 1 -- 172.21.15.112:0/1527006741 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c003e00 con 0x7eff40102300 2024-04-25T18:44:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.664+0000 7eff47400700 1 -- 172.21.15.112:0/1527006741 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff40103060 con 0x7eff40102300 2024-04-25T18:44:01.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.666+0000 7eff3d600700 1 -- 172.21.15.112:0/1527006741 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7eff2c004020 con 0x7eff40102300 2024-04-25T18:44:01.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.667+0000 7eff3d600700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7eff280a3d50 0x7eff280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:01.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.668+0000 7eff3d600700 1 -- 172.21.15.112:0/1527006741 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7eff2c0d0710 con 0x7eff40102300 2024-04-25T18:44:01.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.668+0000 7eff44a00700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7eff280a3d50 0x7eff280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:01.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.668+0000 7eff44a00700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7eff280a3d50 0x7eff280a6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff340098e0 tx=0x7eff34005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:01.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.672+0000 7eff3d600700 1 -- 172.21.15.112:0/1527006741 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff2c016330 con 0x7eff40102300 2024-04-25T18:44:01.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.929+0000 7eff47400700 1 -- 172.21.15.112:0/1527006741 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7eff40106640 con 0x7eff280a3d50 2024-04-25T18:44:01.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.965+0000 7eff3d600700 1 -- 172.21.15.112:0/1527006741 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7eff40106640 con 0x7eff280a3d50 2024-04-25T18:44:01.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 -- 172.21.15.112:0/1527006741 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7eff280a3d50 msgr2=0x7eff280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:01.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7eff280a3d50 0x7eff280a6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff340098e0 tx=0x7eff34005960 comp rx=0 tx=0).stop 2024-04-25T18:44:01.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 -- 172.21.15.112:0/1527006741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff40102300 msgr2=0x7eff40103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:01.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff40102300 0x7eff40103a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00eb30 tx=0x7eff2c00ee40 comp rx=0 tx=0).stop 2024-04-25T18:44:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 -- 172.21.15.112:0/1527006741 shutdown_connections 2024-04-25T18:44:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7eff280a3d50 0x7eff280a6210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff40107990 0x7eff40103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff40101940 0x7eff401034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 --2- 172.21.15.112:0/1527006741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff40102300 0x7eff40103a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 -- 172.21.15.112:0/1527006741 >> 172.21.15.112:0/1527006741 conn(0x7eff400788f0 msgr2=0x7eff4006a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 -- 172.21.15.112:0/1527006741 shutdown_connections 2024-04-25T18:44:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:01.968+0000 7eff47400700 1 -- 172.21.15.112:0/1527006741 wait complete. 2024-04-25T18:44:02.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:02 smithi112 ceph-mon[27864]: pgmap v357: 97 pgs: 97 active+clean; 608 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.1 KiB/s wr, 6 op/s; 72 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T18:44:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:02 smithi171 ceph-mon[33266]: pgmap v357: 97 pgs: 97 active+clean; 608 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.1 KiB/s wr, 6 op/s; 72 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T18:44:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:02 smithi161 ceph-mon[32609]: pgmap v357: 97 pgs: 97 active+clean; 608 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.1 KiB/s wr, 6 op/s; 72 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T18:44:02.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T18:44:03.261 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:03.430 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:03 smithi112 ceph-mon[27864]: from='client.24616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:44:03.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:03 smithi171 ceph-mon[33266]: from='client.24616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:44:03.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:03 smithi161 ceph-mon[32609]: from='client.24616 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:44:04.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:04 smithi171 ceph-mon[33266]: pgmap v358: 97 pgs: 97 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 8 op/s; 72 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T18:44:04.442 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:04 smithi112 ceph-mon[27864]: pgmap v358: 97 pgs: 97 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 8 op/s; 72 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T18:44:04.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:04 smithi161 ceph-mon[32609]: pgmap v358: 97 pgs: 97 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 8 op/s; 72 B/s, 2 keys/s, 0 objects/s recovering 2024-04-25T18:44:05.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.569+0000 7f6f75600700 1 -- 172.21.15.112:0/980283681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f70109c80 msgr2=0x7f6f7010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:05.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.569+0000 7f6f75600700 1 --2- 172.21.15.112:0/980283681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f70109c80 0x7f6f7010dcd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0099c0 tx=0x7f6f5c009cd0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.569+0000 7f6f75600700 1 -- 172.21.15.112:0/980283681 shutdown_connections 2024-04-25T18:44:05.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.569+0000 7f6f75600700 1 --2- 172.21.15.112:0/980283681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f7010e3b0 0x7f6f70115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.569+0000 7f6f75600700 1 --2- 172.21.15.112:0/980283681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f701092d0 0x7f6f701096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.569+0000 7f6f75600700 1 --2- 172.21.15.112:0/980283681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f70109c80 0x7f6f7010dcd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.569+0000 7f6f75600700 1 -- 172.21.15.112:0/980283681 >> 172.21.15.112:0/980283681 conn(0x7f6f70076210 msgr2=0x7f6f70107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:05.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.569+0000 7f6f75600700 1 -- 172.21.15.112:0/980283681 shutdown_connections 2024-04-25T18:44:05.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.569+0000 7f6f75600700 1 -- 172.21.15.112:0/980283681 wait complete. 2024-04-25T18:44:05.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.571+0000 7f6f75600700 1 Processor -- start 2024-04-25T18:44:05.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.571+0000 7f6f75600700 1 -- start start 2024-04-25T18:44:05.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.571+0000 7f6f75600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f701092d0 0x7f6f7019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:05.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.571+0000 7f6f75600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f70109c80 0x7f6f7019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:05.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.571+0000 7f6f75600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f7010e3b0 0x7f6f701a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:05.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.571+0000 7f6f75600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70119220 con 0x7f6f701092d0 2024-04-25T18:44:05.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.571+0000 7f6f75600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701190a0 con 0x7f6f70109c80 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.571+0000 7f6f75600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701193a0 con 0x7f6f7010e3b0 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f701092d0 0x7f6f7019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6e000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f70109c80 0x7f6f7019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6ea00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f701092d0 0x7f6f7019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58306/0 (socket says 172.21.15.112:58306) 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6ea00700 1 -- 172.21.15.112:0/2390852661 learned_addr learned my addr 172.21.15.112:0/2390852661 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6f400700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f7010e3b0 0x7f6f701a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6ea00700 1 -- 172.21.15.112:0/2390852661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f7010e3b0 msgr2=0x7f6f701a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6ea00700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f7010e3b0 0x7f6f701a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6ea00700 1 -- 172.21.15.112:0/2390852661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f70109c80 msgr2=0x7f6f7019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6ea00700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f70109c80 0x7f6f7019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6ea00700 1 -- 172.21.15.112:0/2390852661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f701a4750 con 0x7f6f701092d0 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6e000700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f70109c80 0x7f6f7019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.572+0000 7f6f6ea00700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f701092d0 0x7f6f7019f790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000e9a0 tx=0x7f6f6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:05.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.573+0000 7f6f67400700 1 -- 172.21.15.112:0/2390852661 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000c9c0 con 0x7f6f701092d0 2024-04-25T18:44:05.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.573+0000 7f6f67400700 1 -- 172.21.15.112:0/2390852661 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f6000cb60 con 0x7f6f701092d0 2024-04-25T18:44:05.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.573+0000 7f6f67400700 1 -- 172.21.15.112:0/2390852661 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60018680 con 0x7f6f701092d0 2024-04-25T18:44:05.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.573+0000 7f6f75600700 1 -- 172.21.15.112:0/2390852661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f701a4a40 con 0x7f6f701092d0 2024-04-25T18:44:05.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.573+0000 7f6f75600700 1 -- 172.21.15.112:0/2390852661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f701ac2c0 con 0x7f6f701092d0 2024-04-25T18:44:05.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.574+0000 7f6f75600700 1 -- 172.21.15.112:0/2390852661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f70069730 con 0x7f6f701092d0 2024-04-25T18:44:05.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.576+0000 7f6f67400700 1 -- 172.21.15.112:0/2390852661 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6f60010430 con 0x7f6f701092d0 2024-04-25T18:44:05.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.577+0000 7f6f67400700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6f540a3d50 0x7f6f540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:05.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.577+0000 7f6f67400700 1 -- 172.21.15.112:0/2390852661 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f6f600d0840 con 0x7f6f701092d0 2024-04-25T18:44:05.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.578+0000 7f6f6e000700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6f540a3d50 0x7f6f540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:05.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.578+0000 7f6f6e000700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6f540a3d50 0x7f6f540a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000c00 tx=0x7f6f5c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:05.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.583+0000 7f6f67400700 1 -- 172.21.15.112:0/2390852661 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f6009d040 con 0x7f6f701092d0 2024-04-25T18:44:05.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.845+0000 7f6f75600700 1 -- 172.21.15.112:0/2390852661 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f6f70062e80 con 0x7f6f540a3d50 2024-04-25T18:44:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.858+0000 7f6f67400700 1 -- 172.21.15.112:0/2390852661 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6f70062e80 con 0x7f6f540a3d50 2024-04-25T18:44:05.860 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_1/588a97fc-5973-4692-b8e8-3364c5c8fa39 2024-04-25T18:44:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.861+0000 7f6f75600700 1 -- 172.21.15.112:0/2390852661 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6f540a3d50 msgr2=0x7f6f540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.861+0000 7f6f75600700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6f540a3d50 0x7f6f540a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000c00 tx=0x7f6f5c005fb0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.861+0000 7f6f75600700 1 -- 172.21.15.112:0/2390852661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f701092d0 msgr2=0x7f6f7019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.861+0000 7f6f75600700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f701092d0 0x7f6f7019f790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000e9a0 tx=0x7f6f6000ecb0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.861+0000 7f6f75600700 1 -- 172.21.15.112:0/2390852661 shutdown_connections 2024-04-25T18:44:05.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.861+0000 7f6f75600700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6f540a3d50 0x7f6f540a6210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.861+0000 7f6f75600700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f7010e3b0 0x7f6f701a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.861+0000 7f6f75600700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f701092d0 0x7f6f7019f790 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.861+0000 7f6f75600700 1 --2- 172.21.15.112:0/2390852661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f70109c80 0x7f6f7019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.861+0000 7f6f75600700 1 -- 172.21.15.112:0/2390852661 >> 172.21.15.112:0/2390852661 conn(0x7f6f70076210 msgr2=0x7f6f7010f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.861+0000 7f6f75600700 1 -- 172.21.15.112:0/2390852661 shutdown_connections 2024-04-25T18:44:05.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:05.862+0000 7f6f75600700 1 -- 172.21.15.112:0/2390852661 wait complete. 2024-04-25T18:44:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:06 smithi171 ceph-mon[33266]: pgmap v359: 97 pgs: 97 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.6 KiB/s wr, 8 op/s; 61 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:44:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:06 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:44:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:06 smithi161 ceph-mon[32609]: pgmap v359: 97 pgs: 97 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.6 KiB/s wr, 8 op/s; 61 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:44:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:06 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:44:06.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:06 smithi112 ceph-mon[27864]: pgmap v359: 97 pgs: 97 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.6 KiB/s wr, 8 op/s; 61 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:44:06.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:06 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:44:06.885 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:44:07.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:07 smithi161 ceph-mon[32609]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:44:07.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:07 smithi112 ceph-mon[27864]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:44:07.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:07 smithi171 ceph-mon[33266]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:44:08.014 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:08.442 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:08 smithi112 ceph-mon[27864]: pgmap v360: 97 pgs: 97 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 8 op/s; 55 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:44:08.443 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:08.443 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:08.443 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:08.443 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:08 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:08 smithi161 ceph-mon[32609]: pgmap v360: 97 pgs: 97 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 8 op/s; 55 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:44:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:08 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:08 smithi171 ceph-mon[33266]: pgmap v360: 97 pgs: 97 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 8 op/s; 55 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:44:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:08 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:10.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.461+0000 7f92f3800700 1 -- 172.21.15.112:0/1333696938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92ec108a50 msgr2=0x7f92ec108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:10.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.461+0000 7f92f3800700 1 --2- 172.21.15.112:0/1333696938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92ec108a50 0x7f92ec108e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009990 tx=0x7f92e0009ca0 comp rx=0 tx=0).stop 2024-04-25T18:44:10.464 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:10 smithi112 ceph-mon[27864]: pgmap v361: 97 pgs: 97 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s; 48 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:44:10.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.461+0000 7f92f3800700 1 -- 172.21.15.112:0/1333696938 shutdown_connections 2024-04-25T18:44:10.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.461+0000 7f92f3800700 1 --2- 172.21.15.112:0/1333696938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92ec103450 0x7f92ec10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:10.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.461+0000 7f92f3800700 1 --2- 172.21.15.112:0/1333696938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92ec108a50 0x7f92ec108e30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:10.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.461+0000 7f92f3800700 1 --2- 172.21.15.112:0/1333696938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92ec102a90 0x7f92ec102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:10.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.461+0000 7f92f3800700 1 -- 172.21.15.112:0/1333696938 >> 172.21.15.112:0/1333696938 conn(0x7f92ec079ba0 msgr2=0x7f92ec079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:10.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.461+0000 7f92f3800700 1 -- 172.21.15.112:0/1333696938 shutdown_connections 2024-04-25T18:44:10.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.461+0000 7f92f3800700 1 -- 172.21.15.112:0/1333696938 wait complete. 2024-04-25T18:44:10.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.463+0000 7f92f3800700 1 Processor -- start 2024-04-25T18:44:10.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.463+0000 7f92f3800700 1 -- start start 2024-04-25T18:44:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.463+0000 7f92f3800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92ec102a90 0x7f92ec19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f3800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92ec103450 0x7f92ec19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:10.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f3800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92ec108a50 0x7f92ec19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f3800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec112c30 con 0x7f92ec102a90 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f3800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec112ab0 con 0x7f92ec108a50 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f3800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec112db0 con 0x7f92ec103450 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f0e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92ec102a90 0x7f92ec19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f0e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92ec102a90 0x7f92ec19b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40826/0 (socket says 172.21.15.112:40826) 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f0e00700 1 -- 172.21.15.112:0/295911789 learned_addr learned my addr 172.21.15.112:0/295911789 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92ebe00700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92ec103450 0x7f92ec19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f1800700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92ec108a50 0x7f92ec19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f0e00700 1 -- 172.21.15.112:0/295911789 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92ec103450 msgr2=0x7f92ec19b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f0e00700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92ec103450 0x7f92ec19b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f0e00700 1 -- 172.21.15.112:0/295911789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92ec108a50 msgr2=0x7f92ec19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f0e00700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92ec108a50 0x7f92ec19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.464+0000 7f92f0e00700 1 -- 172.21.15.112:0/295911789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec1a0360 con 0x7f92ec102a90 2024-04-25T18:44:10.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.465+0000 7f92f0e00700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92ec102a90 0x7f92ec19b3a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f92e0000c00 tx=0x7f92e00057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:10.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.465+0000 7f92e9600700 1 -- 172.21.15.112:0/295911789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0027070 con 0x7f92ec102a90 2024-04-25T18:44:10.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.465+0000 7f92e9600700 1 -- 172.21.15.112:0/295911789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e0003680 con 0x7f92ec102a90 2024-04-25T18:44:10.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.465+0000 7f92e9600700 1 -- 172.21.15.112:0/295911789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0022b80 con 0x7f92ec102a90 2024-04-25T18:44:10.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.465+0000 7f92f3800700 1 -- 172.21.15.112:0/295911789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec1a05f0 con 0x7f92ec102a90 2024-04-25T18:44:10.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.465+0000 7f92f3800700 1 -- 172.21.15.112:0/295911789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec1a7ed0 con 0x7f92ec102a90 2024-04-25T18:44:10.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.467+0000 7f92f3800700 1 -- 172.21.15.112:0/295911789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ec104180 con 0x7f92ec102a90 2024-04-25T18:44:10.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.468+0000 7f92e9600700 1 -- 172.21.15.112:0/295911789 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f92e002a070 con 0x7f92ec102a90 2024-04-25T18:44:10.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:10 smithi161 ceph-mon[32609]: pgmap v361: 97 pgs: 97 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s; 48 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:44:10.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.470+0000 7f92e9600700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f92d40a3cd0 0x7f92d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:10.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.470+0000 7f92e9600700 1 -- 172.21.15.112:0/295911789 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f92e00d1540 con 0x7f92ec102a90 2024-04-25T18:44:10.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.470+0000 7f92ebe00700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f92d40a3cd0 0x7f92d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:10.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.471+0000 7f92ebe00700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f92d40a3cd0 0x7f92d40a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f92ec10fcd0 tx=0x7f92d800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:10.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.475+0000 7f92e9600700 1 -- 172.21.15.112:0/295911789 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92e005eff0 con 0x7f92ec102a90 2024-04-25T18:44:10.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:10 smithi171 ceph-mon[33266]: pgmap v361: 97 pgs: 97 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s; 48 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:44:10.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.755+0000 7f92f3800700 1 -- 172.21.15.112:0/295911789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92ec1a80a0 con 0x7f92ec102a90 2024-04-25T18:44:10.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.756+0000 7f92e9600700 1 -- 172.21.15.112:0/295911789 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7f92e0005030 con 0x7f92ec102a90 2024-04-25T18:44:10.758 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:44:10.758 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:43:45.212327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24511,24512,24493]},"id":1}]} 2024-04-25T18:44:10.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 -- 172.21.15.112:0/295911789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f92d40a3cd0 msgr2=0x7f92d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:10.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f92d40a3cd0 0x7f92d40a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f92ec10fcd0 tx=0x7f92d800b410 comp rx=0 tx=0).stop 2024-04-25T18:44:10.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 -- 172.21.15.112:0/295911789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92ec102a90 msgr2=0x7f92ec19b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:10.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92ec102a90 0x7f92ec19b3a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f92e0000c00 tx=0x7f92e00057a0 comp rx=0 tx=0).stop 2024-04-25T18:44:10.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 -- 172.21.15.112:0/295911789 shutdown_connections 2024-04-25T18:44:10.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f92d40a3cd0 0x7f92d40a6190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:10.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92ec108a50 0x7f92ec19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92ec102a90 0x7f92ec19b3a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 --2- 172.21.15.112:0/295911789 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92ec103450 0x7f92ec19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 -- 172.21.15.112:0/295911789 >> 172.21.15.112:0/295911789 conn(0x7f92ec079ba0 msgr2=0x7f92ec1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 -- 172.21.15.112:0/295911789 shutdown_connections 2024-04-25T18:44:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:10.759+0000 7f92f3800700 1 -- 172.21.15.112:0/295911789 wait complete. 2024-04-25T18:44:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-04-25T18:44:11.737 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:44:11.799 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:11.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:11 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:11.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:11 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/295911789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:44:11.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:11 smithi112 ceph-mon[27864]: pgmap v362: 97 pgs: 97 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 7 op/s; 48 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:44:11.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:11 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:11.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:11 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:11.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:11 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/295911789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:44:11.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:11 smithi171 ceph-mon[33266]: pgmap v362: 97 pgs: 97 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 7 op/s; 48 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:44:11.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:11 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:11.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:11 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:11.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:11 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/295911789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:44:11.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:11 smithi161 ceph-mon[32609]: pgmap v362: 97 pgs: 97 active+clean; 634 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 7 op/s; 48 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:44:12.186 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:12.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:12 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:44:12.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:12 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:44:12.610 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:12 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:12.833 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:12 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:44:12.834 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:12 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:44:12.834 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:12 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:12.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:12 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:44:12.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:12 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:44:12.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:12 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:44:13.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:13 smithi112 ceph-mon[27864]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T18:44:13.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:13 smithi171 ceph-mon[33266]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T18:44:13.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:13 smithi161 ceph-mon[32609]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T18:44:14.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.576+0000 7fc655600700 1 -- 172.21.15.112:0/355989380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc650075590 msgr2=0x7fc65010ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:14.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.576+0000 7fc655600700 1 --2- 172.21.15.112:0/355989380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc650075590 0x7fc65010ef00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc638009990 tx=0x7fc638009ca0 comp rx=0 tx=0).stop 2024-04-25T18:44:14.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.576+0000 7fc655600700 1 -- 172.21.15.112:0/355989380 shutdown_connections 2024-04-25T18:44:14.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.576+0000 7fc655600700 1 --2- 172.21.15.112:0/355989380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc65010f440 0x7fc650111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:14.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.576+0000 7fc655600700 1 --2- 172.21.15.112:0/355989380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc650074c70 0x7fc650075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:14.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.576+0000 7fc655600700 1 --2- 172.21.15.112:0/355989380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc650075590 0x7fc65010ef00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.576+0000 7fc655600700 1 -- 172.21.15.112:0/355989380 >> 172.21.15.112:0/355989380 conn(0x7fc650079ba0 msgr2=0x7fc650079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.577+0000 7fc655600700 1 -- 172.21.15.112:0/355989380 shutdown_connections 2024-04-25T18:44:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.577+0000 7fc655600700 1 -- 172.21.15.112:0/355989380 wait complete. 2024-04-25T18:44:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.578+0000 7fc655600700 1 Processor -- start 2024-04-25T18:44:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.578+0000 7fc655600700 1 -- start start 2024-04-25T18:44:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.578+0000 7fc655600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc650074c70 0x7fc65019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.578+0000 7fc655600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc650075590 0x7fc65019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.578+0000 7fc655600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc65010f440 0x7fc65019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.578+0000 7fc655600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc650114e00 con 0x7fc65010f440 2024-04-25T18:44:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.578+0000 7fc655600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc650114c80 con 0x7fc650074c70 2024-04-25T18:44:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.578+0000 7fc655600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc650114f80 con 0x7fc650075590 2024-04-25T18:44:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc64ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc650074c70 0x7fc65019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc64f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc65010f440 0x7fc65019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc64ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc650074c70 0x7fc65019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:60526/0 (socket says 172.21.15.112:60526) 2024-04-25T18:44:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc64ea00700 1 -- 172.21.15.112:0/1372839458 learned_addr learned my addr 172.21.15.112:0/1372839458 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc646000700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc650075590 0x7fc65019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc64ea00700 1 -- 172.21.15.112:0/1372839458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc650075590 msgr2=0x7fc65019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc64ea00700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc650075590 0x7fc65019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc64ea00700 1 -- 172.21.15.112:0/1372839458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc65010f440 msgr2=0x7fc65019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc64ea00700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc65010f440 0x7fc65019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc64ea00700 1 -- 172.21.15.112:0/1372839458 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6501a02b0 con 0x7fc650074c70 2024-04-25T18:44:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc646000700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc650075590 0x7fc65019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:44:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.579+0000 7fc64ea00700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc650074c70 0x7fc65019b2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc64000ef10 tx=0x7fc64000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.580+0000 7fc647e00700 1 -- 172.21.15.112:0/1372839458 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64000c9c0 con 0x7fc650074c70 2024-04-25T18:44:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.580+0000 7fc655600700 1 -- 172.21.15.112:0/1372839458 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6501a0540 con 0x7fc650074c70 2024-04-25T18:44:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.580+0000 7fc647e00700 1 -- 172.21.15.112:0/1372839458 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc640019070 con 0x7fc650074c70 2024-04-25T18:44:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.580+0000 7fc647e00700 1 -- 172.21.15.112:0/1372839458 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640014480 con 0x7fc650074c70 2024-04-25T18:44:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.580+0000 7fc655600700 1 -- 172.21.15.112:0/1372839458 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6501a7e20 con 0x7fc650074c70 2024-04-25T18:44:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.581+0000 7fc655600700 1 -- 172.21.15.112:0/1372839458 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc650069730 con 0x7fc650074c70 2024-04-25T18:44:14.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.583+0000 7fc647e00700 1 -- 172.21.15.112:0/1372839458 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc640012050 con 0x7fc650074c70 2024-04-25T18:44:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.584+0000 7fc647e00700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc6340a3cd0 0x7fc6340a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.585+0000 7fc647e00700 1 -- 172.21.15.112:0/1372839458 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fc6400d0c00 con 0x7fc650074c70 2024-04-25T18:44:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.585+0000 7fc646000700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc6340a3cd0 0x7fc6340a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:14.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.585+0000 7fc646000700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc6340a3cd0 0x7fc6340a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc63800b2e0 tx=0x7fc638005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:14.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.587+0000 7fc647e00700 1 -- 172.21.15.112:0/1372839458 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc640010040 con 0x7fc650074c70 2024-04-25T18:44:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.812+0000 7fc655600700 1 -- 172.21.15.112:0/1372839458 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc65019c600 con 0x7fc650074c70 2024-04-25T18:44:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.814+0000 7fc647e00700 1 -- 172.21.15.112:0/1372839458 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+22709 (secure 0 0 0) 0x7fc64009d400 con 0x7fc650074c70 2024-04-25T18:44:14.817 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:44:14.817 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","created":"2024-04-25T18:33:37.379089+0000","modified":"2024-04-25T18:43:56.620122+0000","last_up_change":"2024-04-25T18:42:00.961646+0000","last_in_change":"2024-04-25T18:41:40.647162+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:37:45.106199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:43:11.017975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:43:15.116917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efb4bc01-e29b-4402-8a7a-3387d1731ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6803","nonce":232254535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6805","nonce":232254535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6809","nonce":232254535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6807","nonce":232254535}]},"public_addr":"172.21.15.112:6803/232254535","cluster_addr":"172.21.15.112:6805/232254535","heartbeat_back_addr":"172.21.15.112:6809/232254535","heartbeat_front_addr":"172.21.15.112:6807/232254535","state":["exists","up"]},{"osd":1,"uuid":"65a83fa2-96a6-4743-8c17-21531f5e885e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6801","nonce":664224744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6803","nonce":664224744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6807","nonce":664224744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6805","nonce":664224744}]},"public_addr":"172.21.15.161:6801/664224744","cluster_addr":"172.21.15.161:6803/664224744","heartbeat_back_addr":"172.21.15.161:6807/664224744","heartbeat_front_addr":"172.21.15.161:6805/664224744","state":["exists","up"]},{"osd":2,"uuid":"7562ec4c-c126-48ec-a51c-56fa461a2e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6801","nonce":345172073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6803","nonce":345172073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6807","nonce":345172073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6805","nonce":345172073}]},"public_addr":"172.21.15.171:6801/345172073","cluster_addr":"172.21.15.171:6803/345172073","heartbeat_back_addr":"172.21.15.171:6807/345172073","heartbeat_front_addr":"172.21.15.171:6805/345172073","state":["exists","up"]},{"osd":3,"uuid":"d7aae8cd-fec7-46d9-8773-25eaaca3cc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6811","nonce":1472407156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6813","nonce":1472407156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6817","nonce":1472407156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6815","nonce":1472407156}]},"public_addr":"172.21.15.112:6811/1472407156","cluster_addr":"172.21.15.112:6813/1472407156","heartbeat_back_addr":"172.21.15.112:6817/1472407156","heartbeat_front_addr":"172.21.15.112:6815/1472407156","state":["exists","up"]},{"osd":4,"uuid":"b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6809","nonce":373204306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6811","nonce":373204306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6815","nonce":373204306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6813","nonce":373204306}]},"public_addr":"172.21.15.161:6809/373204306","cluster_addr":"172.21.15.161:6811/373204306","heartbeat_back_addr":"172.21.15.161:6815/373204306","heartbeat_front_addr":"172.21.15.161:6813/373204306","state":["exists","up"]},{"osd":5,"uuid":"56124d12-bc1f-4cba-8430-938248851776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6809","nonce":2643512313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6811","nonce":2643512313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6815","nonce":2643512313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6813","nonce":2643512313}]},"public_addr":"172.21.15.171:6809/2643512313","cluster_addr":"172.21.15.171:6811/2643512313","heartbeat_back_addr":"172.21.15.171:6815/2643512313","heartbeat_front_addr":"172.21.15.171:6813/2643512313","state":["exists","up"]},{"osd":6,"uuid":"9563a9d3-352b-4686-975c-282ec167248b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6819","nonce":1933654933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6821","nonce":1933654933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6825","nonce":1933654933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6823","nonce":1933654933}]},"public_addr":"172.21.15.112:6819/1933654933","cluster_addr":"172.21.15.112:6821/1933654933","heartbeat_back_addr":"172.21.15.112:6825/1933654933","heartbeat_front_addr":"172.21.15.112:6823/1933654933","state":["exists","up"]},{"osd":7,"uuid":"f2d06e86-6aac-40f4-956e-18fe0b0e3543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6817","nonce":3060201138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6819","nonce":3060201138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6823","nonce":3060201138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6821","nonce":3060201138}]},"public_addr":"172.21.15.161:6817/3060201138","cluster_addr":"172.21.15.161:6819/3060201138","heartbeat_back_addr":"172.21.15.161:6823/3060201138","heartbeat_front_addr":"172.21.15.161:6821/3060201138","state":["exists","up"]},{"osd":8,"uuid":"f006e45f-2bb9-45d0-9020-be7c33c6a825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6817","nonce":835756571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6819","nonce":835756571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6823","nonce":835756571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6821","nonce":835756571}]},"public_addr":"172.21.15.171:6817/835756571","cluster_addr":"172.21.15.171:6819/835756571","heartbeat_back_addr":"172.21.15.171:6823/835756571","heartbeat_front_addr":"172.21.15.171:6821/835756571","state":["exists","up"]},{"osd":9,"uuid":"c5a96491-4151-4633-b3bc-2c214fe51c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6827","nonce":349758528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6829","nonce":349758528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6833","nonce":349758528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6831","nonce":349758528}]},"public_addr":"172.21.15.112:6827/349758528","cluster_addr":"172.21.15.112:6829/349758528","heartbeat_back_addr":"172.21.15.112:6833/349758528","heartbeat_front_addr":"172.21.15.112:6831/349758528","state":["exists","up"]},{"osd":10,"uuid":"244eda3e-25a0-490e-879f-c1e2726c1be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6825","nonce":3013292765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6827","nonce":3013292765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6831","nonce":3013292765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6829","nonce":3013292765}]},"public_addr":"172.21.15.161:6825/3013292765","cluster_addr":"172.21.15.161:6827/3013292765","heartbeat_back_addr":"172.21.15.161:6831/3013292765","heartbeat_front_addr":"172.21.15.161:6829/3013292765","state":["exists","up"]},{"osd":11,"uuid":"356e5c54-bbc3-45ee-bda2-85164ce3cb92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6825","nonce":1010336458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6827","nonce":1010336458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6831","nonce":1010336458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6829","nonce":1010336458}]},"public_addr":"172.21.15.171:6825/1010336458","cluster_addr":"172.21.15.171:6827/1010336458","heartbeat_back_addr":"172.21.15.171:6831/1010336458","heartbeat_front_addr":"172.21.15.171:6829/1010336458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:36:51.024650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:17.026638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:41.239726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:15.148968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:41.029118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:06.225858+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:41.639444+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:07.753400+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:32.292085+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:07.159926+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:33.749782+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:58.594447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/181862246":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/3393454453":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3195850527":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6801/3468460974":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3392048544":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/1329430933":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/735578423":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/3468460974":"2024-04-26T18:34:20.605775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:44:14.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 -- 172.21.15.112:0/1372839458 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc6340a3cd0 msgr2=0x7fc6340a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:14.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc6340a3cd0 0x7fc6340a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc63800b2e0 tx=0x7fc638005960 comp rx=0 tx=0).stop 2024-04-25T18:44:14.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 -- 172.21.15.112:0/1372839458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc650074c70 msgr2=0x7fc65019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc650074c70 0x7fc65019b2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc64000ef10 tx=0x7fc64000c3a0 comp rx=0 tx=0).stop 2024-04-25T18:44:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 -- 172.21.15.112:0/1372839458 shutdown_connections 2024-04-25T18:44:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc6340a3cd0 0x7fc6340a6190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc65010f440 0x7fc65019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc650074c70 0x7fc65019b2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 --2- 172.21.15.112:0/1372839458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc650075590 0x7fc65019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 -- 172.21.15.112:0/1372839458 >> 172.21.15.112:0/1372839458 conn(0x7fc650079ba0 msgr2=0x7fc6500ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:14.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 -- 172.21.15.112:0/1372839458 shutdown_connections 2024-04-25T18:44:14.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:14.818+0000 7fc655600700 1 -- 172.21.15.112:0/1372839458 wait complete. 2024-04-25T18:44:15.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:14 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1372839458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:44:15.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:14 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1372839458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:44:15.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:14 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1372839458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:44:15.929 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T18:44:15.929 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:44:16.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:15 smithi171 ceph-mon[33266]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T18:44:16.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:15 smithi161 ceph-mon[32609]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T18:44:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:15 smithi112 ceph-mon[27864]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T18:44:16.385 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:18 smithi171 ceph-mon[33266]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-25T18:44:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:18 smithi161 ceph-mon[32609]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-25T18:44:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:18 smithi112 ceph-mon[27864]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-25T18:44:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.783+0000 7f02c4400700 1 -- 172.21.15.112:0/3818595757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02bc075590 msgr2=0x7f02bc10ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.783+0000 7f02c4400700 1 --2- 172.21.15.112:0/3818595757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02bc075590 0x7f02bc10ef00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009990 tx=0x7f02a8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:44:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.783+0000 7f02c4400700 1 -- 172.21.15.112:0/3818595757 shutdown_connections 2024-04-25T18:44:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.783+0000 7f02c4400700 1 --2- 172.21.15.112:0/3818595757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc10f440 0x7f02bc111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.783+0000 7f02c4400700 1 --2- 172.21.15.112:0/3818595757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02bc074c70 0x7f02bc075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.783+0000 7f02c4400700 1 --2- 172.21.15.112:0/3818595757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02bc075590 0x7f02bc10ef00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:18.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.783+0000 7f02c4400700 1 -- 172.21.15.112:0/3818595757 >> 172.21.15.112:0/3818595757 conn(0x7f02bc079ba0 msgr2=0x7f02bc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.783+0000 7f02c4400700 1 -- 172.21.15.112:0/3818595757 shutdown_connections 2024-04-25T18:44:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.783+0000 7f02c4400700 1 -- 172.21.15.112:0/3818595757 wait complete. 2024-04-25T18:44:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.784+0000 7f02c4400700 1 Processor -- start 2024-04-25T18:44:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.784+0000 7f02c4400700 1 -- start start 2024-04-25T18:44:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c4400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02bc074c70 0x7f02bc102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c4400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02bc075590 0x7f02bc1030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c4400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc10f440 0x7f02bc1035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c4400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc114e00 con 0x7f02bc074c70 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c4400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc114c80 con 0x7f02bc075590 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c4400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc114f80 con 0x7f02bc10f440 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c1000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02bc075590 0x7f02bc1030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c1a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02bc074c70 0x7f02bc102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c1000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02bc075590 0x7f02bc1030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:60538/0 (socket says 172.21.15.112:60538) 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c1000700 1 -- 172.21.15.112:0/2075525717 learned_addr learned my addr 172.21.15.112:0/2075525717 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c2400700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc10f440 0x7f02bc1035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c1000700 1 -- 172.21.15.112:0/2075525717 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc10f440 msgr2=0x7f02bc1035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c1000700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc10f440 0x7f02bc1035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c1000700 1 -- 172.21.15.112:0/2075525717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02bc074c70 msgr2=0x7f02bc102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c1000700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02bc074c70 0x7f02bc102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:18.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c1000700 1 -- 172.21.15.112:0/2075525717 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc1ab9e0 con 0x7f02bc075590 2024-04-25T18:44:18.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.785+0000 7f02c1a00700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02bc074c70 0x7f02bc102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:44:18.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.786+0000 7f02c1000700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02bc075590 0x7f02bc1030a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f02a8005ea0 tx=0x7f02a8005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:18.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.786+0000 7f02ba000700 1 -- 172.21.15.112:0/2075525717 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a800fb00 con 0x7f02bc075590 2024-04-25T18:44:18.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.786+0000 7f02ba000700 1 -- 172.21.15.112:0/2075525717 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02a8004cc0 con 0x7f02bc075590 2024-04-25T18:44:18.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.786+0000 7f02c4400700 1 -- 172.21.15.112:0/2075525717 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02bc1abc70 con 0x7f02bc075590 2024-04-25T18:44:18.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.786+0000 7f02ba000700 1 -- 172.21.15.112:0/2075525717 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a80054c0 con 0x7f02bc075590 2024-04-25T18:44:18.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.786+0000 7f02c4400700 1 -- 172.21.15.112:0/2075525717 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02bc1ac060 con 0x7f02bc075590 2024-04-25T18:44:18.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.787+0000 7f02c4400700 1 -- 172.21.15.112:0/2075525717 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02bc076360 con 0x7f02bc075590 2024-04-25T18:44:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.789+0000 7f02ba000700 1 -- 172.21.15.112:0/2075525717 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f02a8009d10 con 0x7f02bc075590 2024-04-25T18:44:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.790+0000 7f02ba000700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02a40a3cd0 0x7f02a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.791+0000 7f02ba000700 1 -- 172.21.15.112:0/2075525717 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f02a80d1070 con 0x7f02bc075590 2024-04-25T18:44:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.791+0000 7f02c1a00700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02a40a3cd0 0x7f02a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.791+0000 7f02c1a00700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02a40a3cd0 0x7f02a40a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009480 tx=0x7f02b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:18.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:18.794+0000 7f02ba000700 1 -- 172.21.15.112:0/2075525717 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02a8013070 con 0x7f02bc075590 2024-04-25T18:44:19.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.014+0000 7f02c4400700 1 -- 172.21.15.112:0/2075525717 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02bc02dff0 con 0x7f02a40a3cd0 2024-04-25T18:44:19.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.039+0000 7f02ba000700 1 -- 172.21.15.112:0/2075525717 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258764 (secure 0 0 0) 0x7f02bc02dff0 con 0x7f02a40a3cd0 2024-04-25T18:44:19.047 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:44:19.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 -- 172.21.15.112:0/2075525717 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02a40a3cd0 msgr2=0x7f02a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:19.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02a40a3cd0 0x7f02a40a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009480 tx=0x7f02b000b040 comp rx=0 tx=0).stop 2024-04-25T18:44:19.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 -- 172.21.15.112:0/2075525717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02bc075590 msgr2=0x7f02bc1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:19.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02bc075590 0x7f02bc1030a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f02a8005ea0 tx=0x7f02a8005ed0 comp rx=0 tx=0).stop 2024-04-25T18:44:19.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 -- 172.21.15.112:0/2075525717 shutdown_connections 2024-04-25T18:44:19.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f02a40a3cd0 0x7f02a40a6190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:19.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc10f440 0x7f02bc1035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:19.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02bc074c70 0x7f02bc102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:19.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 --2- 172.21.15.112:0/2075525717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02bc075590 0x7f02bc1030a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:19.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 -- 172.21.15.112:0/2075525717 >> 172.21.15.112:0/2075525717 conn(0x7f02bc079ba0 msgr2=0x7f02bc0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:19.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 -- 172.21.15.112:0/2075525717 shutdown_connections 2024-04-25T18:44:19.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:19.045+0000 7f02c4400700 1 -- 172.21.15.112:0/2075525717 wait complete. 2024-04-25T18:44:19.051 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T18:44:19.914 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-25T18:44:17.118643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657675,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":357,"num_read_kb":299,"num_write":348,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181318,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375840,"kb_used_data":14184,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565984,"statfs":{"total":1151940427776,"available":1151555567616,"internally_reserved":0,"allocated":14524416,"data_stored":7951125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17678,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":62,"num_read_kb":48,"num_write":12,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004901"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630551+0000","last_change":"2024-04-25T18:43:16.262618+0000","last_active":"2024-04-25T18:43:56.630551+0000","last_peered":"2024-04-25T18:43:56.630551+0000","last_clean":"2024-04-25T18:43:56.630551+0000","last_became_active":"2024-04-25T18:43:16.262215+0000","last_became_peered":"2024-04-25T18:43:16.262215+0000","last_unstale":"2024-04-25T18:43:56.630551+0000","last_undegraded":"2024-04-25T18:43:56.630551+0000","last_fullsized":"2024-04-25T18:43:56.630551+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:40.319102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.636714+0000","last_change":"2024-04-25T18:43:16.254085+0000","last_active":"2024-04-25T18:43:56.636714+0000","last_peered":"2024-04-25T18:43:56.636714+0000","last_clean":"2024-04-25T18:43:56.636714+0000","last_became_active":"2024-04-25T18:43:16.248943+0000","last_became_peered":"2024-04-25T18:43:16.248943+0000","last_unstale":"2024-04-25T18:43:56.636714+0000","last_undegraded":"2024-04-25T18:43:56.636714+0000","last_fullsized":"2024-04-25T18:43:56.636714+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:53.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.638807+0000","last_change":"2024-04-25T18:43:16.252684+0000","last_active":"2024-04-25T18:43:56.638807+0000","last_peered":"2024-04-25T18:43:56.638807+0000","last_clean":"2024-04-25T18:43:56.638807+0000","last_became_active":"2024-04-25T18:43:16.252318+0000","last_became_peered":"2024-04-25T18:43:16.252318+0000","last_unstale":"2024-04-25T18:43:56.638807+0000","last_undegraded":"2024-04-25T18:43:56.638807+0000","last_fullsized":"2024-04-25T18:43:56.638807+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:58.904865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628754+0000","last_change":"2024-04-25T18:43:16.247059+0000","last_active":"2024-04-25T18:43:56.628754+0000","last_peered":"2024-04-25T18:43:56.628754+0000","last_clean":"2024-04-25T18:43:56.628754+0000","last_became_active":"2024-04-25T18:43:16.246655+0000","last_became_peered":"2024-04-25T18:43:16.246655+0000","last_unstale":"2024-04-25T18:43:56.628754+0000","last_undegraded":"2024-04-25T18:43:56.628754+0000","last_fullsized":"2024-04-25T18:43:56.628754+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:31.026579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628239+0000","last_change":"2024-04-25T18:43:16.250033+0000","last_active":"2024-04-25T18:43:56.628239+0000","last_peered":"2024-04-25T18:43:56.628239+0000","last_clean":"2024-04-25T18:43:56.628239+0000","last_became_active":"2024-04-25T18:43:16.249017+0000","last_became_peered":"2024-04-25T18:43:16.249017+0000","last_unstale":"2024-04-25T18:43:56.628239+0000","last_undegraded":"2024-04-25T18:43:56.628239+0000","last_fullsized":"2024-04-25T18:43:56.628239+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:34.304263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.642410+0000","last_change":"2024-04-25T18:43:56.642410+0000","last_active":"2024-04-25T18:43:56.642410+0000","last_peered":"2024-04-25T18:43:56.642410+0000","last_clean":"2024-04-25T18:43:56.642410+0000","last_became_active":"2024-04-25T18:43:56.642009+0000","last_became_peered":"2024-04-25T18:43:56.642009+0000","last_unstale":"2024-04-25T18:43:56.642410+0000","last_undegraded":"2024-04-25T18:43:56.642410+0000","last_fullsized":"2024-04-25T18:43:56.642410+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:19.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.647158+0000","last_change":"2024-04-25T18:43:56.647158+0000","last_active":"2024-04-25T18:43:56.647158+0000","last_peered":"2024-04-25T18:43:56.647158+0000","last_clean":"2024-04-25T18:43:56.647158+0000","last_became_active":"2024-04-25T18:43:56.646779+0000","last_became_peered":"2024-04-25T18:43:56.646779+0000","last_unstale":"2024-04-25T18:43:56.647158+0000","last_undegraded":"2024-04-25T18:43:56.647158+0000","last_fullsized":"2024-04-25T18:43:56.647158+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:04:14.055762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.637895+0000","last_change":"2024-04-25T18:43:16.247607+0000","last_active":"2024-04-25T18:43:56.637895+0000","last_peered":"2024-04-25T18:43:56.637895+0000","last_clean":"2024-04-25T18:43:56.637895+0000","last_became_active":"2024-04-25T18:43:16.247234+0000","last_became_peered":"2024-04-25T18:43:16.247234+0000","last_unstale":"2024-04-25T18:43:56.637895+0000","last_undegraded":"2024-04-25T18:43:56.637895+0000","last_fullsized":"2024-04-25T18:43:56.637895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:57.117900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.105056+0000","last_change":"2024-04-25T18:43:16.247909+0000","last_active":"2024-04-25T18:43:57.105056+0000","last_peered":"2024-04-25T18:43:57.105056+0000","last_clean":"2024-04-25T18:43:57.105056+0000","last_became_active":"2024-04-25T18:43:16.247533+0000","last_became_peered":"2024-04-25T18:43:16.247533+0000","last_unstale":"2024-04-25T18:43:57.105056+0000","last_undegraded":"2024-04-25T18:43:57.105056+0000","last_fullsized":"2024-04-25T18:43:57.105056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:02.257832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626680+0000","last_change":"2024-04-25T18:43:16.252088+0000","last_active":"2024-04-25T18:43:56.626680+0000","last_peered":"2024-04-25T18:43:56.626680+0000","last_clean":"2024-04-25T18:43:56.626680+0000","last_became_active":"2024-04-25T18:43:16.251580+0000","last_became_peered":"2024-04-25T18:43:16.251580+0000","last_unstale":"2024-04-25T18:43:56.626680+0000","last_undegraded":"2024-04-25T18:43:56.626680+0000","last_fullsized":"2024-04-25T18:43:56.626680+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:22.992087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630307+0000","last_change":"2024-04-25T18:43:16.239696+0000","last_active":"2024-04-25T18:43:56.630307+0000","last_peered":"2024-04-25T18:43:56.630307+0000","last_clean":"2024-04-25T18:43:56.630307+0000","last_became_active":"2024-04-25T18:43:16.239354+0000","last_became_peered":"2024-04-25T18:43:16.239354+0000","last_unstale":"2024-04-25T18:43:56.630307+0000","last_undegraded":"2024-04-25T18:43:56.630307+0000","last_fullsized":"2024-04-25T18:43:56.630307+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:39.682138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628528+0000","last_change":"2024-04-25T18:43:16.242487+0000","last_active":"2024-04-25T18:43:56.628528+0000","last_peered":"2024-04-25T18:43:56.628528+0000","last_clean":"2024-04-25T18:43:56.628528+0000","last_became_active":"2024-04-25T18:43:16.242158+0000","last_became_peered":"2024-04-25T18:43:16.242158+0000","last_unstale":"2024-04-25T18:43:56.628528+0000","last_undegraded":"2024-04-25T18:43:56.628528+0000","last_fullsized":"2024-04-25T18:43:56.628528+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:36.477440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630372+0000","last_change":"2024-04-25T18:43:16.244154+0000","last_active":"2024-04-25T18:43:56.630372+0000","last_peered":"2024-04-25T18:43:56.630372+0000","last_clean":"2024-04-25T18:43:56.630372+0000","last_became_active":"2024-04-25T18:43:16.243829+0000","last_became_peered":"2024-04-25T18:43:16.243829+0000","last_unstale":"2024-04-25T18:43:56.630372+0000","last_undegraded":"2024-04-25T18:43:56.630372+0000","last_fullsized":"2024-04-25T18:43:56.630372+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:43.262751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.636692+0000","last_change":"2024-04-25T18:43:16.261227+0000","last_active":"2024-04-25T18:43:56.636692+0000","last_peered":"2024-04-25T18:43:56.636692+0000","last_clean":"2024-04-25T18:43:56.636692+0000","last_became_active":"2024-04-25T18:43:16.250468+0000","last_became_peered":"2024-04-25T18:43:16.250468+0000","last_unstale":"2024-04-25T18:43:56.636692+0000","last_undegraded":"2024-04-25T18:43:56.636692+0000","last_fullsized":"2024-04-25T18:43:56.636692+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:44.805643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.636244+0000","last_change":"2024-04-25T18:43:16.257956+0000","last_active":"2024-04-25T18:43:56.636244+0000","last_peered":"2024-04-25T18:43:56.636244+0000","last_clean":"2024-04-25T18:43:56.636244+0000","last_became_active":"2024-04-25T18:43:16.255126+0000","last_became_peered":"2024-04-25T18:43:16.255126+0000","last_unstale":"2024-04-25T18:43:56.636244+0000","last_undegraded":"2024-04-25T18:43:56.636244+0000","last_fullsized":"2024-04-25T18:43:56.636244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:30.047165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104512+0000","last_change":"2024-04-25T18:43:16.256208+0000","last_active":"2024-04-25T18:43:57.104512+0000","last_peered":"2024-04-25T18:43:57.104512+0000","last_clean":"2024-04-25T18:43:57.104512+0000","last_became_active":"2024-04-25T18:43:16.255614+0000","last_became_peered":"2024-04-25T18:43:16.255614+0000","last_unstale":"2024-04-25T18:43:57.104512+0000","last_undegraded":"2024-04-25T18:43:57.104512+0000","last_fullsized":"2024-04-25T18:43:57.104512+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:46.700209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630798+0000","last_change":"2024-04-25T18:43:16.262730+0000","last_active":"2024-04-25T18:43:56.630798+0000","last_peered":"2024-04-25T18:43:56.630798+0000","last_clean":"2024-04-25T18:43:56.630798+0000","last_became_active":"2024-04-25T18:43:16.262192+0000","last_became_peered":"2024-04-25T18:43:16.262192+0000","last_unstale":"2024-04-25T18:43:56.630798+0000","last_undegraded":"2024-04-25T18:43:56.630798+0000","last_fullsized":"2024-04-25T18:43:56.630798+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:29.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.643024+0000","last_change":"2024-04-25T18:43:16.261161+0000","last_active":"2024-04-25T18:43:56.643024+0000","last_peered":"2024-04-25T18:43:56.643024+0000","last_clean":"2024-04-25T18:43:56.643024+0000","last_became_active":"2024-04-25T18:43:16.258217+0000","last_became_peered":"2024-04-25T18:43:16.258217+0000","last_unstale":"2024-04-25T18:43:56.643024+0000","last_undegraded":"2024-04-25T18:43:56.643024+0000","last_fullsized":"2024-04-25T18:43:56.643024+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.564609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626029+0000","last_change":"2024-04-25T18:43:16.266660+0000","last_active":"2024-04-25T18:43:56.626029+0000","last_peered":"2024-04-25T18:43:56.626029+0000","last_clean":"2024-04-25T18:43:56.626029+0000","last_became_active":"2024-04-25T18:43:16.266295+0000","last_became_peered":"2024-04-25T18:43:16.266295+0000","last_unstale":"2024-04-25T18:43:56.626029+0000","last_undegraded":"2024-04-25T18:43:56.626029+0000","last_fullsized":"2024-04-25T18:43:56.626029+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:42.406585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.643296+0000","last_change":"2024-04-25T18:43:12.200172+0000","last_active":"2024-04-25T18:43:56.643296+0000","last_peered":"2024-04-25T18:43:56.643296+0000","last_clean":"2024-04-25T18:43:56.643296+0000","last_became_active":"2024-04-25T18:43:12.199760+0000","last_became_peered":"2024-04-25T18:43:12.199760+0000","last_unstale":"2024-04-25T18:43:56.643296+0000","last_undegraded":"2024-04-25T18:43:56.643296+0000","last_fullsized":"2024-04-25T18:43:56.643296+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:57.460274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.638451+0000","last_change":"2024-04-25T18:43:16.255989+0000","last_active":"2024-04-25T18:43:56.638451+0000","last_peered":"2024-04-25T18:43:56.638451+0000","last_clean":"2024-04-25T18:43:56.638451+0000","last_became_active":"2024-04-25T18:43:16.255114+0000","last_became_peered":"2024-04-25T18:43:16.255114+0000","last_unstale":"2024-04-25T18:43:56.638451+0000","last_undegraded":"2024-04-25T18:43:56.638451+0000","last_fullsized":"2024-04-25T18:43:56.638451+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:52.888876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.639575+0000","last_change":"2024-04-25T18:43:12.208194+0000","last_active":"2024-04-25T18:43:56.639575+0000","last_peered":"2024-04-25T18:43:56.639575+0000","last_clean":"2024-04-25T18:43:56.639575+0000","last_became_active":"2024-04-25T18:43:12.207801+0000","last_became_peered":"2024-04-25T18:43:12.207801+0000","last_unstale":"2024-04-25T18:43:56.639575+0000","last_undegraded":"2024-04-25T18:43:56.639575+0000","last_fullsized":"2024-04-25T18:43:56.639575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:24.742486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.644607+0000","last_change":"2024-04-25T18:43:16.268960+0000","last_active":"2024-04-25T18:43:56.644607+0000","last_peered":"2024-04-25T18:43:56.644607+0000","last_clean":"2024-04-25T18:43:56.644607+0000","last_became_active":"2024-04-25T18:43:16.263626+0000","last_became_peered":"2024-04-25T18:43:16.263626+0000","last_unstale":"2024-04-25T18:43:56.644607+0000","last_undegraded":"2024-04-25T18:43:56.644607+0000","last_fullsized":"2024-04-25T18:43:56.644607+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:57.721918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.110208+0000","last_change":"2024-04-25T18:43:57.110208+0000","last_active":"2024-04-25T18:43:57.110208+0000","last_peered":"2024-04-25T18:43:57.110208+0000","last_clean":"2024-04-25T18:43:57.110208+0000","last_became_active":"2024-04-25T18:43:57.106027+0000","last_became_peered":"2024-04-25T18:43:57.106027+0000","last_unstale":"2024-04-25T18:43:57.110208+0000","last_undegraded":"2024-04-25T18:43:57.110208+0000","last_fullsized":"2024-04-25T18:43:57.110208+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:20.819961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104747+0000","last_change":"2024-04-25T18:43:16.250414+0000","last_active":"2024-04-25T18:43:57.104747+0000","last_peered":"2024-04-25T18:43:57.104747+0000","last_clean":"2024-04-25T18:43:57.104747+0000","last_became_active":"2024-04-25T18:43:16.249832+0000","last_became_peered":"2024-04-25T18:43:16.249832+0000","last_unstale":"2024-04-25T18:43:57.104747+0000","last_undegraded":"2024-04-25T18:43:57.104747+0000","last_fullsized":"2024-04-25T18:43:57.104747+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.577854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628806+0000","last_change":"2024-04-25T18:43:12.195099+0000","last_active":"2024-04-25T18:43:56.628806+0000","last_peered":"2024-04-25T18:43:56.628806+0000","last_clean":"2024-04-25T18:43:56.628806+0000","last_became_active":"2024-04-25T18:43:12.194726+0000","last_became_peered":"2024-04-25T18:43:12.194726+0000","last_unstale":"2024-04-25T18:43:56.628806+0000","last_undegraded":"2024-04-25T18:43:56.628806+0000","last_fullsized":"2024-04-25T18:43:56.628806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.866905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632966+0000","last_change":"2024-04-25T18:43:16.245208+0000","last_active":"2024-04-25T18:43:56.632966+0000","last_peered":"2024-04-25T18:43:56.632966+0000","last_clean":"2024-04-25T18:43:56.632966+0000","last_became_active":"2024-04-25T18:43:16.244836+0000","last_became_peered":"2024-04-25T18:43:16.244836+0000","last_unstale":"2024-04-25T18:43:56.632966+0000","last_undegraded":"2024-04-25T18:43:56.632966+0000","last_fullsized":"2024-04-25T18:43:56.632966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:35.917060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":67,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:11.313551+0000","last_change":"2024-04-25T18:43:12.198252+0000","last_active":"2024-04-25T18:44:11.313551+0000","last_peered":"2024-04-25T18:44:11.313551+0000","last_clean":"2024-04-25T18:44:11.313551+0000","last_became_active":"2024-04-25T18:43:12.197691+0000","last_became_peered":"2024-04-25T18:43:12.197691+0000","last_unstale":"2024-04-25T18:44:11.313551+0000","last_undegraded":"2024-04-25T18:44:11.313551+0000","last_fullsized":"2024-04-25T18:44:11.313551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.900012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628228+0000","last_change":"2024-04-25T18:43:16.255576+0000","last_active":"2024-04-25T18:43:56.628228+0000","last_peered":"2024-04-25T18:43:56.628228+0000","last_clean":"2024-04-25T18:43:56.628228+0000","last_became_active":"2024-04-25T18:43:16.255195+0000","last_became_peered":"2024-04-25T18:43:16.255195+0000","last_unstale":"2024-04-25T18:43:56.628228+0000","last_undegraded":"2024-04-25T18:43:56.628228+0000","last_fullsized":"2024-04-25T18:43:56.628228+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:41.510872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.625977+0000","last_change":"2024-04-25T18:43:12.200786+0000","last_active":"2024-04-25T18:43:56.625977+0000","last_peered":"2024-04-25T18:43:56.625977+0000","last_clean":"2024-04-25T18:43:56.625977+0000","last_became_active":"2024-04-25T18:43:12.200409+0000","last_became_peered":"2024-04-25T18:43:12.200409+0000","last_unstale":"2024-04-25T18:43:56.625977+0000","last_undegraded":"2024-04-25T18:43:56.625977+0000","last_fullsized":"2024-04-25T18:43:56.625977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:52.177787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632730+0000","last_change":"2024-04-25T18:43:16.255563+0000","last_active":"2024-04-25T18:43:56.632730+0000","last_peered":"2024-04-25T18:43:56.632730+0000","last_clean":"2024-04-25T18:43:56.632730+0000","last_became_active":"2024-04-25T18:43:16.255174+0000","last_became_peered":"2024-04-25T18:43:16.255174+0000","last_unstale":"2024-04-25T18:43:56.632730+0000","last_undegraded":"2024-04-25T18:43:56.632730+0000","last_fullsized":"2024-04-25T18:43:56.632730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:32:15.735051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628776+0000","last_change":"2024-04-25T18:43:12.210242+0000","last_active":"2024-04-25T18:43:56.628776+0000","last_peered":"2024-04-25T18:43:56.628776+0000","last_clean":"2024-04-25T18:43:56.628776+0000","last_became_active":"2024-04-25T18:43:12.209600+0000","last_became_peered":"2024-04-25T18:43:12.209600+0000","last_unstale":"2024-04-25T18:43:56.628776+0000","last_undegraded":"2024-04-25T18:43:56.628776+0000","last_fullsized":"2024-04-25T18:43:56.628776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.886712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.652388+0000","last_change":"2024-04-25T18:43:56.649102+0000","last_active":"2024-04-25T18:43:56.652388+0000","last_peered":"2024-04-25T18:43:56.652388+0000","last_clean":"2024-04-25T18:43:56.652388+0000","last_became_active":"2024-04-25T18:43:56.647127+0000","last_became_peered":"2024-04-25T18:43:56.647127+0000","last_unstale":"2024-04-25T18:43:56.652388+0000","last_undegraded":"2024-04-25T18:43:56.652388+0000","last_fullsized":"2024-04-25T18:43:56.652388+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:43.111539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632516+0000","last_change":"2024-04-25T18:43:12.198957+0000","last_active":"2024-04-25T18:43:56.632516+0000","last_peered":"2024-04-25T18:43:56.632516+0000","last_clean":"2024-04-25T18:43:56.632516+0000","last_became_active":"2024-04-25T18:43:12.198605+0000","last_became_peered":"2024-04-25T18:43:12.198605+0000","last_unstale":"2024-04-25T18:43:56.632516+0000","last_undegraded":"2024-04-25T18:43:56.632516+0000","last_fullsized":"2024-04-25T18:43:56.632516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:11.759430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.642145+0000","last_change":"2024-04-25T18:43:16.266829+0000","last_active":"2024-04-25T18:43:56.642145+0000","last_peered":"2024-04-25T18:43:56.642145+0000","last_clean":"2024-04-25T18:43:56.642145+0000","last_became_active":"2024-04-25T18:43:16.266256+0000","last_became_peered":"2024-04-25T18:43:16.266256+0000","last_unstale":"2024-04-25T18:43:56.642145+0000","last_undegraded":"2024-04-25T18:43:56.642145+0000","last_fullsized":"2024-04-25T18:43:56.642145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:43.343613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":61,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:11.312977+0000","last_change":"2024-04-25T18:43:12.201641+0000","last_active":"2024-04-25T18:44:11.312977+0000","last_peered":"2024-04-25T18:44:11.312977+0000","last_clean":"2024-04-25T18:44:11.312977+0000","last_became_active":"2024-04-25T18:43:12.201274+0000","last_became_peered":"2024-04-25T18:43:12.201274+0000","last_unstale":"2024-04-25T18:44:11.312977+0000","last_undegraded":"2024-04-25T18:44:11.312977+0000","last_fullsized":"2024-04-25T18:44:11.312977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:47.683850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.105043+0000","last_change":"2024-04-25T18:43:16.259551+0000","last_active":"2024-04-25T18:43:57.105043+0000","last_peered":"2024-04-25T18:43:57.105043+0000","last_clean":"2024-04-25T18:43:57.105043+0000","last_became_active":"2024-04-25T18:43:16.259165+0000","last_became_peered":"2024-04-25T18:43:16.259165+0000","last_unstale":"2024-04-25T18:43:57.105043+0000","last_undegraded":"2024-04-25T18:43:57.105043+0000","last_fullsized":"2024-04-25T18:43:57.105043+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:23.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630280+0000","last_change":"2024-04-25T18:43:12.197534+0000","last_active":"2024-04-25T18:43:56.630280+0000","last_peered":"2024-04-25T18:43:56.630280+0000","last_clean":"2024-04-25T18:43:56.630280+0000","last_became_active":"2024-04-25T18:43:12.197106+0000","last_became_peered":"2024-04-25T18:43:12.197106+0000","last_unstale":"2024-04-25T18:43:56.630280+0000","last_undegraded":"2024-04-25T18:43:56.630280+0000","last_fullsized":"2024-04-25T18:43:56.630280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:35.399044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.644573+0000","last_change":"2024-04-25T18:43:16.257668+0000","last_active":"2024-04-25T18:43:56.644573+0000","last_peered":"2024-04-25T18:43:56.644573+0000","last_clean":"2024-04-25T18:43:56.644573+0000","last_became_active":"2024-04-25T18:43:16.253609+0000","last_became_peered":"2024-04-25T18:43:16.253609+0000","last_unstale":"2024-04-25T18:43:56.644573+0000","last_undegraded":"2024-04-25T18:43:56.644573+0000","last_fullsized":"2024-04-25T18:43:56.644573+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:36.048386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.633047+0000","last_change":"2024-04-25T18:43:12.207322+0000","last_active":"2024-04-25T18:43:56.633047+0000","last_peered":"2024-04-25T18:43:56.633047+0000","last_clean":"2024-04-25T18:43:56.633047+0000","last_became_active":"2024-04-25T18:43:12.206947+0000","last_became_peered":"2024-04-25T18:43:12.206947+0000","last_unstale":"2024-04-25T18:43:56.633047+0000","last_undegraded":"2024-04-25T18:43:56.633047+0000","last_fullsized":"2024-04-25T18:43:56.633047+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:20.536598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104798+0000","last_change":"2024-04-25T18:43:16.239575+0000","last_active":"2024-04-25T18:43:57.104798+0000","last_peered":"2024-04-25T18:43:57.104798+0000","last_clean":"2024-04-25T18:43:57.104798+0000","last_became_active":"2024-04-25T18:43:16.239208+0000","last_became_peered":"2024-04-25T18:43:16.239208+0000","last_unstale":"2024-04-25T18:43:57.104798+0000","last_undegraded":"2024-04-25T18:43:57.104798+0000","last_fullsized":"2024-04-25T18:43:57.104798+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:58.195855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'30","reported_seq":128,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:12.340265+0000","last_change":"2024-04-25T18:43:12.197199+0000","last_active":"2024-04-25T18:44:12.340265+0000","last_peered":"2024-04-25T18:44:12.340265+0000","last_clean":"2024-04-25T18:44:12.340265+0000","last_became_active":"2024-04-25T18:43:12.196776+0000","last_became_peered":"2024-04-25T18:43:12.196776+0000","last_unstale":"2024-04-25T18:44:12.340265+0000","last_undegraded":"2024-04-25T18:44:12.340265+0000","last_fullsized":"2024-04-25T18:44:12.340265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:54.149782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":72,"num_read_kb":97,"num_write":31,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628449+0000","last_change":"2024-04-25T18:43:16.250993+0000","last_active":"2024-04-25T18:43:56.628449+0000","last_peered":"2024-04-25T18:43:56.628449+0000","last_clean":"2024-04-25T18:43:56.628449+0000","last_became_active":"2024-04-25T18:43:16.250216+0000","last_became_peered":"2024-04-25T18:43:16.250216+0000","last_unstale":"2024-04-25T18:43:56.628449+0000","last_undegraded":"2024-04-25T18:43:56.628449+0000","last_fullsized":"2024-04-25T18:43:56.628449+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:37.996463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626135+0000","last_change":"2024-04-25T18:43:12.204606+0000","last_active":"2024-04-25T18:43:56.626135+0000","last_peered":"2024-04-25T18:43:56.626135+0000","last_clean":"2024-04-25T18:43:56.626135+0000","last_became_active":"2024-04-25T18:43:12.204306+0000","last_became_peered":"2024-04-25T18:43:12.204306+0000","last_unstale":"2024-04-25T18:43:56.626135+0000","last_undegraded":"2024-04-25T18:43:56.626135+0000","last_fullsized":"2024-04-25T18:43:56.626135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:29.903775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628584+0000","last_change":"2024-04-25T18:43:16.250093+0000","last_active":"2024-04-25T18:43:56.628584+0000","last_peered":"2024-04-25T18:43:56.628584+0000","last_clean":"2024-04-25T18:43:56.628584+0000","last_became_active":"2024-04-25T18:43:16.249567+0000","last_became_peered":"2024-04-25T18:43:16.249567+0000","last_unstale":"2024-04-25T18:43:56.628584+0000","last_undegraded":"2024-04-25T18:43:56.628584+0000","last_fullsized":"2024-04-25T18:43:56.628584+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:35.498641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.652890+0000","last_change":"2024-04-25T18:43:56.652488+0000","last_active":"2024-04-25T18:43:56.652890+0000","last_peered":"2024-04-25T18:43:56.652890+0000","last_clean":"2024-04-25T18:43:56.652890+0000","last_became_active":"2024-04-25T18:43:56.647745+0000","last_became_peered":"2024-04-25T18:43:56.647745+0000","last_unstale":"2024-04-25T18:43:56.652890+0000","last_undegraded":"2024-04-25T18:43:56.652890+0000","last_fullsized":"2024-04-25T18:43:56.652890+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:33.474470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.642673+0000","last_change":"2024-04-25T18:43:56.642673+0000","last_active":"2024-04-25T18:43:56.642673+0000","last_peered":"2024-04-25T18:43:56.642673+0000","last_clean":"2024-04-25T18:43:56.642673+0000","last_became_active":"2024-04-25T18:43:56.642313+0000","last_became_peered":"2024-04-25T18:43:56.642313+0000","last_unstale":"2024-04-25T18:43:56.642673+0000","last_undegraded":"2024-04-25T18:43:56.642673+0000","last_fullsized":"2024-04-25T18:43:56.642673+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:45.805738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":65,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:13.328375+0000","last_change":"2024-04-25T18:43:12.197341+0000","last_active":"2024-04-25T18:44:13.328375+0000","last_peered":"2024-04-25T18:44:13.328375+0000","last_clean":"2024-04-25T18:44:13.328375+0000","last_became_active":"2024-04-25T18:43:12.196952+0000","last_became_peered":"2024-04-25T18:43:12.196952+0000","last_unstale":"2024-04-25T18:44:13.328375+0000","last_undegraded":"2024-04-25T18:44:13.328375+0000","last_fullsized":"2024-04-25T18:44:13.328375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:52.902579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.629893+0000","last_change":"2024-04-25T18:43:16.245436+0000","last_active":"2024-04-25T18:43:56.629893+0000","last_peered":"2024-04-25T18:43:56.629893+0000","last_clean":"2024-04-25T18:43:56.629893+0000","last_became_active":"2024-04-25T18:43:16.244943+0000","last_became_peered":"2024-04-25T18:43:16.244943+0000","last_unstale":"2024-04-25T18:43:56.629893+0000","last_undegraded":"2024-04-25T18:43:56.629893+0000","last_fullsized":"2024-04-25T18:43:56.629893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:03.402999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628089+0000","last_change":"2024-04-25T18:43:12.197642+0000","last_active":"2024-04-25T18:43:56.628089+0000","last_peered":"2024-04-25T18:43:56.628089+0000","last_clean":"2024-04-25T18:43:56.628089+0000","last_became_active":"2024-04-25T18:43:12.196395+0000","last_became_peered":"2024-04-25T18:43:12.196395+0000","last_unstale":"2024-04-25T18:43:56.628089+0000","last_undegraded":"2024-04-25T18:43:56.628089+0000","last_fullsized":"2024-04-25T18:43:56.628089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:11.783422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632381+0000","last_change":"2024-04-25T18:43:16.245344+0000","last_active":"2024-04-25T18:43:56.632381+0000","last_peered":"2024-04-25T18:43:56.632381+0000","last_clean":"2024-04-25T18:43:56.632381+0000","last_became_active":"2024-04-25T18:43:16.244984+0000","last_became_peered":"2024-04-25T18:43:16.244984+0000","last_unstale":"2024-04-25T18:43:56.632381+0000","last_undegraded":"2024-04-25T18:43:56.632381+0000","last_fullsized":"2024-04-25T18:43:56.632381+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:04.822629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628582+0000","last_change":"2024-04-25T18:43:12.199685+0000","last_active":"2024-04-25T18:43:56.628582+0000","last_peered":"2024-04-25T18:43:56.628582+0000","last_clean":"2024-04-25T18:43:56.628582+0000","last_became_active":"2024-04-25T18:43:12.199261+0000","last_became_peered":"2024-04-25T18:43:12.199261+0000","last_unstale":"2024-04-25T18:43:56.628582+0000","last_undegraded":"2024-04-25T18:43:56.628582+0000","last_fullsized":"2024-04-25T18:43:56.628582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:09.784616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630711+0000","last_change":"2024-04-25T18:43:16.247745+0000","last_active":"2024-04-25T18:43:56.630711+0000","last_peered":"2024-04-25T18:43:56.630711+0000","last_clean":"2024-04-25T18:43:56.630711+0000","last_became_active":"2024-04-25T18:43:16.247311+0000","last_became_peered":"2024-04-25T18:43:16.247311+0000","last_unstale":"2024-04-25T18:43:56.630711+0000","last_undegraded":"2024-04-25T18:43:56.630711+0000","last_fullsized":"2024-04-25T18:43:56.630711+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:29.925464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.642616+0000","last_change":"2024-04-25T18:43:12.210209+0000","last_active":"2024-04-25T18:43:56.642616+0000","last_peered":"2024-04-25T18:43:56.642616+0000","last_clean":"2024-04-25T18:43:56.642616+0000","last_became_active":"2024-04-25T18:43:12.209708+0000","last_became_peered":"2024-04-25T18:43:12.209708+0000","last_unstale":"2024-04-25T18:43:56.642616+0000","last_undegraded":"2024-04-25T18:43:56.642616+0000","last_fullsized":"2024-04-25T18:43:56.642616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:07:18.331518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'1","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626295+0000","last_change":"2024-04-25T18:43:16.263616+0000","last_active":"2024-04-25T18:43:56.626295+0000","last_peered":"2024-04-25T18:43:56.626295+0000","last_clean":"2024-04-25T18:43:56.626295+0000","last_became_active":"2024-04-25T18:43:16.263287+0000","last_became_peered":"2024-04-25T18:43:16.263287+0000","last_unstale":"2024-04-25T18:43:56.626295+0000","last_undegraded":"2024-04-25T18:43:56.626295+0000","last_fullsized":"2024-04-25T18:43:56.626295+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:46.366457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630314+0000","last_change":"2024-04-25T18:43:12.203705+0000","last_active":"2024-04-25T18:43:56.630314+0000","last_peered":"2024-04-25T18:43:56.630314+0000","last_clean":"2024-04-25T18:43:56.630314+0000","last_became_active":"2024-04-25T18:43:12.203371+0000","last_became_peered":"2024-04-25T18:43:12.203371+0000","last_unstale":"2024-04-25T18:43:56.630314+0000","last_undegraded":"2024-04-25T18:43:56.630314+0000","last_fullsized":"2024-04-25T18:43:56.630314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.647597+0000","last_change":"2024-04-25T18:43:56.647597+0000","last_active":"2024-04-25T18:43:56.647597+0000","last_peered":"2024-04-25T18:43:56.647597+0000","last_clean":"2024-04-25T18:43:56.647597+0000","last_became_active":"2024-04-25T18:43:56.647121+0000","last_became_peered":"2024-04-25T18:43:56.647121+0000","last_unstale":"2024-04-25T18:43:56.647597+0000","last_undegraded":"2024-04-25T18:43:56.647597+0000","last_fullsized":"2024-04-25T18:43:56.647597+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:07.143806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632729+0000","last_change":"2024-04-25T18:43:16.247133+0000","last_active":"2024-04-25T18:43:56.632729+0000","last_peered":"2024-04-25T18:43:56.632729+0000","last_clean":"2024-04-25T18:43:56.632729+0000","last_became_active":"2024-04-25T18:43:16.246232+0000","last_became_peered":"2024-04-25T18:43:16.246232+0000","last_unstale":"2024-04-25T18:43:56.632729+0000","last_undegraded":"2024-04-25T18:43:56.632729+0000","last_fullsized":"2024-04-25T18:43:56.632729+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.288404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":63,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:12.325543+0000","last_change":"2024-04-25T18:43:12.209908+0000","last_active":"2024-04-25T18:44:12.325543+0000","last_peered":"2024-04-25T18:44:12.325543+0000","last_clean":"2024-04-25T18:44:12.325543+0000","last_became_active":"2024-04-25T18:43:12.209042+0000","last_became_peered":"2024-04-25T18:43:12.209042+0000","last_unstale":"2024-04-25T18:44:12.325543+0000","last_undegraded":"2024-04-25T18:44:12.325543+0000","last_fullsized":"2024-04-25T18:44:12.325543+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:36.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.643199+0000","last_change":"2024-04-25T18:43:16.242364+0000","last_active":"2024-04-25T18:43:56.643199+0000","last_peered":"2024-04-25T18:43:56.643199+0000","last_clean":"2024-04-25T18:43:56.643199+0000","last_became_active":"2024-04-25T18:43:16.242019+0000","last_became_peered":"2024-04-25T18:43:16.242019+0000","last_unstale":"2024-04-25T18:43:56.643199+0000","last_undegraded":"2024-04-25T18:43:56.643199+0000","last_fullsized":"2024-04-25T18:43:56.643199+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:13.238129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628408+0000","last_change":"2024-04-25T18:43:12.206606+0000","last_active":"2024-04-25T18:43:56.628408+0000","last_peered":"2024-04-25T18:43:56.628408+0000","last_clean":"2024-04-25T18:43:56.628408+0000","last_became_active":"2024-04-25T18:43:12.206238+0000","last_became_peered":"2024-04-25T18:43:12.206238+0000","last_unstale":"2024-04-25T18:43:56.628408+0000","last_undegraded":"2024-04-25T18:43:56.628408+0000","last_fullsized":"2024-04-25T18:43:56.628408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:07.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630224+0000","last_change":"2024-04-25T18:41:40.527363+0000","last_active":"2024-04-25T18:43:56.630224+0000","last_peered":"2024-04-25T18:43:56.630224+0000","last_clean":"2024-04-25T18:43:56.630224+0000","last_became_active":"2024-04-25T18:41:40.526907+0000","last_became_peered":"2024-04-25T18:41:40.526907+0000","last_unstale":"2024-04-25T18:43:56.630224+0000","last_undegraded":"2024-04-25T18:43:56.630224+0000","last_fullsized":"2024-04-25T18:43:56.630224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_clean_scrub_stamp":"2024-04-25T18:37:45.963565+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:16.689693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628418+0000","last_change":"2024-04-25T18:43:16.247552+0000","last_active":"2024-04-25T18:43:56.628418+0000","last_peered":"2024-04-25T18:43:56.628418+0000","last_clean":"2024-04-25T18:43:56.628418+0000","last_became_active":"2024-04-25T18:43:16.247178+0000","last_became_peered":"2024-04-25T18:43:16.247178+0000","last_unstale":"2024-04-25T18:43:56.628418+0000","last_undegraded":"2024-04-25T18:43:56.628418+0000","last_fullsized":"2024-04-25T18:43:56.628418+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:50.072986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.636197+0000","last_change":"2024-04-25T18:43:56.636197+0000","last_active":"2024-04-25T18:43:56.636197+0000","last_peered":"2024-04-25T18:43:56.636197+0000","last_clean":"2024-04-25T18:43:56.636197+0000","last_became_active":"2024-04-25T18:43:56.633431+0000","last_became_peered":"2024-04-25T18:43:56.633431+0000","last_unstale":"2024-04-25T18:43:56.636197+0000","last_undegraded":"2024-04-25T18:43:56.636197+0000","last_fullsized":"2024-04-25T18:43:56.636197+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:12.463489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628476+0000","last_change":"2024-04-25T18:43:12.202101+0000","last_active":"2024-04-25T18:43:56.628476+0000","last_peered":"2024-04-25T18:43:56.628476+0000","last_clean":"2024-04-25T18:43:56.628476+0000","last_became_active":"2024-04-25T18:43:12.201727+0000","last_became_peered":"2024-04-25T18:43:12.201727+0000","last_unstale":"2024-04-25T18:43:56.628476+0000","last_undegraded":"2024-04-25T18:43:56.628476+0000","last_fullsized":"2024-04-25T18:43:56.628476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:33.132072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.629820+0000","last_change":"2024-04-25T18:43:16.255433+0000","last_active":"2024-04-25T18:43:56.629820+0000","last_peered":"2024-04-25T18:43:56.629820+0000","last_clean":"2024-04-25T18:43:56.629820+0000","last_became_active":"2024-04-25T18:43:16.254866+0000","last_became_peered":"2024-04-25T18:43:16.254866+0000","last_unstale":"2024-04-25T18:43:56.629820+0000","last_undegraded":"2024-04-25T18:43:56.629820+0000","last_fullsized":"2024-04-25T18:43:56.629820+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:07.876973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:07.299116+0000","last_change":"2024-04-25T18:43:56.656998+0000","last_active":"2024-04-25T18:44:07.299116+0000","last_peered":"2024-04-25T18:44:07.299116+0000","last_clean":"2024-04-25T18:44:07.299116+0000","last_became_active":"2024-04-25T18:43:56.647456+0000","last_became_peered":"2024-04-25T18:43:56.647456+0000","last_unstale":"2024-04-25T18:44:07.299116+0000","last_undegraded":"2024-04-25T18:44:07.299116+0000","last_fullsized":"2024-04-25T18:44:07.299116+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:00.719426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.638894+0000","last_change":"2024-04-25T18:43:16.252826+0000","last_active":"2024-04-25T18:43:56.638894+0000","last_peered":"2024-04-25T18:43:56.638894+0000","last_clean":"2024-04-25T18:43:56.638894+0000","last_became_active":"2024-04-25T18:43:16.252435+0000","last_became_peered":"2024-04-25T18:43:16.252435+0000","last_unstale":"2024-04-25T18:43:56.638894+0000","last_undegraded":"2024-04-25T18:43:56.638894+0000","last_fullsized":"2024-04-25T18:43:56.638894+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:13.401006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628602+0000","last_change":"2024-04-25T18:43:12.376121+0000","last_active":"2024-04-25T18:43:56.628602+0000","last_peered":"2024-04-25T18:43:56.628602+0000","last_clean":"2024-04-25T18:43:56.628602+0000","last_became_active":"2024-04-25T18:43:12.375757+0000","last_became_peered":"2024-04-25T18:43:12.375757+0000","last_unstale":"2024-04-25T18:43:56.628602+0000","last_undegraded":"2024-04-25T18:43:56.628602+0000","last_fullsized":"2024-04-25T18:43:56.628602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:15.610442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.644030+0000","last_change":"2024-04-25T18:43:16.266504+0000","last_active":"2024-04-25T18:43:56.644030+0000","last_peered":"2024-04-25T18:43:56.644030+0000","last_clean":"2024-04-25T18:43:56.644030+0000","last_became_active":"2024-04-25T18:43:16.265048+0000","last_became_peered":"2024-04-25T18:43:16.265048+0000","last_unstale":"2024-04-25T18:43:56.644030+0000","last_undegraded":"2024-04-25T18:43:56.644030+0000","last_fullsized":"2024-04-25T18:43:56.644030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:36.924872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104519+0000","last_change":"2024-04-25T18:43:12.207529+0000","last_active":"2024-04-25T18:43:57.104519+0000","last_peered":"2024-04-25T18:43:57.104519+0000","last_clean":"2024-04-25T18:43:57.104519+0000","last_became_active":"2024-04-25T18:43:12.207122+0000","last_became_peered":"2024-04-25T18:43:12.207122+0000","last_unstale":"2024-04-25T18:43:57.104519+0000","last_undegraded":"2024-04-25T18:43:57.104519+0000","last_fullsized":"2024-04-25T18:43:57.104519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:44.518497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628095+0000","last_change":"2024-04-25T18:43:16.249616+0000","last_active":"2024-04-25T18:43:56.628095+0000","last_peered":"2024-04-25T18:43:56.628095+0000","last_clean":"2024-04-25T18:43:56.628095+0000","last_became_active":"2024-04-25T18:43:16.249252+0000","last_became_peered":"2024-04-25T18:43:16.249252+0000","last_unstale":"2024-04-25T18:43:56.628095+0000","last_undegraded":"2024-04-25T18:43:56.628095+0000","last_fullsized":"2024-04-25T18:43:56.628095+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:34.721513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.638233+0000","last_change":"2024-04-25T18:43:12.209304+0000","last_active":"2024-04-25T18:43:56.638233+0000","last_peered":"2024-04-25T18:43:56.638233+0000","last_clean":"2024-04-25T18:43:56.638233+0000","last_became_active":"2024-04-25T18:43:12.208887+0000","last_became_peered":"2024-04-25T18:43:12.208887+0000","last_unstale":"2024-04-25T18:43:56.638233+0000","last_undegraded":"2024-04-25T18:43:56.638233+0000","last_fullsized":"2024-04-25T18:43:56.638233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:22.049809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630290+0000","last_change":"2024-04-25T18:43:16.262482+0000","last_active":"2024-04-25T18:43:56.630290+0000","last_peered":"2024-04-25T18:43:56.630290+0000","last_clean":"2024-04-25T18:43:56.630290+0000","last_became_active":"2024-04-25T18:43:16.262108+0000","last_became_peered":"2024-04-25T18:43:16.262108+0000","last_unstale":"2024-04-25T18:43:56.630290+0000","last_undegraded":"2024-04-25T18:43:56.630290+0000","last_fullsized":"2024-04-25T18:43:56.630290+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:14.777939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.625872+0000","last_change":"2024-04-25T18:43:12.205742+0000","last_active":"2024-04-25T18:43:56.625872+0000","last_peered":"2024-04-25T18:43:56.625872+0000","last_clean":"2024-04-25T18:43:56.625872+0000","last_became_active":"2024-04-25T18:43:12.205314+0000","last_became_peered":"2024-04-25T18:43:12.205314+0000","last_unstale":"2024-04-25T18:43:56.625872+0000","last_undegraded":"2024-04-25T18:43:56.625872+0000","last_fullsized":"2024-04-25T18:43:56.625872+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:49.641568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.639020+0000","last_change":"2024-04-25T18:43:16.266100+0000","last_active":"2024-04-25T18:43:56.639020+0000","last_peered":"2024-04-25T18:43:56.639020+0000","last_clean":"2024-04-25T18:43:56.639020+0000","last_became_active":"2024-04-25T18:43:16.263060+0000","last_became_peered":"2024-04-25T18:43:16.263060+0000","last_unstale":"2024-04-25T18:43:56.639020+0000","last_undegraded":"2024-04-25T18:43:56.639020+0000","last_fullsized":"2024-04-25T18:43:56.639020+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:38.269244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628230+0000","last_change":"2024-04-25T18:43:12.196054+0000","last_active":"2024-04-25T18:43:56.628230+0000","last_peered":"2024-04-25T18:43:56.628230+0000","last_clean":"2024-04-25T18:43:56.628230+0000","last_became_active":"2024-04-25T18:43:12.195503+0000","last_became_peered":"2024-04-25T18:43:12.195503+0000","last_unstale":"2024-04-25T18:43:56.628230+0000","last_undegraded":"2024-04-25T18:43:56.628230+0000","last_fullsized":"2024-04-25T18:43:56.628230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:55.187176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630441+0000","last_change":"2024-04-25T18:43:16.242536+0000","last_active":"2024-04-25T18:43:56.630441+0000","last_peered":"2024-04-25T18:43:56.630441+0000","last_clean":"2024-04-25T18:43:56.630441+0000","last_became_active":"2024-04-25T18:43:16.241060+0000","last_became_peered":"2024-04-25T18:43:16.241060+0000","last_unstale":"2024-04-25T18:43:56.630441+0000","last_undegraded":"2024-04-25T18:43:56.630441+0000","last_fullsized":"2024-04-25T18:43:56.630441+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:38.306190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.653946+0000","last_change":"2024-04-25T18:43:56.653377+0000","last_active":"2024-04-25T18:43:56.653946+0000","last_peered":"2024-04-25T18:43:56.653946+0000","last_clean":"2024-04-25T18:43:56.653946+0000","last_became_active":"2024-04-25T18:43:56.645516+0000","last_became_peered":"2024-04-25T18:43:56.645516+0000","last_unstale":"2024-04-25T18:43:56.653946+0000","last_undegraded":"2024-04-25T18:43:56.653946+0000","last_fullsized":"2024-04-25T18:43:56.653946+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:11.189494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630697+0000","last_change":"2024-04-25T18:43:16.247610+0000","last_active":"2024-04-25T18:43:56.630697+0000","last_peered":"2024-04-25T18:43:56.630697+0000","last_clean":"2024-04-25T18:43:56.630697+0000","last_became_active":"2024-04-25T18:43:16.247273+0000","last_became_peered":"2024-04-25T18:43:16.247273+0000","last_unstale":"2024-04-25T18:43:56.630697+0000","last_undegraded":"2024-04-25T18:43:56.630697+0000","last_fullsized":"2024-04-25T18:43:56.630697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:26.151480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632846+0000","last_change":"2024-04-25T18:43:12.202011+0000","last_active":"2024-04-25T18:43:56.632846+0000","last_peered":"2024-04-25T18:43:56.632846+0000","last_clean":"2024-04-25T18:43:56.632846+0000","last_became_active":"2024-04-25T18:43:12.201447+0000","last_became_peered":"2024-04-25T18:43:12.201447+0000","last_unstale":"2024-04-25T18:43:56.632846+0000","last_undegraded":"2024-04-25T18:43:56.632846+0000","last_fullsized":"2024-04-25T18:43:56.632846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:48.094757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'1","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630412+0000","last_change":"2024-04-25T18:43:16.249033+0000","last_active":"2024-04-25T18:43:56.630412+0000","last_peered":"2024-04-25T18:43:56.630412+0000","last_clean":"2024-04-25T18:43:56.630412+0000","last_became_active":"2024-04-25T18:43:16.248691+0000","last_became_peered":"2024-04-25T18:43:16.248691+0000","last_unstale":"2024-04-25T18:43:56.630412+0000","last_undegraded":"2024-04-25T18:43:56.630412+0000","last_fullsized":"2024-04-25T18:43:56.630412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:37.335824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.629888+0000","last_change":"2024-04-25T18:43:12.205134+0000","last_active":"2024-04-25T18:43:56.629888+0000","last_peered":"2024-04-25T18:43:56.629888+0000","last_clean":"2024-04-25T18:43:56.629888+0000","last_became_active":"2024-04-25T18:43:12.204568+0000","last_became_peered":"2024-04-25T18:43:12.204568+0000","last_unstale":"2024-04-25T18:43:56.629888+0000","last_undegraded":"2024-04-25T18:43:56.629888+0000","last_fullsized":"2024-04-25T18:43:56.629888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:32.806580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630493+0000","last_change":"2024-04-25T18:43:16.247337+0000","last_active":"2024-04-25T18:43:56.630493+0000","last_peered":"2024-04-25T18:43:56.630493+0000","last_clean":"2024-04-25T18:43:56.630493+0000","last_became_active":"2024-04-25T18:43:16.246971+0000","last_became_peered":"2024-04-25T18:43:16.246971+0000","last_unstale":"2024-04-25T18:43:56.630493+0000","last_undegraded":"2024-04-25T18:43:56.630493+0000","last_fullsized":"2024-04-25T18:43:56.630493+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.882941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630424+0000","last_change":"2024-04-25T18:43:16.252196+0000","last_active":"2024-04-25T18:43:56.630424+0000","last_peered":"2024-04-25T18:43:56.630424+0000","last_clean":"2024-04-25T18:43:56.630424+0000","last_became_active":"2024-04-25T18:43:16.251715+0000","last_became_peered":"2024-04-25T18:43:16.251715+0000","last_unstale":"2024-04-25T18:43:56.630424+0000","last_undegraded":"2024-04-25T18:43:56.630424+0000","last_fullsized":"2024-04-25T18:43:56.630424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:17.218925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.625876+0000","last_change":"2024-04-25T18:43:16.247632+0000","last_active":"2024-04-25T18:43:56.625876+0000","last_peered":"2024-04-25T18:43:56.625876+0000","last_clean":"2024-04-25T18:43:56.625876+0000","last_became_active":"2024-04-25T18:43:16.247305+0000","last_became_peered":"2024-04-25T18:43:16.247305+0000","last_unstale":"2024-04-25T18:43:56.625876+0000","last_undegraded":"2024-04-25T18:43:56.625876+0000","last_fullsized":"2024-04-25T18:43:56.625876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:52.576146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.625809+0000","last_change":"2024-04-25T18:43:16.266539+0000","last_active":"2024-04-25T18:43:56.625809+0000","last_peered":"2024-04-25T18:43:56.625809+0000","last_clean":"2024-04-25T18:43:56.625809+0000","last_became_active":"2024-04-25T18:43:16.266083+0000","last_became_peered":"2024-04-25T18:43:16.266083+0000","last_unstale":"2024-04-25T18:43:56.625809+0000","last_undegraded":"2024-04-25T18:43:56.625809+0000","last_fullsized":"2024-04-25T18:43:56.625809+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:49.589019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.637737+0000","last_change":"2024-04-25T18:43:16.264002+0000","last_active":"2024-04-25T18:43:56.637737+0000","last_peered":"2024-04-25T18:43:56.637737+0000","last_clean":"2024-04-25T18:43:56.637737+0000","last_became_active":"2024-04-25T18:43:16.261555+0000","last_became_peered":"2024-04-25T18:43:16.261555+0000","last_unstale":"2024-04-25T18:43:56.637737+0000","last_undegraded":"2024-04-25T18:43:56.637737+0000","last_fullsized":"2024-04-25T18:43:56.637737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:12.637040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.629627+0000","last_change":"2024-04-25T18:43:16.270914+0000","last_active":"2024-04-25T18:43:56.629627+0000","last_peered":"2024-04-25T18:43:56.629627+0000","last_clean":"2024-04-25T18:43:56.629627+0000","last_became_active":"2024-04-25T18:43:16.270117+0000","last_became_peered":"2024-04-25T18:43:16.270117+0000","last_unstale":"2024-04-25T18:43:56.629627+0000","last_undegraded":"2024-04-25T18:43:56.629627+0000","last_fullsized":"2024-04-25T18:43:56.629627+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:19.874006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630713+0000","last_change":"2024-04-25T18:43:16.247032+0000","last_active":"2024-04-25T18:43:56.630713+0000","last_peered":"2024-04-25T18:43:56.630713+0000","last_clean":"2024-04-25T18:43:56.630713+0000","last_became_active":"2024-04-25T18:43:16.246734+0000","last_became_peered":"2024-04-25T18:43:16.246734+0000","last_unstale":"2024-04-25T18:43:56.630713+0000","last_undegraded":"2024-04-25T18:43:56.630713+0000","last_fullsized":"2024-04-25T18:43:56.630713+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:39.159143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626107+0000","last_change":"2024-04-25T18:43:16.245263+0000","last_active":"2024-04-25T18:43:56.626107+0000","last_peered":"2024-04-25T18:43:56.626107+0000","last_clean":"2024-04-25T18:43:56.626107+0000","last_became_active":"2024-04-25T18:43:16.244856+0000","last_became_peered":"2024-04-25T18:43:16.244856+0000","last_unstale":"2024-04-25T18:43:56.626107+0000","last_undegraded":"2024-04-25T18:43:56.626107+0000","last_fullsized":"2024-04-25T18:43:56.626107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:36.157927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.638597+0000","last_change":"2024-04-25T18:43:16.253523+0000","last_active":"2024-04-25T18:43:56.638597+0000","last_peered":"2024-04-25T18:43:56.638597+0000","last_clean":"2024-04-25T18:43:56.638597+0000","last_became_active":"2024-04-25T18:43:16.252689+0000","last_became_peered":"2024-04-25T18:43:16.252689+0000","last_unstale":"2024-04-25T18:43:56.638597+0000","last_undegraded":"2024-04-25T18:43:56.638597+0000","last_fullsized":"2024-04-25T18:43:56.638597+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.035788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632583+0000","last_change":"2024-04-25T18:43:16.255818+0000","last_active":"2024-04-25T18:43:56.632583+0000","last_peered":"2024-04-25T18:43:56.632583+0000","last_clean":"2024-04-25T18:43:56.632583+0000","last_became_active":"2024-04-25T18:43:16.255521+0000","last_became_peered":"2024-04-25T18:43:16.255521+0000","last_unstale":"2024-04-25T18:43:56.632583+0000","last_undegraded":"2024-04-25T18:43:56.632583+0000","last_fullsized":"2024-04-25T18:43:56.632583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:15.457585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.637917+0000","last_change":"2024-04-25T18:43:16.259680+0000","last_active":"2024-04-25T18:43:56.637917+0000","last_peered":"2024-04-25T18:43:56.637917+0000","last_clean":"2024-04-25T18:43:56.637917+0000","last_became_active":"2024-04-25T18:43:16.250065+0000","last_became_peered":"2024-04-25T18:43:16.250065+0000","last_unstale":"2024-04-25T18:43:56.637917+0000","last_undegraded":"2024-04-25T18:43:56.637917+0000","last_fullsized":"2024-04-25T18:43:56.637917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:44.974919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104744+0000","last_change":"2024-04-25T18:43:16.239801+0000","last_active":"2024-04-25T18:43:57.104744+0000","last_peered":"2024-04-25T18:43:57.104744+0000","last_clean":"2024-04-25T18:43:57.104744+0000","last_became_active":"2024-04-25T18:43:16.239475+0000","last_became_peered":"2024-04-25T18:43:16.239475+0000","last_unstale":"2024-04-25T18:43:57.104744+0000","last_undegraded":"2024-04-25T18:43:57.104744+0000","last_fullsized":"2024-04-25T18:43:57.104744+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:57.407476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.629851+0000","last_change":"2024-04-25T18:43:16.270419+0000","last_active":"2024-04-25T18:43:56.629851+0000","last_peered":"2024-04-25T18:43:56.629851+0000","last_clean":"2024-04-25T18:43:56.629851+0000","last_became_active":"2024-04-25T18:43:16.269995+0000","last_became_peered":"2024-04-25T18:43:16.269995+0000","last_unstale":"2024-04-25T18:43:56.629851+0000","last_undegraded":"2024-04-25T18:43:56.629851+0000","last_fullsized":"2024-04-25T18:43:56.629851+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:27.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630663+0000","last_change":"2024-04-25T18:43:16.246948+0000","last_active":"2024-04-25T18:43:56.630663+0000","last_peered":"2024-04-25T18:43:56.630663+0000","last_clean":"2024-04-25T18:43:56.630663+0000","last_became_active":"2024-04-25T18:43:16.246616+0000","last_became_peered":"2024-04-25T18:43:16.246616+0000","last_unstale":"2024-04-25T18:43:56.630663+0000","last_undegraded":"2024-04-25T18:43:56.630663+0000","last_fullsized":"2024-04-25T18:43:56.630663+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:51.817448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67073,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":221,"num_read_kb":180,"num_write":165,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":582,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252894,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":498757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514530,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32864,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712288,"statfs":{"total":95995035648,"available":95961382912,"internally_reserved":0,"allocated":1728512,"data_stored":1153310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710758,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":498993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972397,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":498913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234036,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1090843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430264,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":498382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":561247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":499653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":562406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117066,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":498693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":499401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691902,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1090527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:44:19.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:44:20.394 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:20.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:20 smithi171 ceph-mon[33266]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:44:20.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:20 smithi171 ceph-mon[33266]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-25T18:44:20.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:20 smithi161 ceph-mon[32609]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:44:20.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:20 smithi161 ceph-mon[32609]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-25T18:44:20.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:20 smithi112 ceph-mon[27864]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:44:20.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:20 smithi112 ceph-mon[27864]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-25T18:44:22.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:22 smithi112 ceph-mon[27864]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-25T18:44:22.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:22 smithi161 ceph-mon[32609]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-25T18:44:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.651+0000 7f6fbe200700 1 -- 172.21.15.112:0/1163672762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb8109940 msgr2=0x7f6fb8115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.651+0000 7f6fbe200700 1 --2- 172.21.15.112:0/1163672762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb8109940 0x7f6fb8115de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000b0a0 tx=0x7f6fb000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:44:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.651+0000 7f6fbe200700 1 -- 172.21.15.112:0/1163672762 shutdown_connections 2024-04-25T18:44:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.651+0000 7f6fbe200700 1 --2- 172.21.15.112:0/1163672762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb8109940 0x7f6fb8115de0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.651+0000 7f6fbe200700 1 --2- 172.21.15.112:0/1163672762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fb810f030 0x7f6fb810f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.651+0000 7f6fbe200700 1 --2- 172.21.15.112:0/1163672762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fb8108f80 0x7f6fb8109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.651+0000 7f6fbe200700 1 -- 172.21.15.112:0/1163672762 >> 172.21.15.112:0/1163672762 conn(0x7f6fb80787d0 msgr2=0x7f6fb8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.652+0000 7f6fbe200700 1 -- 172.21.15.112:0/1163672762 shutdown_connections 2024-04-25T18:44:22.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.652+0000 7f6fbe200700 1 -- 172.21.15.112:0/1163672762 wait complete. 2024-04-25T18:44:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.653+0000 7f6fbe200700 1 Processor -- start 2024-04-25T18:44:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.653+0000 7f6fbe200700 1 -- start start 2024-04-25T18:44:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.653+0000 7f6fbe200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb8108f80 0x7f6fb819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.653+0000 7f6fbe200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fb8109940 0x7f6fb819fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.653+0000 7f6fbe200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fb810f030 0x7f6fb81a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.653+0000 7f6fbe200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb8119120 con 0x7f6fb8108f80 2024-04-25T18:44:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.653+0000 7f6fbe200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb8118fa0 con 0x7f6fb8109940 2024-04-25T18:44:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.653+0000 7f6fbe200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb81192a0 con 0x7f6fb810f030 2024-04-25T18:44:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.653+0000 7f6fbce00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb8108f80 0x7f6fb819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.654+0000 7f6fbce00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb8108f80 0x7f6fb819f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42296/0 (socket says 172.21.15.112:42296) 2024-04-25T18:44:22.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.654+0000 7f6fbce00700 1 -- 172.21.15.112:0/3183789708 learned_addr learned my addr 172.21.15.112:0/3183789708 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.654+0000 7f6fbd800700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fb810f030 0x7f6fb81a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.654+0000 7f6fb7e00700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fb8109940 0x7f6fb819fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.654+0000 7f6fbce00700 1 -- 172.21.15.112:0/3183789708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fb810f030 msgr2=0x7f6fb81a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.654+0000 7f6fbce00700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fb810f030 0x7f6fb81a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.654+0000 7f6fbce00700 1 -- 172.21.15.112:0/3183789708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fb8109940 msgr2=0x7f6fb819fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.654+0000 7f6fbce00700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fb8109940 0x7f6fb819fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.654+0000 7f6fbce00700 1 -- 172.21.15.112:0/3183789708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb81a4520 con 0x7f6fb8108f80 2024-04-25T18:44:22.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.654+0000 7f6fbce00700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb8108f80 0x7f6fb819f510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e9a0 tx=0x7f6fac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:22.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.655+0000 7f6fb5600700 1 -- 172.21.15.112:0/3183789708 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00ca10 con 0x7f6fb8108f80 2024-04-25T18:44:22.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.655+0000 7f6fb5600700 1 -- 172.21.15.112:0/3183789708 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fac00cbb0 con 0x7f6fb8108f80 2024-04-25T18:44:22.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.655+0000 7f6fbe200700 1 -- 172.21.15.112:0/3183789708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb81a4810 con 0x7f6fb8108f80 2024-04-25T18:44:22.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.655+0000 7f6fb5600700 1 -- 172.21.15.112:0/3183789708 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac018680 con 0x7f6fb8108f80 2024-04-25T18:44:22.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.655+0000 7f6fbe200700 1 -- 172.21.15.112:0/3183789708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb81ac090 con 0x7f6fb8108f80 2024-04-25T18:44:22.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.656+0000 7f6fbe200700 1 -- 172.21.15.112:0/3183789708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb8069730 con 0x7f6fb8108f80 2024-04-25T18:44:22.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.658+0000 7f6fb5600700 1 -- 172.21.15.112:0/3183789708 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6fac010430 con 0x7f6fb8108f80 2024-04-25T18:44:22.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.659+0000 7f6fb5600700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6fa00a3d50 0x7f6fa00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:22.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.659+0000 7f6fb5600700 1 -- 172.21.15.112:0/3183789708 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f6fac0d0740 con 0x7f6fb8108f80 2024-04-25T18:44:22.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.659+0000 7f6fb7e00700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6fa00a3d50 0x7f6fa00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:22.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.660+0000 7f6fb7e00700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6fa00a3d50 0x7f6fa00a6210 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6fb810e7a0 tx=0x7f6fa4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:22.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.664+0000 7f6fb5600700 1 -- 172.21.15.112:0/3183789708 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fac09cf40 con 0x7f6fb8108f80 2024-04-25T18:44:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:22 smithi171 ceph-mon[33266]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-25T18:44:22.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.884+0000 7f6fbe200700 1 -- 172.21.15.112:0/3183789708 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fb81a0870 con 0x7f6fa00a3d50 2024-04-25T18:44:22.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.894+0000 7f6fb5600700 1 -- 172.21.15.112:0/3183789708 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258739 (secure 0 0 0) 0x7f6fb81a0870 con 0x7f6fa00a3d50 2024-04-25T18:44:22.900 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:44:22.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 -- 172.21.15.112:0/3183789708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6fa00a3d50 msgr2=0x7f6fa00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:22.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6fa00a3d50 0x7f6fa00a6210 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6fb810e7a0 tx=0x7f6fa4009260 comp rx=0 tx=0).stop 2024-04-25T18:44:22.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 -- 172.21.15.112:0/3183789708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb8108f80 msgr2=0x7f6fb819f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:22.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb8108f80 0x7f6fb819f510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e9a0 tx=0x7f6fac00ecb0 comp rx=0 tx=0).stop 2024-04-25T18:44:22.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 -- 172.21.15.112:0/3183789708 shutdown_connections 2024-04-25T18:44:22.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6fa00a3d50 0x7f6fa00a6210 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:22.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fb810f030 0x7f6fb81a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:22.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fb8108f80 0x7f6fb819f510 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:22.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 --2- 172.21.15.112:0/3183789708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fb8109940 0x7f6fb819fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:22.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 -- 172.21.15.112:0/3183789708 >> 172.21.15.112:0/3183789708 conn(0x7f6fb80787d0 msgr2=0x7f6fb80799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:22.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 -- 172.21.15.112:0/3183789708 shutdown_connections 2024-04-25T18:44:22.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:22.901+0000 7f6fbe200700 1 -- 172.21.15.112:0/3183789708 wait complete. 2024-04-25T18:44:22.905 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T18:44:23.793 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-04-25T18:44:21.120138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657675,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":387,"num_read_kb":314,"num_write":348,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181318,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375840,"kb_used_data":14184,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565984,"statfs":{"total":1151940427776,"available":1151555567616,"internally_reserved":0,"allocated":14524416,"data_stored":7951125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8705,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":44,"num_write":9,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004978"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630551+0000","last_change":"2024-04-25T18:43:16.262618+0000","last_active":"2024-04-25T18:43:56.630551+0000","last_peered":"2024-04-25T18:43:56.630551+0000","last_clean":"2024-04-25T18:43:56.630551+0000","last_became_active":"2024-04-25T18:43:16.262215+0000","last_became_peered":"2024-04-25T18:43:16.262215+0000","last_unstale":"2024-04-25T18:43:56.630551+0000","last_undegraded":"2024-04-25T18:43:56.630551+0000","last_fullsized":"2024-04-25T18:43:56.630551+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:40.319102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.636714+0000","last_change":"2024-04-25T18:43:16.254085+0000","last_active":"2024-04-25T18:43:56.636714+0000","last_peered":"2024-04-25T18:43:56.636714+0000","last_clean":"2024-04-25T18:43:56.636714+0000","last_became_active":"2024-04-25T18:43:16.248943+0000","last_became_peered":"2024-04-25T18:43:16.248943+0000","last_unstale":"2024-04-25T18:43:56.636714+0000","last_undegraded":"2024-04-25T18:43:56.636714+0000","last_fullsized":"2024-04-25T18:43:56.636714+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:53.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.638807+0000","last_change":"2024-04-25T18:43:16.252684+0000","last_active":"2024-04-25T18:43:56.638807+0000","last_peered":"2024-04-25T18:43:56.638807+0000","last_clean":"2024-04-25T18:43:56.638807+0000","last_became_active":"2024-04-25T18:43:16.252318+0000","last_became_peered":"2024-04-25T18:43:16.252318+0000","last_unstale":"2024-04-25T18:43:56.638807+0000","last_undegraded":"2024-04-25T18:43:56.638807+0000","last_fullsized":"2024-04-25T18:43:56.638807+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:58.904865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628754+0000","last_change":"2024-04-25T18:43:16.247059+0000","last_active":"2024-04-25T18:43:56.628754+0000","last_peered":"2024-04-25T18:43:56.628754+0000","last_clean":"2024-04-25T18:43:56.628754+0000","last_became_active":"2024-04-25T18:43:16.246655+0000","last_became_peered":"2024-04-25T18:43:16.246655+0000","last_unstale":"2024-04-25T18:43:56.628754+0000","last_undegraded":"2024-04-25T18:43:56.628754+0000","last_fullsized":"2024-04-25T18:43:56.628754+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:31.026579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628239+0000","last_change":"2024-04-25T18:43:16.250033+0000","last_active":"2024-04-25T18:43:56.628239+0000","last_peered":"2024-04-25T18:43:56.628239+0000","last_clean":"2024-04-25T18:43:56.628239+0000","last_became_active":"2024-04-25T18:43:16.249017+0000","last_became_peered":"2024-04-25T18:43:16.249017+0000","last_unstale":"2024-04-25T18:43:56.628239+0000","last_undegraded":"2024-04-25T18:43:56.628239+0000","last_fullsized":"2024-04-25T18:43:56.628239+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:34.304263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.642410+0000","last_change":"2024-04-25T18:43:56.642410+0000","last_active":"2024-04-25T18:43:56.642410+0000","last_peered":"2024-04-25T18:43:56.642410+0000","last_clean":"2024-04-25T18:43:56.642410+0000","last_became_active":"2024-04-25T18:43:56.642009+0000","last_became_peered":"2024-04-25T18:43:56.642009+0000","last_unstale":"2024-04-25T18:43:56.642410+0000","last_undegraded":"2024-04-25T18:43:56.642410+0000","last_fullsized":"2024-04-25T18:43:56.642410+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:19.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.647158+0000","last_change":"2024-04-25T18:43:56.647158+0000","last_active":"2024-04-25T18:43:56.647158+0000","last_peered":"2024-04-25T18:43:56.647158+0000","last_clean":"2024-04-25T18:43:56.647158+0000","last_became_active":"2024-04-25T18:43:56.646779+0000","last_became_peered":"2024-04-25T18:43:56.646779+0000","last_unstale":"2024-04-25T18:43:56.647158+0000","last_undegraded":"2024-04-25T18:43:56.647158+0000","last_fullsized":"2024-04-25T18:43:56.647158+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:04:14.055762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.637895+0000","last_change":"2024-04-25T18:43:16.247607+0000","last_active":"2024-04-25T18:43:56.637895+0000","last_peered":"2024-04-25T18:43:56.637895+0000","last_clean":"2024-04-25T18:43:56.637895+0000","last_became_active":"2024-04-25T18:43:16.247234+0000","last_became_peered":"2024-04-25T18:43:16.247234+0000","last_unstale":"2024-04-25T18:43:56.637895+0000","last_undegraded":"2024-04-25T18:43:56.637895+0000","last_fullsized":"2024-04-25T18:43:56.637895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:57.117900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.105056+0000","last_change":"2024-04-25T18:43:16.247909+0000","last_active":"2024-04-25T18:43:57.105056+0000","last_peered":"2024-04-25T18:43:57.105056+0000","last_clean":"2024-04-25T18:43:57.105056+0000","last_became_active":"2024-04-25T18:43:16.247533+0000","last_became_peered":"2024-04-25T18:43:16.247533+0000","last_unstale":"2024-04-25T18:43:57.105056+0000","last_undegraded":"2024-04-25T18:43:57.105056+0000","last_fullsized":"2024-04-25T18:43:57.105056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:02.257832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626680+0000","last_change":"2024-04-25T18:43:16.252088+0000","last_active":"2024-04-25T18:43:56.626680+0000","last_peered":"2024-04-25T18:43:56.626680+0000","last_clean":"2024-04-25T18:43:56.626680+0000","last_became_active":"2024-04-25T18:43:16.251580+0000","last_became_peered":"2024-04-25T18:43:16.251580+0000","last_unstale":"2024-04-25T18:43:56.626680+0000","last_undegraded":"2024-04-25T18:43:56.626680+0000","last_fullsized":"2024-04-25T18:43:56.626680+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:22.992087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630307+0000","last_change":"2024-04-25T18:43:16.239696+0000","last_active":"2024-04-25T18:43:56.630307+0000","last_peered":"2024-04-25T18:43:56.630307+0000","last_clean":"2024-04-25T18:43:56.630307+0000","last_became_active":"2024-04-25T18:43:16.239354+0000","last_became_peered":"2024-04-25T18:43:16.239354+0000","last_unstale":"2024-04-25T18:43:56.630307+0000","last_undegraded":"2024-04-25T18:43:56.630307+0000","last_fullsized":"2024-04-25T18:43:56.630307+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:39.682138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628528+0000","last_change":"2024-04-25T18:43:16.242487+0000","last_active":"2024-04-25T18:43:56.628528+0000","last_peered":"2024-04-25T18:43:56.628528+0000","last_clean":"2024-04-25T18:43:56.628528+0000","last_became_active":"2024-04-25T18:43:16.242158+0000","last_became_peered":"2024-04-25T18:43:16.242158+0000","last_unstale":"2024-04-25T18:43:56.628528+0000","last_undegraded":"2024-04-25T18:43:56.628528+0000","last_fullsized":"2024-04-25T18:43:56.628528+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:36.477440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630372+0000","last_change":"2024-04-25T18:43:16.244154+0000","last_active":"2024-04-25T18:43:56.630372+0000","last_peered":"2024-04-25T18:43:56.630372+0000","last_clean":"2024-04-25T18:43:56.630372+0000","last_became_active":"2024-04-25T18:43:16.243829+0000","last_became_peered":"2024-04-25T18:43:16.243829+0000","last_unstale":"2024-04-25T18:43:56.630372+0000","last_undegraded":"2024-04-25T18:43:56.630372+0000","last_fullsized":"2024-04-25T18:43:56.630372+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:43.262751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.636692+0000","last_change":"2024-04-25T18:43:16.261227+0000","last_active":"2024-04-25T18:43:56.636692+0000","last_peered":"2024-04-25T18:43:56.636692+0000","last_clean":"2024-04-25T18:43:56.636692+0000","last_became_active":"2024-04-25T18:43:16.250468+0000","last_became_peered":"2024-04-25T18:43:16.250468+0000","last_unstale":"2024-04-25T18:43:56.636692+0000","last_undegraded":"2024-04-25T18:43:56.636692+0000","last_fullsized":"2024-04-25T18:43:56.636692+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:44.805643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.636244+0000","last_change":"2024-04-25T18:43:16.257956+0000","last_active":"2024-04-25T18:43:56.636244+0000","last_peered":"2024-04-25T18:43:56.636244+0000","last_clean":"2024-04-25T18:43:56.636244+0000","last_became_active":"2024-04-25T18:43:16.255126+0000","last_became_peered":"2024-04-25T18:43:16.255126+0000","last_unstale":"2024-04-25T18:43:56.636244+0000","last_undegraded":"2024-04-25T18:43:56.636244+0000","last_fullsized":"2024-04-25T18:43:56.636244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:30.047165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104512+0000","last_change":"2024-04-25T18:43:16.256208+0000","last_active":"2024-04-25T18:43:57.104512+0000","last_peered":"2024-04-25T18:43:57.104512+0000","last_clean":"2024-04-25T18:43:57.104512+0000","last_became_active":"2024-04-25T18:43:16.255614+0000","last_became_peered":"2024-04-25T18:43:16.255614+0000","last_unstale":"2024-04-25T18:43:57.104512+0000","last_undegraded":"2024-04-25T18:43:57.104512+0000","last_fullsized":"2024-04-25T18:43:57.104512+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:46.700209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630798+0000","last_change":"2024-04-25T18:43:16.262730+0000","last_active":"2024-04-25T18:43:56.630798+0000","last_peered":"2024-04-25T18:43:56.630798+0000","last_clean":"2024-04-25T18:43:56.630798+0000","last_became_active":"2024-04-25T18:43:16.262192+0000","last_became_peered":"2024-04-25T18:43:16.262192+0000","last_unstale":"2024-04-25T18:43:56.630798+0000","last_undegraded":"2024-04-25T18:43:56.630798+0000","last_fullsized":"2024-04-25T18:43:56.630798+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:29.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.643024+0000","last_change":"2024-04-25T18:43:16.261161+0000","last_active":"2024-04-25T18:43:56.643024+0000","last_peered":"2024-04-25T18:43:56.643024+0000","last_clean":"2024-04-25T18:43:56.643024+0000","last_became_active":"2024-04-25T18:43:16.258217+0000","last_became_peered":"2024-04-25T18:43:16.258217+0000","last_unstale":"2024-04-25T18:43:56.643024+0000","last_undegraded":"2024-04-25T18:43:56.643024+0000","last_fullsized":"2024-04-25T18:43:56.643024+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.564609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626029+0000","last_change":"2024-04-25T18:43:16.266660+0000","last_active":"2024-04-25T18:43:56.626029+0000","last_peered":"2024-04-25T18:43:56.626029+0000","last_clean":"2024-04-25T18:43:56.626029+0000","last_became_active":"2024-04-25T18:43:16.266295+0000","last_became_peered":"2024-04-25T18:43:16.266295+0000","last_unstale":"2024-04-25T18:43:56.626029+0000","last_undegraded":"2024-04-25T18:43:56.626029+0000","last_fullsized":"2024-04-25T18:43:56.626029+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:42.406585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.643296+0000","last_change":"2024-04-25T18:43:12.200172+0000","last_active":"2024-04-25T18:43:56.643296+0000","last_peered":"2024-04-25T18:43:56.643296+0000","last_clean":"2024-04-25T18:43:56.643296+0000","last_became_active":"2024-04-25T18:43:12.199760+0000","last_became_peered":"2024-04-25T18:43:12.199760+0000","last_unstale":"2024-04-25T18:43:56.643296+0000","last_undegraded":"2024-04-25T18:43:56.643296+0000","last_fullsized":"2024-04-25T18:43:56.643296+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:57.460274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.638451+0000","last_change":"2024-04-25T18:43:16.255989+0000","last_active":"2024-04-25T18:43:56.638451+0000","last_peered":"2024-04-25T18:43:56.638451+0000","last_clean":"2024-04-25T18:43:56.638451+0000","last_became_active":"2024-04-25T18:43:16.255114+0000","last_became_peered":"2024-04-25T18:43:16.255114+0000","last_unstale":"2024-04-25T18:43:56.638451+0000","last_undegraded":"2024-04-25T18:43:56.638451+0000","last_fullsized":"2024-04-25T18:43:56.638451+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:52.888876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.639575+0000","last_change":"2024-04-25T18:43:12.208194+0000","last_active":"2024-04-25T18:43:56.639575+0000","last_peered":"2024-04-25T18:43:56.639575+0000","last_clean":"2024-04-25T18:43:56.639575+0000","last_became_active":"2024-04-25T18:43:12.207801+0000","last_became_peered":"2024-04-25T18:43:12.207801+0000","last_unstale":"2024-04-25T18:43:56.639575+0000","last_undegraded":"2024-04-25T18:43:56.639575+0000","last_fullsized":"2024-04-25T18:43:56.639575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:24.742486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.644607+0000","last_change":"2024-04-25T18:43:16.268960+0000","last_active":"2024-04-25T18:43:56.644607+0000","last_peered":"2024-04-25T18:43:56.644607+0000","last_clean":"2024-04-25T18:43:56.644607+0000","last_became_active":"2024-04-25T18:43:16.263626+0000","last_became_peered":"2024-04-25T18:43:16.263626+0000","last_unstale":"2024-04-25T18:43:56.644607+0000","last_undegraded":"2024-04-25T18:43:56.644607+0000","last_fullsized":"2024-04-25T18:43:56.644607+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:57.721918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.110208+0000","last_change":"2024-04-25T18:43:57.110208+0000","last_active":"2024-04-25T18:43:57.110208+0000","last_peered":"2024-04-25T18:43:57.110208+0000","last_clean":"2024-04-25T18:43:57.110208+0000","last_became_active":"2024-04-25T18:43:57.106027+0000","last_became_peered":"2024-04-25T18:43:57.106027+0000","last_unstale":"2024-04-25T18:43:57.110208+0000","last_undegraded":"2024-04-25T18:43:57.110208+0000","last_fullsized":"2024-04-25T18:43:57.110208+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:20.819961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104747+0000","last_change":"2024-04-25T18:43:16.250414+0000","last_active":"2024-04-25T18:43:57.104747+0000","last_peered":"2024-04-25T18:43:57.104747+0000","last_clean":"2024-04-25T18:43:57.104747+0000","last_became_active":"2024-04-25T18:43:16.249832+0000","last_became_peered":"2024-04-25T18:43:16.249832+0000","last_unstale":"2024-04-25T18:43:57.104747+0000","last_undegraded":"2024-04-25T18:43:57.104747+0000","last_fullsized":"2024-04-25T18:43:57.104747+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.577854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628806+0000","last_change":"2024-04-25T18:43:12.195099+0000","last_active":"2024-04-25T18:43:56.628806+0000","last_peered":"2024-04-25T18:43:56.628806+0000","last_clean":"2024-04-25T18:43:56.628806+0000","last_became_active":"2024-04-25T18:43:12.194726+0000","last_became_peered":"2024-04-25T18:43:12.194726+0000","last_unstale":"2024-04-25T18:43:56.628806+0000","last_undegraded":"2024-04-25T18:43:56.628806+0000","last_fullsized":"2024-04-25T18:43:56.628806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.866905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632966+0000","last_change":"2024-04-25T18:43:16.245208+0000","last_active":"2024-04-25T18:43:56.632966+0000","last_peered":"2024-04-25T18:43:56.632966+0000","last_clean":"2024-04-25T18:43:56.632966+0000","last_became_active":"2024-04-25T18:43:16.244836+0000","last_became_peered":"2024-04-25T18:43:16.244836+0000","last_unstale":"2024-04-25T18:43:56.632966+0000","last_undegraded":"2024-04-25T18:43:56.632966+0000","last_fullsized":"2024-04-25T18:43:56.632966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:35.917060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":72,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:16.321145+0000","last_change":"2024-04-25T18:43:12.198252+0000","last_active":"2024-04-25T18:44:16.321145+0000","last_peered":"2024-04-25T18:44:16.321145+0000","last_clean":"2024-04-25T18:44:16.321145+0000","last_became_active":"2024-04-25T18:43:12.197691+0000","last_became_peered":"2024-04-25T18:43:12.197691+0000","last_unstale":"2024-04-25T18:44:16.321145+0000","last_undegraded":"2024-04-25T18:44:16.321145+0000","last_fullsized":"2024-04-25T18:44:16.321145+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.900012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628228+0000","last_change":"2024-04-25T18:43:16.255576+0000","last_active":"2024-04-25T18:43:56.628228+0000","last_peered":"2024-04-25T18:43:56.628228+0000","last_clean":"2024-04-25T18:43:56.628228+0000","last_became_active":"2024-04-25T18:43:16.255195+0000","last_became_peered":"2024-04-25T18:43:16.255195+0000","last_unstale":"2024-04-25T18:43:56.628228+0000","last_undegraded":"2024-04-25T18:43:56.628228+0000","last_fullsized":"2024-04-25T18:43:56.628228+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:41.510872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.625977+0000","last_change":"2024-04-25T18:43:12.200786+0000","last_active":"2024-04-25T18:43:56.625977+0000","last_peered":"2024-04-25T18:43:56.625977+0000","last_clean":"2024-04-25T18:43:56.625977+0000","last_became_active":"2024-04-25T18:43:12.200409+0000","last_became_peered":"2024-04-25T18:43:12.200409+0000","last_unstale":"2024-04-25T18:43:56.625977+0000","last_undegraded":"2024-04-25T18:43:56.625977+0000","last_fullsized":"2024-04-25T18:43:56.625977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:52.177787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632730+0000","last_change":"2024-04-25T18:43:16.255563+0000","last_active":"2024-04-25T18:43:56.632730+0000","last_peered":"2024-04-25T18:43:56.632730+0000","last_clean":"2024-04-25T18:43:56.632730+0000","last_became_active":"2024-04-25T18:43:16.255174+0000","last_became_peered":"2024-04-25T18:43:16.255174+0000","last_unstale":"2024-04-25T18:43:56.632730+0000","last_undegraded":"2024-04-25T18:43:56.632730+0000","last_fullsized":"2024-04-25T18:43:56.632730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:32:15.735051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628776+0000","last_change":"2024-04-25T18:43:12.210242+0000","last_active":"2024-04-25T18:43:56.628776+0000","last_peered":"2024-04-25T18:43:56.628776+0000","last_clean":"2024-04-25T18:43:56.628776+0000","last_became_active":"2024-04-25T18:43:12.209600+0000","last_became_peered":"2024-04-25T18:43:12.209600+0000","last_unstale":"2024-04-25T18:43:56.628776+0000","last_undegraded":"2024-04-25T18:43:56.628776+0000","last_fullsized":"2024-04-25T18:43:56.628776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.886712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.652388+0000","last_change":"2024-04-25T18:43:56.649102+0000","last_active":"2024-04-25T18:43:56.652388+0000","last_peered":"2024-04-25T18:43:56.652388+0000","last_clean":"2024-04-25T18:43:56.652388+0000","last_became_active":"2024-04-25T18:43:56.647127+0000","last_became_peered":"2024-04-25T18:43:56.647127+0000","last_unstale":"2024-04-25T18:43:56.652388+0000","last_undegraded":"2024-04-25T18:43:56.652388+0000","last_fullsized":"2024-04-25T18:43:56.652388+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:43.111539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632516+0000","last_change":"2024-04-25T18:43:12.198957+0000","last_active":"2024-04-25T18:43:56.632516+0000","last_peered":"2024-04-25T18:43:56.632516+0000","last_clean":"2024-04-25T18:43:56.632516+0000","last_became_active":"2024-04-25T18:43:12.198605+0000","last_became_peered":"2024-04-25T18:43:12.198605+0000","last_unstale":"2024-04-25T18:43:56.632516+0000","last_undegraded":"2024-04-25T18:43:56.632516+0000","last_fullsized":"2024-04-25T18:43:56.632516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:11.759430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.642145+0000","last_change":"2024-04-25T18:43:16.266829+0000","last_active":"2024-04-25T18:43:56.642145+0000","last_peered":"2024-04-25T18:43:56.642145+0000","last_clean":"2024-04-25T18:43:56.642145+0000","last_became_active":"2024-04-25T18:43:16.266256+0000","last_became_peered":"2024-04-25T18:43:16.266256+0000","last_unstale":"2024-04-25T18:43:56.642145+0000","last_undegraded":"2024-04-25T18:43:56.642145+0000","last_fullsized":"2024-04-25T18:43:56.642145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:43.343613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":66,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:16.320580+0000","last_change":"2024-04-25T18:43:12.201641+0000","last_active":"2024-04-25T18:44:16.320580+0000","last_peered":"2024-04-25T18:44:16.320580+0000","last_clean":"2024-04-25T18:44:16.320580+0000","last_became_active":"2024-04-25T18:43:12.201274+0000","last_became_peered":"2024-04-25T18:43:12.201274+0000","last_unstale":"2024-04-25T18:44:16.320580+0000","last_undegraded":"2024-04-25T18:44:16.320580+0000","last_fullsized":"2024-04-25T18:44:16.320580+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:47.683850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.105043+0000","last_change":"2024-04-25T18:43:16.259551+0000","last_active":"2024-04-25T18:43:57.105043+0000","last_peered":"2024-04-25T18:43:57.105043+0000","last_clean":"2024-04-25T18:43:57.105043+0000","last_became_active":"2024-04-25T18:43:16.259165+0000","last_became_peered":"2024-04-25T18:43:16.259165+0000","last_unstale":"2024-04-25T18:43:57.105043+0000","last_undegraded":"2024-04-25T18:43:57.105043+0000","last_fullsized":"2024-04-25T18:43:57.105043+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:23.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630280+0000","last_change":"2024-04-25T18:43:12.197534+0000","last_active":"2024-04-25T18:43:56.630280+0000","last_peered":"2024-04-25T18:43:56.630280+0000","last_clean":"2024-04-25T18:43:56.630280+0000","last_became_active":"2024-04-25T18:43:12.197106+0000","last_became_peered":"2024-04-25T18:43:12.197106+0000","last_unstale":"2024-04-25T18:43:56.630280+0000","last_undegraded":"2024-04-25T18:43:56.630280+0000","last_fullsized":"2024-04-25T18:43:56.630280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:35.399044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.644573+0000","last_change":"2024-04-25T18:43:16.257668+0000","last_active":"2024-04-25T18:43:56.644573+0000","last_peered":"2024-04-25T18:43:56.644573+0000","last_clean":"2024-04-25T18:43:56.644573+0000","last_became_active":"2024-04-25T18:43:16.253609+0000","last_became_peered":"2024-04-25T18:43:16.253609+0000","last_unstale":"2024-04-25T18:43:56.644573+0000","last_undegraded":"2024-04-25T18:43:56.644573+0000","last_fullsized":"2024-04-25T18:43:56.644573+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:36.048386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.633047+0000","last_change":"2024-04-25T18:43:12.207322+0000","last_active":"2024-04-25T18:43:56.633047+0000","last_peered":"2024-04-25T18:43:56.633047+0000","last_clean":"2024-04-25T18:43:56.633047+0000","last_became_active":"2024-04-25T18:43:12.206947+0000","last_became_peered":"2024-04-25T18:43:12.206947+0000","last_unstale":"2024-04-25T18:43:56.633047+0000","last_undegraded":"2024-04-25T18:43:56.633047+0000","last_fullsized":"2024-04-25T18:43:56.633047+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:20.536598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104798+0000","last_change":"2024-04-25T18:43:16.239575+0000","last_active":"2024-04-25T18:43:57.104798+0000","last_peered":"2024-04-25T18:43:57.104798+0000","last_clean":"2024-04-25T18:43:57.104798+0000","last_became_active":"2024-04-25T18:43:16.239208+0000","last_became_peered":"2024-04-25T18:43:16.239208+0000","last_unstale":"2024-04-25T18:43:57.104798+0000","last_undegraded":"2024-04-25T18:43:57.104798+0000","last_fullsized":"2024-04-25T18:43:57.104798+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:58.195855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'30","reported_seq":138,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:17.351867+0000","last_change":"2024-04-25T18:43:12.197199+0000","last_active":"2024-04-25T18:44:17.351867+0000","last_peered":"2024-04-25T18:44:17.351867+0000","last_clean":"2024-04-25T18:44:17.351867+0000","last_became_active":"2024-04-25T18:43:12.196776+0000","last_became_peered":"2024-04-25T18:43:12.196776+0000","last_unstale":"2024-04-25T18:44:17.351867+0000","last_undegraded":"2024-04-25T18:44:17.351867+0000","last_fullsized":"2024-04-25T18:44:17.351867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:54.149782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":102,"num_write":31,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628449+0000","last_change":"2024-04-25T18:43:16.250993+0000","last_active":"2024-04-25T18:43:56.628449+0000","last_peered":"2024-04-25T18:43:56.628449+0000","last_clean":"2024-04-25T18:43:56.628449+0000","last_became_active":"2024-04-25T18:43:16.250216+0000","last_became_peered":"2024-04-25T18:43:16.250216+0000","last_unstale":"2024-04-25T18:43:56.628449+0000","last_undegraded":"2024-04-25T18:43:56.628449+0000","last_fullsized":"2024-04-25T18:43:56.628449+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:37.996463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626135+0000","last_change":"2024-04-25T18:43:12.204606+0000","last_active":"2024-04-25T18:43:56.626135+0000","last_peered":"2024-04-25T18:43:56.626135+0000","last_clean":"2024-04-25T18:43:56.626135+0000","last_became_active":"2024-04-25T18:43:12.204306+0000","last_became_peered":"2024-04-25T18:43:12.204306+0000","last_unstale":"2024-04-25T18:43:56.626135+0000","last_undegraded":"2024-04-25T18:43:56.626135+0000","last_fullsized":"2024-04-25T18:43:56.626135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:29.903775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628584+0000","last_change":"2024-04-25T18:43:16.250093+0000","last_active":"2024-04-25T18:43:56.628584+0000","last_peered":"2024-04-25T18:43:56.628584+0000","last_clean":"2024-04-25T18:43:56.628584+0000","last_became_active":"2024-04-25T18:43:16.249567+0000","last_became_peered":"2024-04-25T18:43:16.249567+0000","last_unstale":"2024-04-25T18:43:56.628584+0000","last_undegraded":"2024-04-25T18:43:56.628584+0000","last_fullsized":"2024-04-25T18:43:56.628584+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:35.498641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.652890+0000","last_change":"2024-04-25T18:43:56.652488+0000","last_active":"2024-04-25T18:43:56.652890+0000","last_peered":"2024-04-25T18:43:56.652890+0000","last_clean":"2024-04-25T18:43:56.652890+0000","last_became_active":"2024-04-25T18:43:56.647745+0000","last_became_peered":"2024-04-25T18:43:56.647745+0000","last_unstale":"2024-04-25T18:43:56.652890+0000","last_undegraded":"2024-04-25T18:43:56.652890+0000","last_fullsized":"2024-04-25T18:43:56.652890+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:33.474470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.642673+0000","last_change":"2024-04-25T18:43:56.642673+0000","last_active":"2024-04-25T18:43:56.642673+0000","last_peered":"2024-04-25T18:43:56.642673+0000","last_clean":"2024-04-25T18:43:56.642673+0000","last_became_active":"2024-04-25T18:43:56.642313+0000","last_became_peered":"2024-04-25T18:43:56.642313+0000","last_unstale":"2024-04-25T18:43:56.642673+0000","last_undegraded":"2024-04-25T18:43:56.642673+0000","last_fullsized":"2024-04-25T18:43:56.642673+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:45.805738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":70,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:18.337523+0000","last_change":"2024-04-25T18:43:12.197341+0000","last_active":"2024-04-25T18:44:18.337523+0000","last_peered":"2024-04-25T18:44:18.337523+0000","last_clean":"2024-04-25T18:44:18.337523+0000","last_became_active":"2024-04-25T18:43:12.196952+0000","last_became_peered":"2024-04-25T18:43:12.196952+0000","last_unstale":"2024-04-25T18:44:18.337523+0000","last_undegraded":"2024-04-25T18:44:18.337523+0000","last_fullsized":"2024-04-25T18:44:18.337523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:52.902579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.629893+0000","last_change":"2024-04-25T18:43:16.245436+0000","last_active":"2024-04-25T18:43:56.629893+0000","last_peered":"2024-04-25T18:43:56.629893+0000","last_clean":"2024-04-25T18:43:56.629893+0000","last_became_active":"2024-04-25T18:43:16.244943+0000","last_became_peered":"2024-04-25T18:43:16.244943+0000","last_unstale":"2024-04-25T18:43:56.629893+0000","last_undegraded":"2024-04-25T18:43:56.629893+0000","last_fullsized":"2024-04-25T18:43:56.629893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:03.402999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628089+0000","last_change":"2024-04-25T18:43:12.197642+0000","last_active":"2024-04-25T18:43:56.628089+0000","last_peered":"2024-04-25T18:43:56.628089+0000","last_clean":"2024-04-25T18:43:56.628089+0000","last_became_active":"2024-04-25T18:43:12.196395+0000","last_became_peered":"2024-04-25T18:43:12.196395+0000","last_unstale":"2024-04-25T18:43:56.628089+0000","last_undegraded":"2024-04-25T18:43:56.628089+0000","last_fullsized":"2024-04-25T18:43:56.628089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:11.783422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632381+0000","last_change":"2024-04-25T18:43:16.245344+0000","last_active":"2024-04-25T18:43:56.632381+0000","last_peered":"2024-04-25T18:43:56.632381+0000","last_clean":"2024-04-25T18:43:56.632381+0000","last_became_active":"2024-04-25T18:43:16.244984+0000","last_became_peered":"2024-04-25T18:43:16.244984+0000","last_unstale":"2024-04-25T18:43:56.632381+0000","last_undegraded":"2024-04-25T18:43:56.632381+0000","last_fullsized":"2024-04-25T18:43:56.632381+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:04.822629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628582+0000","last_change":"2024-04-25T18:43:12.199685+0000","last_active":"2024-04-25T18:43:56.628582+0000","last_peered":"2024-04-25T18:43:56.628582+0000","last_clean":"2024-04-25T18:43:56.628582+0000","last_became_active":"2024-04-25T18:43:12.199261+0000","last_became_peered":"2024-04-25T18:43:12.199261+0000","last_unstale":"2024-04-25T18:43:56.628582+0000","last_undegraded":"2024-04-25T18:43:56.628582+0000","last_fullsized":"2024-04-25T18:43:56.628582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:09.784616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630711+0000","last_change":"2024-04-25T18:43:16.247745+0000","last_active":"2024-04-25T18:43:56.630711+0000","last_peered":"2024-04-25T18:43:56.630711+0000","last_clean":"2024-04-25T18:43:56.630711+0000","last_became_active":"2024-04-25T18:43:16.247311+0000","last_became_peered":"2024-04-25T18:43:16.247311+0000","last_unstale":"2024-04-25T18:43:56.630711+0000","last_undegraded":"2024-04-25T18:43:56.630711+0000","last_fullsized":"2024-04-25T18:43:56.630711+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:29.925464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.642616+0000","last_change":"2024-04-25T18:43:12.210209+0000","last_active":"2024-04-25T18:43:56.642616+0000","last_peered":"2024-04-25T18:43:56.642616+0000","last_clean":"2024-04-25T18:43:56.642616+0000","last_became_active":"2024-04-25T18:43:12.209708+0000","last_became_peered":"2024-04-25T18:43:12.209708+0000","last_unstale":"2024-04-25T18:43:56.642616+0000","last_undegraded":"2024-04-25T18:43:56.642616+0000","last_fullsized":"2024-04-25T18:43:56.642616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:07:18.331518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'1","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626295+0000","last_change":"2024-04-25T18:43:16.263616+0000","last_active":"2024-04-25T18:43:56.626295+0000","last_peered":"2024-04-25T18:43:56.626295+0000","last_clean":"2024-04-25T18:43:56.626295+0000","last_became_active":"2024-04-25T18:43:16.263287+0000","last_became_peered":"2024-04-25T18:43:16.263287+0000","last_unstale":"2024-04-25T18:43:56.626295+0000","last_undegraded":"2024-04-25T18:43:56.626295+0000","last_fullsized":"2024-04-25T18:43:56.626295+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:46.366457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630314+0000","last_change":"2024-04-25T18:43:12.203705+0000","last_active":"2024-04-25T18:43:56.630314+0000","last_peered":"2024-04-25T18:43:56.630314+0000","last_clean":"2024-04-25T18:43:56.630314+0000","last_became_active":"2024-04-25T18:43:12.203371+0000","last_became_peered":"2024-04-25T18:43:12.203371+0000","last_unstale":"2024-04-25T18:43:56.630314+0000","last_undegraded":"2024-04-25T18:43:56.630314+0000","last_fullsized":"2024-04-25T18:43:56.630314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.647597+0000","last_change":"2024-04-25T18:43:56.647597+0000","last_active":"2024-04-25T18:43:56.647597+0000","last_peered":"2024-04-25T18:43:56.647597+0000","last_clean":"2024-04-25T18:43:56.647597+0000","last_became_active":"2024-04-25T18:43:56.647121+0000","last_became_peered":"2024-04-25T18:43:56.647121+0000","last_unstale":"2024-04-25T18:43:56.647597+0000","last_undegraded":"2024-04-25T18:43:56.647597+0000","last_fullsized":"2024-04-25T18:43:56.647597+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:07.143806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632729+0000","last_change":"2024-04-25T18:43:16.247133+0000","last_active":"2024-04-25T18:43:56.632729+0000","last_peered":"2024-04-25T18:43:56.632729+0000","last_clean":"2024-04-25T18:43:56.632729+0000","last_became_active":"2024-04-25T18:43:16.246232+0000","last_became_peered":"2024-04-25T18:43:16.246232+0000","last_unstale":"2024-04-25T18:43:56.632729+0000","last_undegraded":"2024-04-25T18:43:56.632729+0000","last_fullsized":"2024-04-25T18:43:56.632729+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.288404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":68,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:17.334542+0000","last_change":"2024-04-25T18:43:12.209908+0000","last_active":"2024-04-25T18:44:17.334542+0000","last_peered":"2024-04-25T18:44:17.334542+0000","last_clean":"2024-04-25T18:44:17.334542+0000","last_became_active":"2024-04-25T18:43:12.209042+0000","last_became_peered":"2024-04-25T18:43:12.209042+0000","last_unstale":"2024-04-25T18:44:17.334542+0000","last_undegraded":"2024-04-25T18:44:17.334542+0000","last_fullsized":"2024-04-25T18:44:17.334542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:36.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.643199+0000","last_change":"2024-04-25T18:43:16.242364+0000","last_active":"2024-04-25T18:43:56.643199+0000","last_peered":"2024-04-25T18:43:56.643199+0000","last_clean":"2024-04-25T18:43:56.643199+0000","last_became_active":"2024-04-25T18:43:16.242019+0000","last_became_peered":"2024-04-25T18:43:16.242019+0000","last_unstale":"2024-04-25T18:43:56.643199+0000","last_undegraded":"2024-04-25T18:43:56.643199+0000","last_fullsized":"2024-04-25T18:43:56.643199+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:13.238129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628408+0000","last_change":"2024-04-25T18:43:12.206606+0000","last_active":"2024-04-25T18:43:56.628408+0000","last_peered":"2024-04-25T18:43:56.628408+0000","last_clean":"2024-04-25T18:43:56.628408+0000","last_became_active":"2024-04-25T18:43:12.206238+0000","last_became_peered":"2024-04-25T18:43:12.206238+0000","last_unstale":"2024-04-25T18:43:56.628408+0000","last_undegraded":"2024-04-25T18:43:56.628408+0000","last_fullsized":"2024-04-25T18:43:56.628408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:07.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630224+0000","last_change":"2024-04-25T18:41:40.527363+0000","last_active":"2024-04-25T18:43:56.630224+0000","last_peered":"2024-04-25T18:43:56.630224+0000","last_clean":"2024-04-25T18:43:56.630224+0000","last_became_active":"2024-04-25T18:41:40.526907+0000","last_became_peered":"2024-04-25T18:41:40.526907+0000","last_unstale":"2024-04-25T18:43:56.630224+0000","last_undegraded":"2024-04-25T18:43:56.630224+0000","last_fullsized":"2024-04-25T18:43:56.630224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_clean_scrub_stamp":"2024-04-25T18:37:45.963565+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:16.689693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628418+0000","last_change":"2024-04-25T18:43:16.247552+0000","last_active":"2024-04-25T18:43:56.628418+0000","last_peered":"2024-04-25T18:43:56.628418+0000","last_clean":"2024-04-25T18:43:56.628418+0000","last_became_active":"2024-04-25T18:43:16.247178+0000","last_became_peered":"2024-04-25T18:43:16.247178+0000","last_unstale":"2024-04-25T18:43:56.628418+0000","last_undegraded":"2024-04-25T18:43:56.628418+0000","last_fullsized":"2024-04-25T18:43:56.628418+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:50.072986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.636197+0000","last_change":"2024-04-25T18:43:56.636197+0000","last_active":"2024-04-25T18:43:56.636197+0000","last_peered":"2024-04-25T18:43:56.636197+0000","last_clean":"2024-04-25T18:43:56.636197+0000","last_became_active":"2024-04-25T18:43:56.633431+0000","last_became_peered":"2024-04-25T18:43:56.633431+0000","last_unstale":"2024-04-25T18:43:56.636197+0000","last_undegraded":"2024-04-25T18:43:56.636197+0000","last_fullsized":"2024-04-25T18:43:56.636197+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:12.463489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628476+0000","last_change":"2024-04-25T18:43:12.202101+0000","last_active":"2024-04-25T18:43:56.628476+0000","last_peered":"2024-04-25T18:43:56.628476+0000","last_clean":"2024-04-25T18:43:56.628476+0000","last_became_active":"2024-04-25T18:43:12.201727+0000","last_became_peered":"2024-04-25T18:43:12.201727+0000","last_unstale":"2024-04-25T18:43:56.628476+0000","last_undegraded":"2024-04-25T18:43:56.628476+0000","last_fullsized":"2024-04-25T18:43:56.628476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:33.132072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.629820+0000","last_change":"2024-04-25T18:43:16.255433+0000","last_active":"2024-04-25T18:43:56.629820+0000","last_peered":"2024-04-25T18:43:56.629820+0000","last_clean":"2024-04-25T18:43:56.629820+0000","last_became_active":"2024-04-25T18:43:16.254866+0000","last_became_peered":"2024-04-25T18:43:16.254866+0000","last_unstale":"2024-04-25T18:43:56.629820+0000","last_undegraded":"2024-04-25T18:43:56.629820+0000","last_fullsized":"2024-04-25T18:43:56.629820+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:07.876973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:44:07.299116+0000","last_change":"2024-04-25T18:43:56.656998+0000","last_active":"2024-04-25T18:44:07.299116+0000","last_peered":"2024-04-25T18:44:07.299116+0000","last_clean":"2024-04-25T18:44:07.299116+0000","last_became_active":"2024-04-25T18:43:56.647456+0000","last_became_peered":"2024-04-25T18:43:56.647456+0000","last_unstale":"2024-04-25T18:44:07.299116+0000","last_undegraded":"2024-04-25T18:44:07.299116+0000","last_fullsized":"2024-04-25T18:44:07.299116+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:00.719426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.638894+0000","last_change":"2024-04-25T18:43:16.252826+0000","last_active":"2024-04-25T18:43:56.638894+0000","last_peered":"2024-04-25T18:43:56.638894+0000","last_clean":"2024-04-25T18:43:56.638894+0000","last_became_active":"2024-04-25T18:43:16.252435+0000","last_became_peered":"2024-04-25T18:43:16.252435+0000","last_unstale":"2024-04-25T18:43:56.638894+0000","last_undegraded":"2024-04-25T18:43:56.638894+0000","last_fullsized":"2024-04-25T18:43:56.638894+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:13.401006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628602+0000","last_change":"2024-04-25T18:43:12.376121+0000","last_active":"2024-04-25T18:43:56.628602+0000","last_peered":"2024-04-25T18:43:56.628602+0000","last_clean":"2024-04-25T18:43:56.628602+0000","last_became_active":"2024-04-25T18:43:12.375757+0000","last_became_peered":"2024-04-25T18:43:12.375757+0000","last_unstale":"2024-04-25T18:43:56.628602+0000","last_undegraded":"2024-04-25T18:43:56.628602+0000","last_fullsized":"2024-04-25T18:43:56.628602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:15.610442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.644030+0000","last_change":"2024-04-25T18:43:16.266504+0000","last_active":"2024-04-25T18:43:56.644030+0000","last_peered":"2024-04-25T18:43:56.644030+0000","last_clean":"2024-04-25T18:43:56.644030+0000","last_became_active":"2024-04-25T18:43:16.265048+0000","last_became_peered":"2024-04-25T18:43:16.265048+0000","last_unstale":"2024-04-25T18:43:56.644030+0000","last_undegraded":"2024-04-25T18:43:56.644030+0000","last_fullsized":"2024-04-25T18:43:56.644030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:36.924872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104519+0000","last_change":"2024-04-25T18:43:12.207529+0000","last_active":"2024-04-25T18:43:57.104519+0000","last_peered":"2024-04-25T18:43:57.104519+0000","last_clean":"2024-04-25T18:43:57.104519+0000","last_became_active":"2024-04-25T18:43:12.207122+0000","last_became_peered":"2024-04-25T18:43:12.207122+0000","last_unstale":"2024-04-25T18:43:57.104519+0000","last_undegraded":"2024-04-25T18:43:57.104519+0000","last_fullsized":"2024-04-25T18:43:57.104519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:44.518497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628095+0000","last_change":"2024-04-25T18:43:16.249616+0000","last_active":"2024-04-25T18:43:56.628095+0000","last_peered":"2024-04-25T18:43:56.628095+0000","last_clean":"2024-04-25T18:43:56.628095+0000","last_became_active":"2024-04-25T18:43:16.249252+0000","last_became_peered":"2024-04-25T18:43:16.249252+0000","last_unstale":"2024-04-25T18:43:56.628095+0000","last_undegraded":"2024-04-25T18:43:56.628095+0000","last_fullsized":"2024-04-25T18:43:56.628095+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:34.721513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.638233+0000","last_change":"2024-04-25T18:43:12.209304+0000","last_active":"2024-04-25T18:43:56.638233+0000","last_peered":"2024-04-25T18:43:56.638233+0000","last_clean":"2024-04-25T18:43:56.638233+0000","last_became_active":"2024-04-25T18:43:12.208887+0000","last_became_peered":"2024-04-25T18:43:12.208887+0000","last_unstale":"2024-04-25T18:43:56.638233+0000","last_undegraded":"2024-04-25T18:43:56.638233+0000","last_fullsized":"2024-04-25T18:43:56.638233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:22.049809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630290+0000","last_change":"2024-04-25T18:43:16.262482+0000","last_active":"2024-04-25T18:43:56.630290+0000","last_peered":"2024-04-25T18:43:56.630290+0000","last_clean":"2024-04-25T18:43:56.630290+0000","last_became_active":"2024-04-25T18:43:16.262108+0000","last_became_peered":"2024-04-25T18:43:16.262108+0000","last_unstale":"2024-04-25T18:43:56.630290+0000","last_undegraded":"2024-04-25T18:43:56.630290+0000","last_fullsized":"2024-04-25T18:43:56.630290+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:14.777939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.625872+0000","last_change":"2024-04-25T18:43:12.205742+0000","last_active":"2024-04-25T18:43:56.625872+0000","last_peered":"2024-04-25T18:43:56.625872+0000","last_clean":"2024-04-25T18:43:56.625872+0000","last_became_active":"2024-04-25T18:43:12.205314+0000","last_became_peered":"2024-04-25T18:43:12.205314+0000","last_unstale":"2024-04-25T18:43:56.625872+0000","last_undegraded":"2024-04-25T18:43:56.625872+0000","last_fullsized":"2024-04-25T18:43:56.625872+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:49.641568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.639020+0000","last_change":"2024-04-25T18:43:16.266100+0000","last_active":"2024-04-25T18:43:56.639020+0000","last_peered":"2024-04-25T18:43:56.639020+0000","last_clean":"2024-04-25T18:43:56.639020+0000","last_became_active":"2024-04-25T18:43:16.263060+0000","last_became_peered":"2024-04-25T18:43:16.263060+0000","last_unstale":"2024-04-25T18:43:56.639020+0000","last_undegraded":"2024-04-25T18:43:56.639020+0000","last_fullsized":"2024-04-25T18:43:56.639020+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:38.269244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628230+0000","last_change":"2024-04-25T18:43:12.196054+0000","last_active":"2024-04-25T18:43:56.628230+0000","last_peered":"2024-04-25T18:43:56.628230+0000","last_clean":"2024-04-25T18:43:56.628230+0000","last_became_active":"2024-04-25T18:43:12.195503+0000","last_became_peered":"2024-04-25T18:43:12.195503+0000","last_unstale":"2024-04-25T18:43:56.628230+0000","last_undegraded":"2024-04-25T18:43:56.628230+0000","last_fullsized":"2024-04-25T18:43:56.628230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:55.187176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630441+0000","last_change":"2024-04-25T18:43:16.242536+0000","last_active":"2024-04-25T18:43:56.630441+0000","last_peered":"2024-04-25T18:43:56.630441+0000","last_clean":"2024-04-25T18:43:56.630441+0000","last_became_active":"2024-04-25T18:43:16.241060+0000","last_became_peered":"2024-04-25T18:43:16.241060+0000","last_unstale":"2024-04-25T18:43:56.630441+0000","last_undegraded":"2024-04-25T18:43:56.630441+0000","last_fullsized":"2024-04-25T18:43:56.630441+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:38.306190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.653946+0000","last_change":"2024-04-25T18:43:56.653377+0000","last_active":"2024-04-25T18:43:56.653946+0000","last_peered":"2024-04-25T18:43:56.653946+0000","last_clean":"2024-04-25T18:43:56.653946+0000","last_became_active":"2024-04-25T18:43:56.645516+0000","last_became_peered":"2024-04-25T18:43:56.645516+0000","last_unstale":"2024-04-25T18:43:56.653946+0000","last_undegraded":"2024-04-25T18:43:56.653946+0000","last_fullsized":"2024-04-25T18:43:56.653946+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:11.189494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630697+0000","last_change":"2024-04-25T18:43:16.247610+0000","last_active":"2024-04-25T18:43:56.630697+0000","last_peered":"2024-04-25T18:43:56.630697+0000","last_clean":"2024-04-25T18:43:56.630697+0000","last_became_active":"2024-04-25T18:43:16.247273+0000","last_became_peered":"2024-04-25T18:43:16.247273+0000","last_unstale":"2024-04-25T18:43:56.630697+0000","last_undegraded":"2024-04-25T18:43:56.630697+0000","last_fullsized":"2024-04-25T18:43:56.630697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:26.151480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632846+0000","last_change":"2024-04-25T18:43:12.202011+0000","last_active":"2024-04-25T18:43:56.632846+0000","last_peered":"2024-04-25T18:43:56.632846+0000","last_clean":"2024-04-25T18:43:56.632846+0000","last_became_active":"2024-04-25T18:43:12.201447+0000","last_became_peered":"2024-04-25T18:43:12.201447+0000","last_unstale":"2024-04-25T18:43:56.632846+0000","last_undegraded":"2024-04-25T18:43:56.632846+0000","last_fullsized":"2024-04-25T18:43:56.632846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:48.094757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'1","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630412+0000","last_change":"2024-04-25T18:43:16.249033+0000","last_active":"2024-04-25T18:43:56.630412+0000","last_peered":"2024-04-25T18:43:56.630412+0000","last_clean":"2024-04-25T18:43:56.630412+0000","last_became_active":"2024-04-25T18:43:16.248691+0000","last_became_peered":"2024-04-25T18:43:16.248691+0000","last_unstale":"2024-04-25T18:43:56.630412+0000","last_undegraded":"2024-04-25T18:43:56.630412+0000","last_fullsized":"2024-04-25T18:43:56.630412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:37.335824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.629888+0000","last_change":"2024-04-25T18:43:12.205134+0000","last_active":"2024-04-25T18:43:56.629888+0000","last_peered":"2024-04-25T18:43:56.629888+0000","last_clean":"2024-04-25T18:43:56.629888+0000","last_became_active":"2024-04-25T18:43:12.204568+0000","last_became_peered":"2024-04-25T18:43:12.204568+0000","last_unstale":"2024-04-25T18:43:56.629888+0000","last_undegraded":"2024-04-25T18:43:56.629888+0000","last_fullsized":"2024-04-25T18:43:56.629888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:32.806580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630493+0000","last_change":"2024-04-25T18:43:16.247337+0000","last_active":"2024-04-25T18:43:56.630493+0000","last_peered":"2024-04-25T18:43:56.630493+0000","last_clean":"2024-04-25T18:43:56.630493+0000","last_became_active":"2024-04-25T18:43:16.246971+0000","last_became_peered":"2024-04-25T18:43:16.246971+0000","last_unstale":"2024-04-25T18:43:56.630493+0000","last_undegraded":"2024-04-25T18:43:56.630493+0000","last_fullsized":"2024-04-25T18:43:56.630493+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.882941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630424+0000","last_change":"2024-04-25T18:43:16.252196+0000","last_active":"2024-04-25T18:43:56.630424+0000","last_peered":"2024-04-25T18:43:56.630424+0000","last_clean":"2024-04-25T18:43:56.630424+0000","last_became_active":"2024-04-25T18:43:16.251715+0000","last_became_peered":"2024-04-25T18:43:16.251715+0000","last_unstale":"2024-04-25T18:43:56.630424+0000","last_undegraded":"2024-04-25T18:43:56.630424+0000","last_fullsized":"2024-04-25T18:43:56.630424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:17.218925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.625876+0000","last_change":"2024-04-25T18:43:16.247632+0000","last_active":"2024-04-25T18:43:56.625876+0000","last_peered":"2024-04-25T18:43:56.625876+0000","last_clean":"2024-04-25T18:43:56.625876+0000","last_became_active":"2024-04-25T18:43:16.247305+0000","last_became_peered":"2024-04-25T18:43:16.247305+0000","last_unstale":"2024-04-25T18:43:56.625876+0000","last_undegraded":"2024-04-25T18:43:56.625876+0000","last_fullsized":"2024-04-25T18:43:56.625876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:52.576146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.625809+0000","last_change":"2024-04-25T18:43:16.266539+0000","last_active":"2024-04-25T18:43:56.625809+0000","last_peered":"2024-04-25T18:43:56.625809+0000","last_clean":"2024-04-25T18:43:56.625809+0000","last_became_active":"2024-04-25T18:43:16.266083+0000","last_became_peered":"2024-04-25T18:43:16.266083+0000","last_unstale":"2024-04-25T18:43:56.625809+0000","last_undegraded":"2024-04-25T18:43:56.625809+0000","last_fullsized":"2024-04-25T18:43:56.625809+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:49.589019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.637737+0000","last_change":"2024-04-25T18:43:16.264002+0000","last_active":"2024-04-25T18:43:56.637737+0000","last_peered":"2024-04-25T18:43:56.637737+0000","last_clean":"2024-04-25T18:43:56.637737+0000","last_became_active":"2024-04-25T18:43:16.261555+0000","last_became_peered":"2024-04-25T18:43:16.261555+0000","last_unstale":"2024-04-25T18:43:56.637737+0000","last_undegraded":"2024-04-25T18:43:56.637737+0000","last_fullsized":"2024-04-25T18:43:56.637737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:12.637040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.629627+0000","last_change":"2024-04-25T18:43:16.270914+0000","last_active":"2024-04-25T18:43:56.629627+0000","last_peered":"2024-04-25T18:43:56.629627+0000","last_clean":"2024-04-25T18:43:56.629627+0000","last_became_active":"2024-04-25T18:43:16.270117+0000","last_became_peered":"2024-04-25T18:43:16.270117+0000","last_unstale":"2024-04-25T18:43:56.629627+0000","last_undegraded":"2024-04-25T18:43:56.629627+0000","last_fullsized":"2024-04-25T18:43:56.629627+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:19.874006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630713+0000","last_change":"2024-04-25T18:43:16.247032+0000","last_active":"2024-04-25T18:43:56.630713+0000","last_peered":"2024-04-25T18:43:56.630713+0000","last_clean":"2024-04-25T18:43:56.630713+0000","last_became_active":"2024-04-25T18:43:16.246734+0000","last_became_peered":"2024-04-25T18:43:16.246734+0000","last_unstale":"2024-04-25T18:43:56.630713+0000","last_undegraded":"2024-04-25T18:43:56.630713+0000","last_fullsized":"2024-04-25T18:43:56.630713+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:39.159143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626107+0000","last_change":"2024-04-25T18:43:16.245263+0000","last_active":"2024-04-25T18:43:56.626107+0000","last_peered":"2024-04-25T18:43:56.626107+0000","last_clean":"2024-04-25T18:43:56.626107+0000","last_became_active":"2024-04-25T18:43:16.244856+0000","last_became_peered":"2024-04-25T18:43:16.244856+0000","last_unstale":"2024-04-25T18:43:56.626107+0000","last_undegraded":"2024-04-25T18:43:56.626107+0000","last_fullsized":"2024-04-25T18:43:56.626107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:36.157927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.638597+0000","last_change":"2024-04-25T18:43:16.253523+0000","last_active":"2024-04-25T18:43:56.638597+0000","last_peered":"2024-04-25T18:43:56.638597+0000","last_clean":"2024-04-25T18:43:56.638597+0000","last_became_active":"2024-04-25T18:43:16.252689+0000","last_became_peered":"2024-04-25T18:43:16.252689+0000","last_unstale":"2024-04-25T18:43:56.638597+0000","last_undegraded":"2024-04-25T18:43:56.638597+0000","last_fullsized":"2024-04-25T18:43:56.638597+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.035788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632583+0000","last_change":"2024-04-25T18:43:16.255818+0000","last_active":"2024-04-25T18:43:56.632583+0000","last_peered":"2024-04-25T18:43:56.632583+0000","last_clean":"2024-04-25T18:43:56.632583+0000","last_became_active":"2024-04-25T18:43:16.255521+0000","last_became_peered":"2024-04-25T18:43:16.255521+0000","last_unstale":"2024-04-25T18:43:56.632583+0000","last_undegraded":"2024-04-25T18:43:56.632583+0000","last_fullsized":"2024-04-25T18:43:56.632583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:15.457585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.637917+0000","last_change":"2024-04-25T18:43:16.259680+0000","last_active":"2024-04-25T18:43:56.637917+0000","last_peered":"2024-04-25T18:43:56.637917+0000","last_clean":"2024-04-25T18:43:56.637917+0000","last_became_active":"2024-04-25T18:43:16.250065+0000","last_became_peered":"2024-04-25T18:43:16.250065+0000","last_unstale":"2024-04-25T18:43:56.637917+0000","last_undegraded":"2024-04-25T18:43:56.637917+0000","last_fullsized":"2024-04-25T18:43:56.637917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:44.974919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104744+0000","last_change":"2024-04-25T18:43:16.239801+0000","last_active":"2024-04-25T18:43:57.104744+0000","last_peered":"2024-04-25T18:43:57.104744+0000","last_clean":"2024-04-25T18:43:57.104744+0000","last_became_active":"2024-04-25T18:43:16.239475+0000","last_became_peered":"2024-04-25T18:43:16.239475+0000","last_unstale":"2024-04-25T18:43:57.104744+0000","last_undegraded":"2024-04-25T18:43:57.104744+0000","last_fullsized":"2024-04-25T18:43:57.104744+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:57.407476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.629851+0000","last_change":"2024-04-25T18:43:16.270419+0000","last_active":"2024-04-25T18:43:56.629851+0000","last_peered":"2024-04-25T18:43:56.629851+0000","last_clean":"2024-04-25T18:43:56.629851+0000","last_became_active":"2024-04-25T18:43:16.269995+0000","last_became_peered":"2024-04-25T18:43:16.269995+0000","last_unstale":"2024-04-25T18:43:56.629851+0000","last_undegraded":"2024-04-25T18:43:56.629851+0000","last_fullsized":"2024-04-25T18:43:56.629851+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:27.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630663+0000","last_change":"2024-04-25T18:43:16.246948+0000","last_active":"2024-04-25T18:43:56.630663+0000","last_peered":"2024-04-25T18:43:56.630663+0000","last_clean":"2024-04-25T18:43:56.630663+0000","last_became_active":"2024-04-25T18:43:16.246616+0000","last_became_peered":"2024-04-25T18:43:16.246616+0000","last_unstale":"2024-04-25T18:43:56.630663+0000","last_undegraded":"2024-04-25T18:43:56.630663+0000","last_fullsized":"2024-04-25T18:43:56.630663+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:51.817448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67073,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":251,"num_read_kb":195,"num_write":165,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":582,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252894,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":498757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514531,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32864,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712288,"statfs":{"total":95995035648,"available":95961382912,"internally_reserved":0,"allocated":1728512,"data_stored":1153310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710759,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":498993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972398,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":498913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234037,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1090843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430265,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":498382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":561247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":499653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":562406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117067,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":498693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":499401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691903,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1090527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:44:23.798 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T18:44:23.798 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T18:44:23.799 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T18:44:24.293 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:24 smithi171 ceph-mon[33266]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:44:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:24 smithi171 ceph-mon[33266]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T18:44:24.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:24 smithi112 ceph-mon[27864]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:44:24.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:24 smithi112 ceph-mon[27864]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T18:44:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:24 smithi161 ceph-mon[32609]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:44:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:24 smithi161 ceph-mon[32609]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T18:44:26.445 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:26 smithi112 ceph-mon[27864]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:26.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:26 smithi161 ceph-mon[32609]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:26.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:26 smithi171 ceph-mon[33266]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.780+0000 7fbcf7000700 1 -- 172.21.15.112:0/3544660027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf0104cd0 msgr2=0x7fbcf01050f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:26.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.780+0000 7fbcf7000700 1 --2- 172.21.15.112:0/3544660027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf0104cd0 0x7fbcf01050f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbce00099c0 tx=0x7fbce0009cd0 comp rx=0 tx=0).stop 2024-04-25T18:44:26.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.780+0000 7fbcf7000700 1 -- 172.21.15.112:0/3544660027 shutdown_connections 2024-04-25T18:44:26.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.780+0000 7fbcf7000700 1 --2- 172.21.15.112:0/3544660027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcf0106870 0x7fbcf010d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:26.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.780+0000 7fbcf7000700 1 --2- 172.21.15.112:0/3544660027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf0104cd0 0x7fbcf01050f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:26.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.780+0000 7fbcf7000700 1 --2- 172.21.15.112:0/3544660027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcf0105ed0 0x7fbcf0106330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:26.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.780+0000 7fbcf7000700 1 -- 172.21.15.112:0/3544660027 >> 172.21.15.112:0/3544660027 conn(0x7fbcf0100290 msgr2=0x7fbcf01026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:26.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.781+0000 7fbcf7000700 1 -- 172.21.15.112:0/3544660027 shutdown_connections 2024-04-25T18:44:26.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.781+0000 7fbcf7000700 1 -- 172.21.15.112:0/3544660027 wait complete. 2024-04-25T18:44:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcf7000700 1 Processor -- start 2024-04-25T18:44:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcf7000700 1 -- start start 2024-04-25T18:44:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcf7000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcf0104cd0 0x7fbcf019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcf7000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf0105ed0 0x7fbcf019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcf7000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcf0106870 0x7fbcf01a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcf7000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0110760 con 0x7fbcf0105ed0 2024-04-25T18:44:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcf7000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01105e0 con 0x7fbcf0106870 2024-04-25T18:44:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcf7000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01108e0 con 0x7fbcf0104cd0 2024-04-25T18:44:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcef400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf0105ed0 0x7fbcf019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcef400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf0105ed0 0x7fbcf019b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42318/0 (socket says 172.21.15.112:42318) 2024-04-25T18:44:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcef400700 1 -- 172.21.15.112:0/562596770 learned_addr learned my addr 172.21.15.112:0/562596770 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcefe00700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcf0104cd0 0x7fbcf019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.782+0000 7fbcf5000700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcf0106870 0x7fbcf01a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.783+0000 7fbcef400700 1 -- 172.21.15.112:0/562596770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcf0104cd0 msgr2=0x7fbcf019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.783+0000 7fbcef400700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcf0104cd0 0x7fbcf019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.783+0000 7fbcef400700 1 -- 172.21.15.112:0/562596770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcf0106870 msgr2=0x7fbcf01a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.783+0000 7fbcef400700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcf0106870 0x7fbcf01a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.783+0000 7fbcef400700 1 -- 172.21.15.112:0/562596770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf01a2980 con 0x7fbcf0105ed0 2024-04-25T18:44:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.783+0000 7fbcef400700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf0105ed0 0x7fbcf019b270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00e9c0 tx=0x7fbcdc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.783+0000 7fbcecc00700 1 -- 172.21.15.112:0/562596770 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc00cbd0 con 0x7fbcf0105ed0 2024-04-25T18:44:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.784+0000 7fbcecc00700 1 -- 172.21.15.112:0/562596770 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc00ee60 con 0x7fbcf0105ed0 2024-04-25T18:44:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.784+0000 7fbcf7000700 1 -- 172.21.15.112:0/562596770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf01a2c70 con 0x7fbcf0105ed0 2024-04-25T18:44:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.784+0000 7fbcecc00700 1 -- 172.21.15.112:0/562596770 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0055c0 con 0x7fbcf0105ed0 2024-04-25T18:44:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.784+0000 7fbcf7000700 1 -- 172.21.15.112:0/562596770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf01a31b0 con 0x7fbcf0105ed0 2024-04-25T18:44:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.785+0000 7fbcf7000700 1 -- 172.21.15.112:0/562596770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf0050f80 con 0x7fbcf0105ed0 2024-04-25T18:44:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.787+0000 7fbcecc00700 1 -- 172.21.15.112:0/562596770 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbcdc004750 con 0x7fbcf0105ed0 2024-04-25T18:44:26.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.788+0000 7fbcecc00700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbcd40a3e00 0x7fbcd40a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:26.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.788+0000 7fbcecc00700 1 -- 172.21.15.112:0/562596770 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fbcdc0d07f0 con 0x7fbcf0105ed0 2024-04-25T18:44:26.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.788+0000 7fbcefe00700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbcd40a3e00 0x7fbcd40a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:26.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.789+0000 7fbcefe00700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbcd40a3e00 0x7fbcd40a62c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009990 tx=0x7fbce0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:26.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:26.793+0000 7fbcecc00700 1 -- 172.21.15.112:0/562596770 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcdc09cff0 con 0x7fbcf0105ed0 2024-04-25T18:44:27.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.113+0000 7fbcf7000700 1 -- 172.21.15.112:0/562596770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fbcf002e190 con 0x7fbcf0105ed0 2024-04-25T18:44:27.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.119+0000 7fbcecc00700 1 -- 172.21.15.112:0/562596770 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fbcdc09c8f0 con 0x7fbcf0105ed0 2024-04-25T18:44:27.120 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-04-25T18:44:27.120 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCLpCpmEeHeBhAA4xG+a15C4crabu0vrARNEQ== 2024-04-25T18:44:27.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 -- 172.21.15.112:0/562596770 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbcd40a3e00 msgr2=0x7fbcd40a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:27.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbcd40a3e00 0x7fbcd40a62c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009990 tx=0x7fbce0005fb0 comp rx=0 tx=0).stop 2024-04-25T18:44:27.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 -- 172.21.15.112:0/562596770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf0105ed0 msgr2=0x7fbcf019b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf0105ed0 0x7fbcf019b270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00e9c0 tx=0x7fbcdc00ecd0 comp rx=0 tx=0).stop 2024-04-25T18:44:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 -- 172.21.15.112:0/562596770 shutdown_connections 2024-04-25T18:44:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fbcd40a3e00 0x7fbcd40a62c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcf0106870 0x7fbcf01a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcf0104cd0 0x7fbcf019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 --2- 172.21.15.112:0/562596770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcf0105ed0 0x7fbcf019b270 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:27.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 -- 172.21.15.112:0/562596770 >> 172.21.15.112:0/562596770 conn(0x7fbcf0100290 msgr2=0x7fbcf0109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:27.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 -- 172.21.15.112:0/562596770 shutdown_connections 2024-04-25T18:44:27.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:27.122+0000 7fbcf7000700 1 -- 172.21.15.112:0/562596770 wait complete. 2024-04-25T18:44:27.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:27 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/562596770' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:44:27.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:27 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/562596770' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T18:44:27.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:27 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/562596770' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:44:27.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:27 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/562596770' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T18:44:27.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:27 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/562596770' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:44:27.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:27 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/562596770' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T18:44:28.095 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:44:28.096 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T18:44:28.096 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T18:44:28.202 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T18:44:28.202 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T18:44:28.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T18:44:28.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:28 smithi112 ceph-mon[27864]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:44:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:28 smithi161 ceph-mon[32609]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:44:28.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:28 smithi171 ceph-mon[33266]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:44:28.695 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:30 smithi161 ceph-mon[32609]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:30.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:30 smithi112 ceph-mon[27864]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:30.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:30 smithi171 ceph-mon[33266]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.156+0000 7f5758000700 1 -- 172.21.15.112:0/2676371494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57501034b0 msgr2=0x7f575010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.156+0000 7f5758000700 1 --2- 172.21.15.112:0/2676371494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57501034b0 0x7f575010f920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f574800b0d0 tx=0x7f574800b3e0 comp rx=0 tx=0).stop 2024-04-25T18:44:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.156+0000 7f5758000700 1 -- 172.21.15.112:0/2676371494 shutdown_connections 2024-04-25T18:44:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.156+0000 7f5758000700 1 --2- 172.21.15.112:0/2676371494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57501034b0 0x7f575010f920 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.156+0000 7f5758000700 1 --2- 172.21.15.112:0/2676371494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5750107940 0x7f5750107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.157+0000 7f5758000700 1 --2- 172.21.15.112:0/2676371494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5750102af0 0x7f5750102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.157+0000 7f5758000700 1 -- 172.21.15.112:0/2676371494 >> 172.21.15.112:0/2676371494 conn(0x7f5750079ba0 msgr2=0x7f5750079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:31.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.157+0000 7f5758000700 1 -- 172.21.15.112:0/2676371494 shutdown_connections 2024-04-25T18:44:31.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.157+0000 7f5758000700 1 -- 172.21.15.112:0/2676371494 wait complete. 2024-04-25T18:44:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.158+0000 7f5758000700 1 Processor -- start 2024-04-25T18:44:31.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.158+0000 7f5758000700 1 -- start start 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.158+0000 7f5758000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5750102af0 0x7f575019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.158+0000 7f5758000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57501034b0 0x7f575019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.158+0000 7f5758000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5750107940 0x7f575019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.158+0000 7f5758000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5750112ca0 con 0x7f5750102af0 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.158+0000 7f5758000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5750112b20 con 0x7f5750107940 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.158+0000 7f5758000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5750112e20 con 0x7f57501034b0 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5755600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5750102af0 0x7f575019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5754c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57501034b0 0x7f575019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5754c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57501034b0 0x7f575019b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:43452/0 (socket says 172.21.15.112:43452) 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5755600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5750102af0 0x7f575019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44274/0 (socket says 172.21.15.112:44274) 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5754c00700 1 -- 172.21.15.112:0/3754306472 learned_addr learned my addr 172.21.15.112:0/3754306472 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:31.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5756000700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5750107940 0x7f575019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5755600700 1 -- 172.21.15.112:0/3754306472 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57501034b0 msgr2=0x7f575019b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5755600700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57501034b0 0x7f575019b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5755600700 1 -- 172.21.15.112:0/3754306472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5750107940 msgr2=0x7f575019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5755600700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5750107940 0x7f575019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5755600700 1 -- 172.21.15.112:0/3754306472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57501a02f0 con 0x7f5750102af0 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5754c00700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57501034b0 0x7f575019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.159+0000 7f5755600700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5750102af0 0x7f575019b2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f574400e9f0 tx=0x7f574400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.160+0000 7f574e000700 1 -- 172.21.15.112:0/3754306472 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574400c9c0 con 0x7f5750102af0 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.160+0000 7f5758000700 1 -- 172.21.15.112:0/3754306472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57501a05e0 con 0x7f5750102af0 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.160+0000 7f574e000700 1 -- 172.21.15.112:0/3754306472 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f574400cb60 con 0x7f5750102af0 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.160+0000 7f574e000700 1 -- 172.21.15.112:0/3754306472 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744018680 con 0x7f5750102af0 2024-04-25T18:44:31.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.160+0000 7f5758000700 1 -- 172.21.15.112:0/3754306472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57501a7e60 con 0x7f5750102af0 2024-04-25T18:44:31.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.161+0000 7f5758000700 1 -- 172.21.15.112:0/3754306472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5750069730 con 0x7f5750102af0 2024-04-25T18:44:31.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.163+0000 7f574e000700 1 -- 172.21.15.112:0/3754306472 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5744010430 con 0x7f5750102af0 2024-04-25T18:44:31.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.164+0000 7f574e000700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f57380a3d50 0x7f57380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:31.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.164+0000 7f574e000700 1 -- 172.21.15.112:0/3754306472 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f57440d0a30 con 0x7f5750102af0 2024-04-25T18:44:31.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.164+0000 7f5754c00700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f57380a3d50 0x7f57380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:31.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.165+0000 7f5754c00700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f57380a3d50 0x7f57380a6210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5750102a90 tx=0x7f573c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:31.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.167+0000 7f574e000700 1 -- 172.21.15.112:0/3754306472 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f574409d230 con 0x7f5750102af0 2024-04-25T18:44:31.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.460+0000 7f5758000700 1 -- 172.21.15.112:0/3754306472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f57501a8090 con 0x7f5750102af0 2024-04-25T18:44:31.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.477+0000 7f574e000700 1 -- 172.21.15.112:0/3754306472 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+265002 (secure 0 0 0) 0x7f574409cb30 con 0x7f5750102af0 2024-04-25T18:44:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:44:31.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 -- 172.21.15.112:0/3754306472 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f57380a3d50 msgr2=0x7f57380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:31.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f57380a3d50 0x7f57380a6210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5750102a90 tx=0x7f573c009260 comp rx=0 tx=0).stop 2024-04-25T18:44:31.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 -- 172.21.15.112:0/3754306472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5750102af0 msgr2=0x7f575019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:31.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5750102af0 0x7f575019b2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f574400e9f0 tx=0x7f574400ed00 comp rx=0 tx=0).stop 2024-04-25T18:44:31.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 -- 172.21.15.112:0/3754306472 shutdown_connections 2024-04-25T18:44:31.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f57380a3d50 0x7f57380a6210 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:31.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5750107940 0x7f575019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:31.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5750102af0 0x7f575019b2e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:31.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 --2- 172.21.15.112:0/3754306472 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57501034b0 0x7f575019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:31.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 -- 172.21.15.112:0/3754306472 >> 172.21.15.112:0/3754306472 conn(0x7f5750079ba0 msgr2=0x7f5750100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:31.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 -- 172.21.15.112:0/3754306472 shutdown_connections 2024-04-25T18:44:31.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:31.482+0000 7f5758000700 1 -- 172.21.15.112:0/3754306472 wait complete. 2024-04-25T18:44:32.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:32 smithi112 ceph-mon[27864]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:32.439 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:32 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3754306472' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T18:44:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:32 smithi161 ceph-mon[32609]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:32 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3754306472' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T18:44:32.495 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":605196912},{"type":"v1","addr":"172.21.15.112:6801","nonce":605196912}]},"active_addr":"172.21.15.112:6801/605196912","active_change":"2024-04-25T18:34:54.948489+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.112:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2936464788}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2665612708}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":698167759}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":722306877}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2056294305}]}]} 2024-04-25T18:44:32.504 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T18:44:32.504 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T18:44:32.505 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:44:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:32 smithi171 ceph-mon[33266]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:32 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3754306472' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T18:44:32.948 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:34 smithi161 ceph-mon[32609]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:34.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:34 smithi112 ceph-mon[27864]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:34.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:34 smithi171 ceph-mon[33266]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:44:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.366+0000 7ff578a00700 1 -- 172.21.15.112:0/256283980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff574103450 msgr2=0x7ff57410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.366+0000 7ff578a00700 1 --2- 172.21.15.112:0/256283980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff574103450 0x7ff57410f8b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00b0a0 tx=0x7ff56c00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:44:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.366+0000 7ff578a00700 1 -- 172.21.15.112:0/256283980 shutdown_connections 2024-04-25T18:44:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.366+0000 7ff578a00700 1 --2- 172.21.15.112:0/256283980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff574103450 0x7ff57410f8b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.366+0000 7ff578a00700 1 --2- 172.21.15.112:0/256283980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff574108a50 0x7ff574108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.366+0000 7ff578a00700 1 --2- 172.21.15.112:0/256283980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff574102a90 0x7ff574102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.366+0000 7ff578a00700 1 -- 172.21.15.112:0/256283980 >> 172.21.15.112:0/256283980 conn(0x7ff574079ba0 msgr2=0x7ff574079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.366+0000 7ff578a00700 1 -- 172.21.15.112:0/256283980 shutdown_connections 2024-04-25T18:44:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.366+0000 7ff578a00700 1 -- 172.21.15.112:0/256283980 wait complete. 2024-04-25T18:44:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.367+0000 7ff578a00700 1 Processor -- start 2024-04-25T18:44:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.367+0000 7ff578a00700 1 -- start start 2024-04-25T18:44:35.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.368+0000 7ff578a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff574102a90 0x7ff574074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:35.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.368+0000 7ff578a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff574103450 0x7ff574075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:35.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.368+0000 7ff578a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff574108a50 0x7ff5740757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:35.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.368+0000 7ff578a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff574112c50 con 0x7ff574103450 2024-04-25T18:44:35.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.368+0000 7ff578a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff574112ad0 con 0x7ff574108a50 2024-04-25T18:44:35.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.368+0000 7ff578a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff574112dd0 con 0x7ff574102a90 2024-04-25T18:44:35.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.368+0000 7ff569400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff574103450 0x7ff574075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:35.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.368+0000 7ff571e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff574102a90 0x7ff574074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:35.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.368+0000 7ff569400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff574103450 0x7ff574075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44300/0 (socket says 172.21.15.112:44300) 2024-04-25T18:44:35.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.368+0000 7ff572800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff574108a50 0x7ff5740757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:35.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.368+0000 7ff569400700 1 -- 172.21.15.112:0/1605659614 learned_addr learned my addr 172.21.15.112:0/1605659614 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:35.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.369+0000 7ff572800700 1 -- 172.21.15.112:0/1605659614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff574102a90 msgr2=0x7ff574074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:35.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.369+0000 7ff572800700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff574102a90 0x7ff574074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:35.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.369+0000 7ff572800700 1 -- 172.21.15.112:0/1605659614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff574103450 msgr2=0x7ff574075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:35.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.369+0000 7ff572800700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff574103450 0x7ff574075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:35.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.369+0000 7ff572800700 1 -- 172.21.15.112:0/1605659614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5741a7650 con 0x7ff574108a50 2024-04-25T18:44:35.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.369+0000 7ff571e00700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff574102a90 0x7ff574074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:44:35.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.369+0000 7ff572800700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff574108a50 0x7ff5740757c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00b070 tx=0x7ff56c004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:35.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.369+0000 7ff56b400700 1 -- 172.21.15.112:0/1605659614 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c021070 con 0x7ff574108a50 2024-04-25T18:44:35.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.369+0000 7ff578a00700 1 -- 172.21.15.112:0/1605659614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5741a78e0 con 0x7ff574108a50 2024-04-25T18:44:35.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.369+0000 7ff56b400700 1 -- 172.21.15.112:0/1605659614 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff56c007e30 con 0x7ff574108a50 2024-04-25T18:44:35.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.370+0000 7ff56b400700 1 -- 172.21.15.112:0/1605659614 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0045f0 con 0x7ff574108a50 2024-04-25T18:44:35.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.370+0000 7ff578a00700 1 -- 172.21.15.112:0/1605659614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5741a7e70 con 0x7ff574108a50 2024-04-25T18:44:35.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.371+0000 7ff578a00700 1 -- 172.21.15.112:0/1605659614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff574069730 con 0x7ff574108a50 2024-04-25T18:44:35.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.373+0000 7ff56b400700 1 -- 172.21.15.112:0/1605659614 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff56c007980 con 0x7ff574108a50 2024-04-25T18:44:35.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.374+0000 7ff56b400700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff5580a3d50 0x7ff5580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:35.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.374+0000 7ff56b400700 1 -- 172.21.15.112:0/1605659614 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7ff56c00f020 con 0x7ff574108a50 2024-04-25T18:44:35.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.374+0000 7ff571e00700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff5580a3d50 0x7ff5580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:35.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.375+0000 7ff571e00700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff5580a3d50 0x7ff5580a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff564009fd0 tx=0x7ff564009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:35.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.377+0000 7ff56b400700 1 -- 172.21.15.112:0/1605659614 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff56c02a020 con 0x7ff574108a50 2024-04-25T18:44:35.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.608+0000 7ff578a00700 1 -- 172.21.15.112:0/1605659614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff57402e140 con 0x7ff574108a50 2024-04-25T18:44:35.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.610+0000 7ff56b400700 1 -- 172.21.15.112:0/1605659614 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+22709 (secure 0 0 0) 0x7ff56c09e4f0 con 0x7ff574108a50 2024-04-25T18:44:35.611 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:44:35.611 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","created":"2024-04-25T18:33:37.379089+0000","modified":"2024-04-25T18:43:56.620122+0000","last_up_change":"2024-04-25T18:42:00.961646+0000","last_in_change":"2024-04-25T18:41:40.647162+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:37:45.106199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:43:11.017975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:43:15.116917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efb4bc01-e29b-4402-8a7a-3387d1731ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6803","nonce":232254535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6805","nonce":232254535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6809","nonce":232254535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6807","nonce":232254535}]},"public_addr":"172.21.15.112:6803/232254535","cluster_addr":"172.21.15.112:6805/232254535","heartbeat_back_addr":"172.21.15.112:6809/232254535","heartbeat_front_addr":"172.21.15.112:6807/232254535","state":["exists","up"]},{"osd":1,"uuid":"65a83fa2-96a6-4743-8c17-21531f5e885e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6801","nonce":664224744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6803","nonce":664224744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6807","nonce":664224744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6805","nonce":664224744}]},"public_addr":"172.21.15.161:6801/664224744","cluster_addr":"172.21.15.161:6803/664224744","heartbeat_back_addr":"172.21.15.161:6807/664224744","heartbeat_front_addr":"172.21.15.161:6805/664224744","state":["exists","up"]},{"osd":2,"uuid":"7562ec4c-c126-48ec-a51c-56fa461a2e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6801","nonce":345172073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6803","nonce":345172073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6807","nonce":345172073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6805","nonce":345172073}]},"public_addr":"172.21.15.171:6801/345172073","cluster_addr":"172.21.15.171:6803/345172073","heartbeat_back_addr":"172.21.15.171:6807/345172073","heartbeat_front_addr":"172.21.15.171:6805/345172073","state":["exists","up"]},{"osd":3,"uuid":"d7aae8cd-fec7-46d9-8773-25eaaca3cc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6811","nonce":1472407156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6813","nonce":1472407156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6817","nonce":1472407156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6815","nonce":1472407156}]},"public_addr":"172.21.15.112:6811/1472407156","cluster_addr":"172.21.15.112:6813/1472407156","heartbeat_back_addr":"172.21.15.112:6817/1472407156","heartbeat_front_addr":"172.21.15.112:6815/1472407156","state":["exists","up"]},{"osd":4,"uuid":"b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6809","nonce":373204306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6811","nonce":373204306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6815","nonce":373204306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6813","nonce":373204306}]},"public_addr":"172.21.15.161:6809/373204306","cluster_addr":"172.21.15.161:6811/373204306","heartbeat_back_addr":"172.21.15.161:6815/373204306","heartbeat_front_addr":"172.21.15.161:6813/373204306","state":["exists","up"]},{"osd":5,"uuid":"56124d12-bc1f-4cba-8430-938248851776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6809","nonce":2643512313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6811","nonce":2643512313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6815","nonce":2643512313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6813","nonce":2643512313}]},"public_addr":"172.21.15.171:6809/2643512313","cluster_addr":"172.21.15.171:6811/2643512313","heartbeat_back_addr":"172.21.15.171:6815/2643512313","heartbeat_front_addr":"172.21.15.171:6813/2643512313","state":["exists","up"]},{"osd":6,"uuid":"9563a9d3-352b-4686-975c-282ec167248b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6819","nonce":1933654933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6821","nonce":1933654933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6825","nonce":1933654933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6823","nonce":1933654933}]},"public_addr":"172.21.15.112:6819/1933654933","cluster_addr":"172.21.15.112:6821/1933654933","heartbeat_back_addr":"172.21.15.112:6825/1933654933","heartbeat_front_addr":"172.21.15.112:6823/1933654933","state":["exists","up"]},{"osd":7,"uuid":"f2d06e86-6aac-40f4-956e-18fe0b0e3543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6817","nonce":3060201138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6819","nonce":3060201138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6823","nonce":3060201138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6821","nonce":3060201138}]},"public_addr":"172.21.15.161:6817/3060201138","cluster_addr":"172.21.15.161:6819/3060201138","heartbeat_back_addr":"172.21.15.161:6823/3060201138","heartbeat_front_addr":"172.21.15.161:6821/3060201138","state":["exists","up"]},{"osd":8,"uuid":"f006e45f-2bb9-45d0-9020-be7c33c6a825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6817","nonce":835756571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6819","nonce":835756571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6823","nonce":835756571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6821","nonce":835756571}]},"public_addr":"172.21.15.171:6817/835756571","cluster_addr":"172.21.15.171:6819/835756571","heartbeat_back_addr":"172.21.15.171:6823/835756571","heartbeat_front_addr":"172.21.15.171:6821/835756571","state":["exists","up"]},{"osd":9,"uuid":"c5a96491-4151-4633-b3bc-2c214fe51c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6827","nonce":349758528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6829","nonce":349758528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6833","nonce":349758528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6831","nonce":349758528}]},"public_addr":"172.21.15.112:6827/349758528","cluster_addr":"172.21.15.112:6829/349758528","heartbeat_back_addr":"172.21.15.112:6833/349758528","heartbeat_front_addr":"172.21.15.112:6831/349758528","state":["exists","up"]},{"osd":10,"uuid":"244eda3e-25a0-490e-879f-c1e2726c1be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6825","nonce":3013292765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6827","nonce":3013292765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6831","nonce":3013292765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6829","nonce":3013292765}]},"public_addr":"172.21.15.161:6825/3013292765","cluster_addr":"172.21.15.161:6827/3013292765","heartbeat_back_addr":"172.21.15.161:6831/3013292765","heartbeat_front_addr":"172.21.15.161:6829/3013292765","state":["exists","up"]},{"osd":11,"uuid":"356e5c54-bbc3-45ee-bda2-85164ce3cb92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6825","nonce":1010336458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6827","nonce":1010336458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6831","nonce":1010336458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6829","nonce":1010336458}]},"public_addr":"172.21.15.171:6825/1010336458","cluster_addr":"172.21.15.171:6827/1010336458","heartbeat_back_addr":"172.21.15.171:6831/1010336458","heartbeat_front_addr":"172.21.15.171:6829/1010336458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:36:51.024650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:17.026638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:41.239726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:15.148968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:41.029118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:06.225858+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:41.639444+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:07.753400+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:32.292085+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:07.159926+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:33.749782+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:58.594447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/181862246":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/3393454453":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3195850527":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6801/3468460974":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3392048544":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/1329430933":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/735578423":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/3468460974":"2024-04-26T18:34:20.605775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:44:35.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.612+0000 7ff578a00700 1 -- 172.21.15.112:0/1605659614 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff5580a3d50 msgr2=0x7ff5580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:35.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.612+0000 7ff578a00700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff5580a3d50 0x7ff5580a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff564009fd0 tx=0x7ff564009260 comp rx=0 tx=0).stop 2024-04-25T18:44:35.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.612+0000 7ff578a00700 1 -- 172.21.15.112:0/1605659614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff574108a50 msgr2=0x7ff5740757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:35.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.612+0000 7ff578a00700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff574108a50 0x7ff5740757c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00b070 tx=0x7ff56c004c80 comp rx=0 tx=0).stop 2024-04-25T18:44:35.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.613+0000 7ff578a00700 1 -- 172.21.15.112:0/1605659614 shutdown_connections 2024-04-25T18:44:35.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.613+0000 7ff578a00700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff5580a3d50 0x7ff5580a6210 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:35.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.613+0000 7ff578a00700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff574108a50 0x7ff5740757c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:35.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.613+0000 7ff578a00700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff574102a90 0x7ff574074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:35.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.613+0000 7ff578a00700 1 --2- 172.21.15.112:0/1605659614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff574103450 0x7ff574075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:35.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.613+0000 7ff578a00700 1 -- 172.21.15.112:0/1605659614 >> 172.21.15.112:0/1605659614 conn(0x7ff574079ba0 msgr2=0x7ff574101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:35.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.613+0000 7ff578a00700 1 -- 172.21.15.112:0/1605659614 shutdown_connections 2024-04-25T18:44:35.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:35.613+0000 7ff578a00700 1 -- 172.21.15.112:0/1605659614 wait complete. 2024-04-25T18:44:36.359 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:36 smithi112 ceph-mon[27864]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:36.359 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:36 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1605659614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:44:36.444 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T18:44:36.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:44:36.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:36 smithi171 ceph-mon[33266]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:36.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:36 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1605659614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:44:36.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:36 smithi161 ceph-mon[32609]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:36.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:36 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1605659614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:44:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:38 smithi171 ceph-mon[33266]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:44:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:38 smithi161 ceph-mon[32609]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:44:38.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:38 smithi112 ceph-mon[27864]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:44:39.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.267+0000 7f336ac00700 1 -- 172.21.15.112:0/2893336516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3364102a90 msgr2=0x7f3364102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:39.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.267+0000 7f336ac00700 1 --2- 172.21.15.112:0/2893336516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3364102a90 0x7f3364102f10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3350009990 tx=0x7f3350009ca0 comp rx=0 tx=0).stop 2024-04-25T18:44:39.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.267+0000 7f336ac00700 1 -- 172.21.15.112:0/2893336516 shutdown_connections 2024-04-25T18:44:39.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.267+0000 7f336ac00700 1 --2- 172.21.15.112:0/2893336516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3364103450 0x7f336410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:39.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.267+0000 7f336ac00700 1 --2- 172.21.15.112:0/2893336516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364108a50 0x7f3364108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:39.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.267+0000 7f336ac00700 1 --2- 172.21.15.112:0/2893336516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3364102a90 0x7f3364102f10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:39.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.267+0000 7f336ac00700 1 -- 172.21.15.112:0/2893336516 >> 172.21.15.112:0/2893336516 conn(0x7f3364079ba0 msgr2=0x7f3364079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:39.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.267+0000 7f336ac00700 1 -- 172.21.15.112:0/2893336516 shutdown_connections 2024-04-25T18:44:39.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.267+0000 7f336ac00700 1 -- 172.21.15.112:0/2893336516 wait complete. 2024-04-25T18:44:39.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.268+0000 7f336ac00700 1 Processor -- start 2024-04-25T18:44:39.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f336ac00700 1 -- start start 2024-04-25T18:44:39.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f336ac00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3364102a90 0x7f336419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:39.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f336ac00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364103450 0x7f336419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:39.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f336ac00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3364108a50 0x7f336419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:39.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f336ac00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3364112c50 con 0x7f3364103450 2024-04-25T18:44:39.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f336ac00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3364112ad0 con 0x7f3364108a50 2024-04-25T18:44:39.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f336ac00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3364112dd0 con 0x7f3364102a90 2024-04-25T18:44:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f3363400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364103450 0x7f336419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f3363e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3364102a90 0x7f336419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f3368c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3364108a50 0x7f336419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f3363400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364103450 0x7f336419b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44316/0 (socket says 172.21.15.112:44316) 2024-04-25T18:44:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.269+0000 7f3363400700 1 -- 172.21.15.112:0/1744530679 learned_addr learned my addr 172.21.15.112:0/1744530679 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.270+0000 7f3363e00700 1 -- 172.21.15.112:0/1744530679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3364108a50 msgr2=0x7f336419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.270+0000 7f3363e00700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3364108a50 0x7f336419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.270+0000 7f3363e00700 1 -- 172.21.15.112:0/1744530679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364103450 msgr2=0x7f336419b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.270+0000 7f3363e00700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364103450 0x7f336419b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.270+0000 7f3363e00700 1 -- 172.21.15.112:0/1744530679 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33641a0430 con 0x7f3364102a90 2024-04-25T18:44:39.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.270+0000 7f3368c00700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3364108a50 0x7f336419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:44:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.270+0000 7f3363400700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364103450 0x7f336419b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:44:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.270+0000 7f3363e00700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3364102a90 0x7f336419b420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f335400e9f0 tx=0x7f335400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.271+0000 7f3360c00700 1 -- 172.21.15.112:0/1744530679 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335400cb70 con 0x7f3364102a90 2024-04-25T18:44:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.271+0000 7f3360c00700 1 -- 172.21.15.112:0/1744530679 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f335400ee80 con 0x7f3364102a90 2024-04-25T18:44:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.271+0000 7f336ac00700 1 -- 172.21.15.112:0/1744530679 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33641a0720 con 0x7f3364102a90 2024-04-25T18:44:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.271+0000 7f3360c00700 1 -- 172.21.15.112:0/1744530679 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354003d40 con 0x7f3364102a90 2024-04-25T18:44:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.271+0000 7f336ac00700 1 -- 172.21.15.112:0/1744530679 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33641a7fa0 con 0x7f3364102a90 2024-04-25T18:44:39.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.272+0000 7f336ac00700 1 -- 172.21.15.112:0/1744530679 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3364069730 con 0x7f3364102a90 2024-04-25T18:44:39.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.274+0000 7f3360c00700 1 -- 172.21.15.112:0/1744530679 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3354003ee0 con 0x7f3364102a90 2024-04-25T18:44:39.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.275+0000 7f3360c00700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33480a3d50 0x7f33480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:39.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.275+0000 7f3360c00700 1 -- 172.21.15.112:0/1744530679 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f33540d0620 con 0x7f3364102a90 2024-04-25T18:44:39.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.275+0000 7f3363400700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33480a3d50 0x7f33480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:39.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.276+0000 7f3363400700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33480a3d50 0x7f33480a6210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f336410fc60 tx=0x7f335001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:39.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.280+0000 7f3360c00700 1 -- 172.21.15.112:0/1744530679 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f335409ce20 con 0x7f3364102a90 2024-04-25T18:44:39.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.507+0000 7f336ac00700 1 -- 172.21.15.112:0/1744530679 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33641a81a0 con 0x7f3364102a90 2024-04-25T18:44:39.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.510+0000 7f3360c00700 1 -- 172.21.15.112:0/1744530679 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+22709 (secure 0 0 0) 0x7f335409c720 con 0x7f3364102a90 2024-04-25T18:44:39.512 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:44:39.512 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","created":"2024-04-25T18:33:37.379089+0000","modified":"2024-04-25T18:43:56.620122+0000","last_up_change":"2024-04-25T18:42:00.961646+0000","last_in_change":"2024-04-25T18:41:40.647162+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:37:45.106199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:43:11.017975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:43:15.116917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efb4bc01-e29b-4402-8a7a-3387d1731ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6803","nonce":232254535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6805","nonce":232254535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6809","nonce":232254535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6807","nonce":232254535}]},"public_addr":"172.21.15.112:6803/232254535","cluster_addr":"172.21.15.112:6805/232254535","heartbeat_back_addr":"172.21.15.112:6809/232254535","heartbeat_front_addr":"172.21.15.112:6807/232254535","state":["exists","up"]},{"osd":1,"uuid":"65a83fa2-96a6-4743-8c17-21531f5e885e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6801","nonce":664224744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6803","nonce":664224744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6807","nonce":664224744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6805","nonce":664224744}]},"public_addr":"172.21.15.161:6801/664224744","cluster_addr":"172.21.15.161:6803/664224744","heartbeat_back_addr":"172.21.15.161:6807/664224744","heartbeat_front_addr":"172.21.15.161:6805/664224744","state":["exists","up"]},{"osd":2,"uuid":"7562ec4c-c126-48ec-a51c-56fa461a2e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6801","nonce":345172073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6803","nonce":345172073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6807","nonce":345172073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6805","nonce":345172073}]},"public_addr":"172.21.15.171:6801/345172073","cluster_addr":"172.21.15.171:6803/345172073","heartbeat_back_addr":"172.21.15.171:6807/345172073","heartbeat_front_addr":"172.21.15.171:6805/345172073","state":["exists","up"]},{"osd":3,"uuid":"d7aae8cd-fec7-46d9-8773-25eaaca3cc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6811","nonce":1472407156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6813","nonce":1472407156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6817","nonce":1472407156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6815","nonce":1472407156}]},"public_addr":"172.21.15.112:6811/1472407156","cluster_addr":"172.21.15.112:6813/1472407156","heartbeat_back_addr":"172.21.15.112:6817/1472407156","heartbeat_front_addr":"172.21.15.112:6815/1472407156","state":["exists","up"]},{"osd":4,"uuid":"b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6809","nonce":373204306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6811","nonce":373204306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6815","nonce":373204306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6813","nonce":373204306}]},"public_addr":"172.21.15.161:6809/373204306","cluster_addr":"172.21.15.161:6811/373204306","heartbeat_back_addr":"172.21.15.161:6815/373204306","heartbeat_front_addr":"172.21.15.161:6813/373204306","state":["exists","up"]},{"osd":5,"uuid":"56124d12-bc1f-4cba-8430-938248851776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6809","nonce":2643512313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6811","nonce":2643512313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6815","nonce":2643512313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6813","nonce":2643512313}]},"public_addr":"172.21.15.171:6809/2643512313","cluster_addr":"172.21.15.171:6811/2643512313","heartbeat_back_addr":"172.21.15.171:6815/2643512313","heartbeat_front_addr":"172.21.15.171:6813/2643512313","state":["exists","up"]},{"osd":6,"uuid":"9563a9d3-352b-4686-975c-282ec167248b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6819","nonce":1933654933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6821","nonce":1933654933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6825","nonce":1933654933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6823","nonce":1933654933}]},"public_addr":"172.21.15.112:6819/1933654933","cluster_addr":"172.21.15.112:6821/1933654933","heartbeat_back_addr":"172.21.15.112:6825/1933654933","heartbeat_front_addr":"172.21.15.112:6823/1933654933","state":["exists","up"]},{"osd":7,"uuid":"f2d06e86-6aac-40f4-956e-18fe0b0e3543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6817","nonce":3060201138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6819","nonce":3060201138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6823","nonce":3060201138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6821","nonce":3060201138}]},"public_addr":"172.21.15.161:6817/3060201138","cluster_addr":"172.21.15.161:6819/3060201138","heartbeat_back_addr":"172.21.15.161:6823/3060201138","heartbeat_front_addr":"172.21.15.161:6821/3060201138","state":["exists","up"]},{"osd":8,"uuid":"f006e45f-2bb9-45d0-9020-be7c33c6a825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6817","nonce":835756571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6819","nonce":835756571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6823","nonce":835756571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6821","nonce":835756571}]},"public_addr":"172.21.15.171:6817/835756571","cluster_addr":"172.21.15.171:6819/835756571","heartbeat_back_addr":"172.21.15.171:6823/835756571","heartbeat_front_addr":"172.21.15.171:6821/835756571","state":["exists","up"]},{"osd":9,"uuid":"c5a96491-4151-4633-b3bc-2c214fe51c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6827","nonce":349758528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6829","nonce":349758528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6833","nonce":349758528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6831","nonce":349758528}]},"public_addr":"172.21.15.112:6827/349758528","cluster_addr":"172.21.15.112:6829/349758528","heartbeat_back_addr":"172.21.15.112:6833/349758528","heartbeat_front_addr":"172.21.15.112:6831/349758528","state":["exists","up"]},{"osd":10,"uuid":"244eda3e-25a0-490e-879f-c1e2726c1be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6825","nonce":3013292765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6827","nonce":3013292765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6831","nonce":3013292765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6829","nonce":3013292765}]},"public_addr":"172.21.15.161:6825/3013292765","cluster_addr":"172.21.15.161:6827/3013292765","heartbeat_back_addr":"172.21.15.161:6831/3013292765","heartbeat_front_addr":"172.21.15.161:6829/3013292765","state":["exists","up"]},{"osd":11,"uuid":"356e5c54-bbc3-45ee-bda2-85164ce3cb92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6825","nonce":1010336458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6827","nonce":1010336458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6831","nonce":1010336458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6829","nonce":1010336458}]},"public_addr":"172.21.15.171:6825/1010336458","cluster_addr":"172.21.15.171:6827/1010336458","heartbeat_back_addr":"172.21.15.171:6831/1010336458","heartbeat_front_addr":"172.21.15.171:6829/1010336458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:36:51.024650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:17.026638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:41.239726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:15.148968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:41.029118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:06.225858+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:41.639444+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:07.753400+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:32.292085+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:07.159926+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:33.749782+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:58.594447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/181862246":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/3393454453":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3195850527":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6801/3468460974":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3392048544":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/1329430933":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/735578423":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/3468460974":"2024-04-26T18:34:20.605775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 -- 172.21.15.112:0/1744530679 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33480a3d50 msgr2=0x7f33480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33480a3d50 0x7f33480a6210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f336410fc60 tx=0x7f335001a040 comp rx=0 tx=0).stop 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 -- 172.21.15.112:0/1744530679 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3364102a90 msgr2=0x7f336419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3364102a90 0x7f336419b420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f335400e9f0 tx=0x7f335400ed00 comp rx=0 tx=0).stop 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 -- 172.21.15.112:0/1744530679 shutdown_connections 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33480a3d50 0x7f33480a6210 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3364108a50 0x7f336419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3364102a90 0x7f336419b420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 --2- 172.21.15.112:0/1744530679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364103450 0x7f336419b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 -- 172.21.15.112:0/1744530679 >> 172.21.15.112:0/1744530679 conn(0x7f3364079ba0 msgr2=0x7f3364101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 -- 172.21.15.112:0/1744530679 shutdown_connections 2024-04-25T18:44:39.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:39.513+0000 7f336ac00700 1 -- 172.21.15.112:0/1744530679 wait complete. 2024-04-25T18:44:40.367 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T18:44:40.368 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T18:44:40.368 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T18:44:40.369 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T18:44:40.369 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T18:44:40.369 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T18:44:40.370 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T18:44:40.370 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T18:44:40.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T18:44:40.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T18:44:40.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T18:44:40.372 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T18:44:40.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:40 smithi112 ceph-mon[27864]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:40.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:40 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1744530679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:44:40.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:40 smithi171 ceph-mon[33266]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:40.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:40 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1744530679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:44:40.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:40 smithi161 ceph-mon[32609]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:40.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:40 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1744530679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:44:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:41.447 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:41.461 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:41.549 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:41.591 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:41.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:41 smithi112 ceph-mon[27864]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:41.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:41 smithi171 ceph-mon[33266]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:41.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:41 smithi161 ceph-mon[32609]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:41.733 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:41.738 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:42.531 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:44:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:44 smithi161 ceph-mon[32609]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:44.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:44 smithi112 ceph-mon[27864]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:44.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:44 smithi171 ceph-mon[33266]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:46.422 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:46 smithi112 ceph-mon[27864]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:46.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:46 smithi161 ceph-mon[32609]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:46.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:46 smithi171 ceph-mon[33266]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:48.464 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:48 smithi112 ceph-mon[27864]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:44:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:48 smithi161 ceph-mon[32609]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:44:48.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:48 smithi171 ceph-mon[33266]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:44:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:50 smithi161 ceph-mon[32609]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:50 smithi112 ceph-mon[27864]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:50.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:50 smithi171 ceph-mon[33266]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:52.464 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:52 smithi112 ceph-mon[27864]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:52 smithi161 ceph-mon[32609]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:52.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:52 smithi171 ceph-mon[33266]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:54 smithi161 ceph-mon[32609]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:54 smithi112 ceph-mon[27864]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:54.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:54 smithi171 ceph-mon[33266]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:44:55.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.629+0000 7fb85d600700 1 -- 172.21.15.112:0/157855631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb858105890 msgr2=0x7fb8581098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.629+0000 7fb85d600700 1 --2- 172.21.15.112:0/157855631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb858105890 0x7fb8581098e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb8400099c0 tx=0x7fb840009cd0 comp rx=0 tx=0).stop 2024-04-25T18:44:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.629+0000 7fb85d600700 1 -- 172.21.15.112:0/157855631 shutdown_connections 2024-04-25T18:44:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.629+0000 7fb85d600700 1 --2- 172.21.15.112:0/157855631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb858109fc0 0x7fb858111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.629+0000 7fb85d600700 1 --2- 172.21.15.112:0/157855631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb858104ee0 0x7fb8581052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.629+0000 7fb85d600700 1 --2- 172.21.15.112:0/157855631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb858105890 0x7fb8581098e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.629+0000 7fb85d600700 1 -- 172.21.15.112:0/157855631 >> 172.21.15.112:0/157855631 conn(0x7fb858100770 msgr2=0x7fb858102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.630+0000 7fb85d600700 1 -- 172.21.15.112:0/157855631 shutdown_connections 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.631+0000 7fb85d600700 1 -- 172.21.15.112:0/157855631 wait complete. 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.632+0000 7fb85d600700 1 Processor -- start 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.632+0000 7fb85d600700 1 -- start start 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.632+0000 7fb85d600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb858104ee0 0x7fb85819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.632+0000 7fb85d600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb858105890 0x7fb85819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.632+0000 7fb85d600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb858109fc0 0x7fb85819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.632+0000 7fb85d600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb858114e50 con 0x7fb858104ee0 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.632+0000 7fb85d600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb858114cd0 con 0x7fb858105890 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.632+0000 7fb85d600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb858114fd0 con 0x7fb858109fc0 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb856a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb858104ee0 0x7fb85819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb84fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb858105890 0x7fb85819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb856a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb858104ee0 0x7fb85819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51048/0 (socket says 172.21.15.112:51048) 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb856a00700 1 -- 172.21.15.112:0/834132776 learned_addr learned my addr 172.21.15.112:0/834132776 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb857400700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb858109fc0 0x7fb85819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb856a00700 1 -- 172.21.15.112:0/834132776 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb858109fc0 msgr2=0x7fb85819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb856a00700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb858109fc0 0x7fb85819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb856a00700 1 -- 172.21.15.112:0/834132776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb858105890 msgr2=0x7fb85819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb856a00700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb858105890 0x7fb85819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb856a00700 1 -- 172.21.15.112:0/834132776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8581a0340 con 0x7fb858104ee0 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb84fe00700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb858105890 0x7fb85819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb857400700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb858109fc0 0x7fb85819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.633+0000 7fb856a00700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb858104ee0 0x7fb85819b380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb84800e9a0 tx=0x7fb84800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.634+0000 7fb84f400700 1 -- 172.21.15.112:0/834132776 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84800ca10 con 0x7fb858104ee0 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.634+0000 7fb84f400700 1 -- 172.21.15.112:0/834132776 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb84800cbb0 con 0x7fb858104ee0 2024-04-25T18:44:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.634+0000 7fb85d600700 1 -- 172.21.15.112:0/834132776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8581a0630 con 0x7fb858104ee0 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.634+0000 7fb84f400700 1 -- 172.21.15.112:0/834132776 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848018680 con 0x7fb858104ee0 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.634+0000 7fb85d600700 1 -- 172.21.15.112:0/834132776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8581a7eb0 con 0x7fb858104ee0 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.635+0000 7fb85d600700 1 -- 172.21.15.112:0/834132776 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb858063190 con 0x7fb858104ee0 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.637+0000 7fb84f400700 1 -- 172.21.15.112:0/834132776 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb848010430 con 0x7fb858104ee0 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.639+0000 7fb84f400700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fb83c0a3d50 0x7fb83c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.639+0000 7fb84f400700 1 -- 172.21.15.112:0/834132776 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fb8480d0ca0 con 0x7fb858104ee0 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.639+0000 7fb84fe00700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fb83c0a3d50 0x7fb83c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.639+0000 7fb84f400700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458] conn(0x7fb83c0ad990 0x7fb83c0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.639+0000 7fb84f400700 1 -- 172.21.15.112:0/834132776 --> [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb848016ec0 con 0x7fb83c0ad990 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.639+0000 7fb84f400700 1 -- 172.21.15.112:0/834132776 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fb8480d1050 con 0x7fb858104ee0 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.639+0000 7fb857400700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458] conn(0x7fb83c0ad990 0x7fb83c0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.640+0000 7fb84fe00700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fb83c0a3d50 0x7fb83c0a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb840000c00 tx=0x7fb840005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.640+0000 7fb857400700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458] conn(0x7fb83c0ad990 0x7fb83c0afe30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.641+0000 7fb84f400700 1 -- 172.21.15.112:0/834132776 <== osd.11 v2:172.21.15.171:6824/1010336458 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb848016ec0 con 0x7fb83c0ad990 2024-04-25T18:44:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.659+0000 7fb85d600700 1 -- 172.21.15.112:0/834132776 --> [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb858063610 con 0x7fb83c0ad990 2024-04-25T18:44:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.659+0000 7fb84f400700 1 -- 172.21.15.112:0/834132776 <== osd.11 v2:172.21.15.171:6824/1010336458 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb858063610 con 0x7fb83c0ad990 2024-04-25T18:44:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.660+0000 7fb85d600700 1 -- 172.21.15.112:0/834132776 >> [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458] conn(0x7fb83c0ad990 msgr2=0x7fb83c0afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.660+0000 7fb85d600700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458] conn(0x7fb83c0ad990 0x7fb83c0afe30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.660+0000 7fb85d600700 1 -- 172.21.15.112:0/834132776 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fb83c0a3d50 msgr2=0x7fb83c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.660+0000 7fb85d600700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fb83c0a3d50 0x7fb83c0a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb840000c00 tx=0x7fb840005fb0 comp rx=0 tx=0).stop 2024-04-25T18:44:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.660+0000 7fb85d600700 1 -- 172.21.15.112:0/834132776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb858104ee0 msgr2=0x7fb85819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.660+0000 7fb85d600700 1 --2- 172.21.15.112:0/834132776 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb858104ee0 0x7fb85819b380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb84800e9a0 tx=0x7fb84800ecb0 comp rx=0 tx=0).stop 2024-04-25T18:44:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.660+0000 7fb857400700 1 -- 172.21.15.112:0/834132776 reap_dead start 2024-04-25T18:44:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.660+0000 7fb85d600700 1 -- 172.21.15.112:0/834132776 shutdown_connections 2024-04-25T18:44:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.660+0000 7fb85d600700 1 -- 172.21.15.112:0/834132776 >> 172.21.15.112:0/834132776 conn(0x7fb858100770 msgr2=0x7fb858102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.660+0000 7fb85d600700 1 -- 172.21.15.112:0/834132776 shutdown_connections 2024-04-25T18:44:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:53.660+0000 7fb85d600700 1 -- 172.21.15.112:0/834132776 wait complete. 2024-04-25T18:44:56.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:56 smithi161 ceph-mon[32609]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:56.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:56 smithi112 ceph-mon[27864]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:56.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:56 smithi171 ceph-mon[33266]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:44:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.862+0000 7f33e9200700 1 -- 172.21.15.112:0/598488867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33e4075590 msgr2=0x7f33e410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.862+0000 7f33e9200700 1 --2- 172.21.15.112:0/598488867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33e4075590 0x7f33e410ef00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009990 tx=0x7f33cc009ca0 comp rx=0 tx=0).stop 2024-04-25T18:44:57.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.862+0000 7f33e9200700 1 -- 172.21.15.112:0/598488867 shutdown_connections 2024-04-25T18:44:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.862+0000 7f33e9200700 1 --2- 172.21.15.112:0/598488867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33e410f440 0x7f33e4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.862+0000 7f33e9200700 1 --2- 172.21.15.112:0/598488867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e4074c70 0x7f33e4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.862+0000 7f33e9200700 1 --2- 172.21.15.112:0/598488867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33e4075590 0x7f33e410ef00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.862+0000 7f33e9200700 1 -- 172.21.15.112:0/598488867 >> 172.21.15.112:0/598488867 conn(0x7f33e4079ba0 msgr2=0x7f33e4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.862+0000 7f33e9200700 1 -- 172.21.15.112:0/598488867 shutdown_connections 2024-04-25T18:44:57.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.862+0000 7f33e9200700 1 -- 172.21.15.112:0/598488867 wait complete. 2024-04-25T18:44:57.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.863+0000 7f33e9200700 1 Processor -- start 2024-04-25T18:44:57.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33e9200700 1 -- start start 2024-04-25T18:44:57.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33e9200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33e4074c70 0x7f33e419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:57.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33e9200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33e4075590 0x7f33e419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:57.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33e9200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e410f440 0x7f33e419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33e9200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4114e70 con 0x7f33e4074c70 2024-04-25T18:44:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33e9200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4114cf0 con 0x7f33e410f440 2024-04-25T18:44:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33e9200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4114ff0 con 0x7f33e4075590 2024-04-25T18:44:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33e1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33e4074c70 0x7f33e419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33dbe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33e4075590 0x7f33e419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33e2800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e410f440 0x7f33e419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33e1e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33e4074c70 0x7f33e419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51100/0 (socket says 172.21.15.112:51100) 2024-04-25T18:44:57.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.864+0000 7f33e1e00700 1 -- 172.21.15.112:0/814384847 learned_addr learned my addr 172.21.15.112:0/814384847 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.865+0000 7f33dbe00700 1 -- 172.21.15.112:0/814384847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e410f440 msgr2=0x7f33e419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.865+0000 7f33dbe00700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33e410f440 0x7f33e419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.865+0000 7f33dbe00700 1 -- 172.21.15.112:0/814384847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33e4074c70 msgr2=0x7f33e419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.865+0000 7f33dbe00700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33e4074c70 0x7f33e419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.865+0000 7f33dbe00700 1 -- 172.21.15.112:0/814384847 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e41a0370 con 0x7f33e4075590 2024-04-25T18:44:57.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.865+0000 7f33dbe00700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33e4075590 0x7f33e419b8a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f33cc000c00 tx=0x7f33cc0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:57.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.866+0000 7f33daa00700 1 -- 172.21.15.112:0/814384847 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc026070 con 0x7f33e4075590 2024-04-25T18:44:57.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.866+0000 7f33daa00700 1 -- 172.21.15.112:0/814384847 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33cc021410 con 0x7f33e4075590 2024-04-25T18:44:57.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.866+0000 7f33daa00700 1 -- 172.21.15.112:0/814384847 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc017650 con 0x7f33e4075590 2024-04-25T18:44:57.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.866+0000 7f33e9200700 1 -- 172.21.15.112:0/814384847 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e41a0600 con 0x7f33e4075590 2024-04-25T18:44:57.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.866+0000 7f33e9200700 1 -- 172.21.15.112:0/814384847 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41a7ee0 con 0x7f33e4075590 2024-04-25T18:44:57.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.867+0000 7f33e9200700 1 -- 172.21.15.112:0/814384847 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33e41a0a90 con 0x7f33e4075590 2024-04-25T18:44:57.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.869+0000 7f33daa00700 1 -- 172.21.15.112:0/814384847 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f33cc029070 con 0x7f33e4075590 2024-04-25T18:44:57.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.871+0000 7f33daa00700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33c80a3d50 0x7f33c80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.871+0000 7f33daa00700 1 -- 172.21.15.112:0/814384847 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f33cc0d1c40 con 0x7f33e4075590 2024-04-25T18:44:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.871+0000 7f33e1e00700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33c80a3d50 0x7f33c80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.871+0000 7f33daa00700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138] conn(0x7f33c80ad990 0x7f33c80afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.871+0000 7f33daa00700 1 -- 172.21.15.112:0/814384847 --> [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f33c80b0500 con 0x7f33c80ad990 2024-04-25T18:44:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.871+0000 7f33daa00700 1 -- 172.21.15.112:0/814384847 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f33cc0d1ff0 con 0x7f33e4075590 2024-04-25T18:44:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.871+0000 7f33e2800700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138] conn(0x7f33c80ad990 0x7f33c80afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.872+0000 7f33e1e00700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33c80a3d50 0x7f33c80a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009d20 tx=0x7f33d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.872+0000 7f33e2800700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138] conn(0x7f33c80ad990 0x7f33c80afe30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.873+0000 7f33daa00700 1 -- 172.21.15.112:0/814384847 <== osd.7 v2:172.21.15.161:6816/3060201138 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f33c80b0500 con 0x7f33c80ad990 2024-04-25T18:44:57.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.891+0000 7f33e9200700 1 -- 172.21.15.112:0/814384847 --> [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f33e40634b0 con 0x7f33c80ad990 2024-04-25T18:44:57.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.891+0000 7f33daa00700 1 -- 172.21.15.112:0/814384847 <== osd.7 v2:172.21.15.161:6816/3060201138 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f33e40634b0 con 0x7f33c80ad990 2024-04-25T18:44:57.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.892+0000 7f33e9200700 1 -- 172.21.15.112:0/814384847 >> [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138] conn(0x7f33c80ad990 msgr2=0x7f33c80afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:57.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.892+0000 7f33e9200700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138] conn(0x7f33c80ad990 0x7f33c80afe30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:57.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.892+0000 7f33e9200700 1 -- 172.21.15.112:0/814384847 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33c80a3d50 msgr2=0x7f33c80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.892+0000 7f33e9200700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33c80a3d50 0x7f33c80a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009d20 tx=0x7f33d4009260 comp rx=0 tx=0).stop 2024-04-25T18:44:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.892+0000 7f33e9200700 1 -- 172.21.15.112:0/814384847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33e4075590 msgr2=0x7f33e419b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.892+0000 7f33e9200700 1 --2- 172.21.15.112:0/814384847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33e4075590 0x7f33e419b8a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f33cc000c00 tx=0x7f33cc0047b0 comp rx=0 tx=0).stop 2024-04-25T18:44:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.892+0000 7f33e2800700 1 -- 172.21.15.112:0/814384847 reap_dead start 2024-04-25T18:44:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.893+0000 7f33e9200700 1 -- 172.21.15.112:0/814384847 shutdown_connections 2024-04-25T18:44:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.893+0000 7f33e9200700 1 -- 172.21.15.112:0/814384847 >> 172.21.15.112:0/814384847 conn(0x7f33e4079ba0 msgr2=0x7f33e40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.893+0000 7f33e9200700 1 -- 172.21.15.112:0/814384847 shutdown_connections 2024-04-25T18:44:57.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:57.893+0000 7f33e9200700 1 -- 172.21.15.112:0/814384847 wait complete. 2024-04-25T18:44:58.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:44:58 smithi161 ceph-mon[32609]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:44:58.477 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:44:58 smithi112 ceph-mon[27864]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:44:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:44:58 smithi171 ceph-mon[33266]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:44:58.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.793+0000 7fa450200700 1 -- 172.21.15.112:0/3829017662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa44806b070 msgr2=0x7fa44810efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:58.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.793+0000 7fa450200700 1 --2- 172.21.15.112:0/3829017662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa44806b070 0x7fa44810efb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa434009990 tx=0x7fa434009ca0 comp rx=0 tx=0).stop 2024-04-25T18:44:58.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.793+0000 7fa450200700 1 -- 172.21.15.112:0/3829017662 shutdown_connections 2024-04-25T18:44:58.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.793+0000 7fa450200700 1 --2- 172.21.15.112:0/3829017662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44810f4f0 0x7fa4481118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:58.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.793+0000 7fa450200700 1 --2- 172.21.15.112:0/3829017662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa44806a750 0x7fa44806ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:58.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.793+0000 7fa450200700 1 --2- 172.21.15.112:0/3829017662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa44806b070 0x7fa44810efb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:58.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.793+0000 7fa450200700 1 -- 172.21.15.112:0/3829017662 >> 172.21.15.112:0/3829017662 conn(0x7fa4480788f0 msgr2=0x7fa448078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:58.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.793+0000 7fa450200700 1 -- 172.21.15.112:0/3829017662 shutdown_connections 2024-04-25T18:44:58.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.793+0000 7fa450200700 1 -- 172.21.15.112:0/3829017662 wait complete. 2024-04-25T18:44:58.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.794+0000 7fa450200700 1 Processor -- start 2024-04-25T18:44:58.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.794+0000 7fa450200700 1 -- start start 2024-04-25T18:44:58.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa450200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa44806a750 0x7fa44819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa450200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44806b070 0x7fa44819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa450200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa44810f4f0 0x7fa44819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa450200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa448114e00 con 0x7fa44810f4f0 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa450200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa448114c80 con 0x7fa44806a750 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa450200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa448114f80 con 0x7fa44806b070 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa44e200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa44810f4f0 0x7fa44819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa44e200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa44810f4f0 0x7fa44819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51128/0 (socket says 172.21.15.112:51128) 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa44ce00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44806b070 0x7fa44819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa44e200700 1 -- 172.21.15.112:0/2269401585 learned_addr learned my addr 172.21.15.112:0/2269401585 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa44d800700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa44806a750 0x7fa44819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa44ce00700 1 -- 172.21.15.112:0/2269401585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa44806a750 msgr2=0x7fa44819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:58.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa44ce00700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa44806a750 0x7fa44819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:58.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa44ce00700 1 -- 172.21.15.112:0/2269401585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa44810f4f0 msgr2=0x7fa44819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:58.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa44ce00700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa44810f4f0 0x7fa44819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:58.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa44ce00700 1 -- 172.21.15.112:0/2269401585 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4481a0320 con 0x7fa44806b070 2024-04-25T18:44:58.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.795+0000 7fa44d800700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa44806a750 0x7fa44819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:44:58.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.796+0000 7fa44ce00700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44806b070 0x7fa44819b850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa4340059d0 tx=0x7fa4340047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:58.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.796+0000 7fa446000700 1 -- 172.21.15.112:0/2269401585 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434027070 con 0x7fa44806b070 2024-04-25T18:44:58.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.796+0000 7fa446000700 1 -- 172.21.15.112:0/2269401585 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa434022470 con 0x7fa44806b070 2024-04-25T18:44:58.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.796+0000 7fa446000700 1 -- 172.21.15.112:0/2269401585 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434017650 con 0x7fa44806b070 2024-04-25T18:44:58.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.796+0000 7fa450200700 1 -- 172.21.15.112:0/2269401585 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4481a05b0 con 0x7fa44806b070 2024-04-25T18:44:58.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.796+0000 7fa450200700 1 -- 172.21.15.112:0/2269401585 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4481a7e90 con 0x7fa44806b070 2024-04-25T18:44:58.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.797+0000 7fa450200700 1 -- 172.21.15.112:0/2269401585 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa4480631d0 con 0x7fa44806b070 2024-04-25T18:44:58.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.799+0000 7fa446000700 1 -- 172.21.15.112:0/2269401585 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa43402a070 con 0x7fa44806b070 2024-04-25T18:44:58.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.801+0000 7fa446000700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4300a3d50 0x7fa4300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:58.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.801+0000 7fa446000700 1 -- 172.21.15.112:0/2269401585 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa4340d1a20 con 0x7fa44806b070 2024-04-25T18:44:58.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.801+0000 7fa44d800700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4300a3d50 0x7fa4300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:58.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.801+0000 7fa446000700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744] conn(0x7fa4300ad990 0x7fa4300afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:58.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.801+0000 7fa446000700 1 -- 172.21.15.112:0/2269401585 --> [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa434018e50 con 0x7fa4300ad990 2024-04-25T18:44:58.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.801+0000 7fa446000700 1 -- 172.21.15.112:0/2269401585 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fa4340d1dd0 con 0x7fa44806b070 2024-04-25T18:44:58.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.801+0000 7fa44e200700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744] conn(0x7fa4300ad990 0x7fa4300afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:58.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.802+0000 7fa44d800700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4300a3d50 0x7fa4300a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009480 tx=0x7fa43c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:58.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.802+0000 7fa44e200700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744] conn(0x7fa4300ad990 0x7fa4300afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:58.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.803+0000 7fa446000700 1 -- 172.21.15.112:0/2269401585 <== osd.1 v2:172.21.15.161:6800/664224744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa434018e50 con 0x7fa4300ad990 2024-04-25T18:44:58.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.820+0000 7fa450200700 1 -- 172.21.15.112:0/2269401585 --> [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa448063650 con 0x7fa4300ad990 2024-04-25T18:44:58.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.821+0000 7fa446000700 1 -- 172.21.15.112:0/2269401585 <== osd.1 v2:172.21.15.161:6800/664224744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa448063650 con 0x7fa4300ad990 2024-04-25T18:44:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.822+0000 7fa450200700 1 -- 172.21.15.112:0/2269401585 >> [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744] conn(0x7fa4300ad990 msgr2=0x7fa4300afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.822+0000 7fa450200700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744] conn(0x7fa4300ad990 0x7fa4300afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.822+0000 7fa450200700 1 -- 172.21.15.112:0/2269401585 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4300a3d50 msgr2=0x7fa4300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.822+0000 7fa450200700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4300a3d50 0x7fa4300a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009480 tx=0x7fa43c00b040 comp rx=0 tx=0).stop 2024-04-25T18:44:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.822+0000 7fa450200700 1 -- 172.21.15.112:0/2269401585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44806b070 msgr2=0x7fa44819b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.822+0000 7fa450200700 1 --2- 172.21.15.112:0/2269401585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44806b070 0x7fa44819b850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa4340059d0 tx=0x7fa4340047b0 comp rx=0 tx=0).stop 2024-04-25T18:44:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.822+0000 7fa44e200700 1 -- 172.21.15.112:0/2269401585 reap_dead start 2024-04-25T18:44:58.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.822+0000 7fa450200700 1 -- 172.21.15.112:0/2269401585 shutdown_connections 2024-04-25T18:44:58.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.822+0000 7fa450200700 1 -- 172.21.15.112:0/2269401585 >> 172.21.15.112:0/2269401585 conn(0x7fa4480788f0 msgr2=0x7fa4481109c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:58.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.822+0000 7fa450200700 1 -- 172.21.15.112:0/2269401585 shutdown_connections 2024-04-25T18:44:58.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.822+0000 7fa450200700 1 -- 172.21.15.112:0/2269401585 wait complete. 2024-04-25T18:44:59.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.234+0000 7f95efc00700 1 -- 172.21.15.112:0/2972730858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95e81092d0 msgr2=0x7f95e81096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:59.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.234+0000 7f95efc00700 1 --2- 172.21.15.112:0/2972730858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95e81092d0 0x7f95e81096b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00b0d0 tx=0x7f95dc00b3e0 comp rx=0 tx=0).stop 2024-04-25T18:44:59.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.234+0000 7f95efc00700 1 -- 172.21.15.112:0/2972730858 shutdown_connections 2024-04-25T18:44:59.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.234+0000 7f95efc00700 1 --2- 172.21.15.112:0/2972730858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95e810e3b0 0x7f95e8115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:59.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.234+0000 7f95efc00700 1 --2- 172.21.15.112:0/2972730858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95e81092d0 0x7f95e81096b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:59.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.234+0000 7f95efc00700 1 --2- 172.21.15.112:0/2972730858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95e8109c80 0x7f95e810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:59.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.234+0000 7f95efc00700 1 -- 172.21.15.112:0/2972730858 >> 172.21.15.112:0/2972730858 conn(0x7f95e8076210 msgr2=0x7f95e8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:59.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.235+0000 7f95efc00700 1 -- 172.21.15.112:0/2972730858 shutdown_connections 2024-04-25T18:44:59.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.235+0000 7f95efc00700 1 -- 172.21.15.112:0/2972730858 wait complete. 2024-04-25T18:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.236+0000 7f95efc00700 1 Processor -- start 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.236+0000 7f95efc00700 1 -- start start 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.236+0000 7f95efc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95e81092d0 0x7f95e81a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.236+0000 7f95efc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95e8109c80 0x7f95e81a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.236+0000 7f95efc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95e810e3b0 0x7f95e819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.236+0000 7f95efc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81192b0 con 0x7f95e810e3b0 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.236+0000 7f95efc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f95e8119130 con 0x7f95e81092d0 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.236+0000 7f95efc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f95e8119430 con 0x7f95e8109c80 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95edc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95e810e3b0 0x7f95e819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95e7e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95e8109c80 0x7f95e81a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95ed200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95e81092d0 0x7f95e81a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95ed200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95e81092d0 0x7f95e81a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:57490/0 (socket says 172.21.15.112:57490) 2024-04-25T18:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95ed200700 1 -- 172.21.15.112:0/3154589741 learned_addr learned my addr 172.21.15.112:0/3154589741 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95edc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95e810e3b0 0x7f95e819f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51148/0 (socket says 172.21.15.112:51148) 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95e7e00700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95e8109c80 0x7f95e81a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:46476/0 (socket says 172.21.15.112:46476) 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95ed200700 1 -- 172.21.15.112:0/3154589741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95e8109c80 msgr2=0x7f95e81a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95ed200700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95e8109c80 0x7f95e81a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95ed200700 1 -- 172.21.15.112:0/3154589741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95e810e3b0 msgr2=0x7f95e819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95ed200700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95e810e3b0 0x7f95e819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95ed200700 1 -- 172.21.15.112:0/3154589741 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e819fd50 con 0x7f95e81092d0 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.237+0000 7f95ed200700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95e81092d0 0x7f95e81a53e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95dc007a20 tx=0x7f95dc007a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.238+0000 7f95e5600700 1 -- 172.21.15.112:0/3154589741 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc01f070 con 0x7f95e81092d0 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.238+0000 7f95e5600700 1 -- 172.21.15.112:0/3154589741 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95dc004c50 con 0x7f95e81092d0 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.238+0000 7f95efc00700 1 -- 172.21.15.112:0/3154589741 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e819ffe0 con 0x7f95e81092d0 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.238+0000 7f95e5600700 1 -- 172.21.15.112:0/3154589741 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc01b690 con 0x7f95e81092d0 2024-04-25T18:44:59.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.238+0000 7f95efc00700 1 -- 172.21.15.112:0/3154589741 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e81ac2b0 con 0x7f95e81092d0 2024-04-25T18:44:59.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.239+0000 7f95efc00700 1 -- 172.21.15.112:0/3154589741 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f95e80631d0 con 0x7f95e81092d0 2024-04-25T18:44:59.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.241+0000 7f95e5600700 1 -- 172.21.15.112:0/3154589741 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f95dc022070 con 0x7f95e81092d0 2024-04-25T18:44:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.242+0000 7f95e5600700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f95d00a3d50 0x7f95d00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.242+0000 7f95e5600700 1 -- 172.21.15.112:0/3154589741 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f95dc00f020 con 0x7f95e81092d0 2024-04-25T18:44:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.242+0000 7f95e7e00700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f95d00a3d50 0x7f95d00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.243+0000 7f95e5600700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073] conn(0x7f95d00ad990 0x7f95d00afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:44:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.243+0000 7f95e5600700 1 -- 172.21.15.112:0/3154589741 --> [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95d00b0500 con 0x7f95d00ad990 2024-04-25T18:44:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.243+0000 7f95e5600700 1 -- 172.21.15.112:0/3154589741 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f95dc0d1bb0 con 0x7f95e81092d0 2024-04-25T18:44:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.243+0000 7f95edc00700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073] conn(0x7f95d00ad990 0x7f95d00afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:44:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.243+0000 7f95e7e00700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f95d00a3d50 0x7f95d00a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f95e8108910 tx=0x7f95d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:59.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.244+0000 7f95edc00700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073] conn(0x7f95d00ad990 0x7f95d00afe30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:44:59.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.245+0000 7f95e5600700 1 -- 172.21.15.112:0/3154589741 <== osd.2 v2:172.21.15.171:6800/345172073 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f95d00b0500 con 0x7f95d00ad990 2024-04-25T18:44:59.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.263+0000 7f95efc00700 1 -- 172.21.15.112:0/3154589741 --> [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f95e80635d0 con 0x7f95d00ad990 2024-04-25T18:44:59.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.263+0000 7f95e5600700 1 -- 172.21.15.112:0/3154589741 <== osd.2 v2:172.21.15.171:6800/345172073 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f95e80635d0 con 0x7f95d00ad990 2024-04-25T18:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.264+0000 7f95efc00700 1 -- 172.21.15.112:0/3154589741 >> [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073] conn(0x7f95d00ad990 msgr2=0x7f95d00afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.264+0000 7f95efc00700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073] conn(0x7f95d00ad990 0x7f95d00afe30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.264+0000 7f95efc00700 1 -- 172.21.15.112:0/3154589741 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f95d00a3d50 msgr2=0x7f95d00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.264+0000 7f95efc00700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f95d00a3d50 0x7f95d00a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f95e8108910 tx=0x7f95d4009260 comp rx=0 tx=0).stop 2024-04-25T18:44:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.264+0000 7f95efc00700 1 -- 172.21.15.112:0/3154589741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95e81092d0 msgr2=0x7f95e81a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.264+0000 7f95efc00700 1 --2- 172.21.15.112:0/3154589741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95e81092d0 0x7f95e81a53e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95dc007a20 tx=0x7f95dc007a50 comp rx=0 tx=0).stop 2024-04-25T18:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.264+0000 7f95edc00700 1 -- 172.21.15.112:0/3154589741 reap_dead start 2024-04-25T18:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.264+0000 7f95efc00700 1 -- 172.21.15.112:0/3154589741 shutdown_connections 2024-04-25T18:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.264+0000 7f95efc00700 1 -- 172.21.15.112:0/3154589741 >> 172.21.15.112:0/3154589741 conn(0x7f95e8076210 msgr2=0x7f95e8077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.264+0000 7f95efc00700 1 -- 172.21.15.112:0/3154589741 shutdown_connections 2024-04-25T18:44:59.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:59.265+0000 7f95efc00700 1 -- 172.21.15.112:0/3154589741 wait complete. 2024-04-25T18:45:00.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:00 smithi112 ceph-mon[27864]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:00.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.135+0000 7fade2a00700 1 -- 172.21.15.112:0/1978471397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faddc109c80 msgr2=0x7faddc10dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:00.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.135+0000 7fade2a00700 1 --2- 172.21.15.112:0/1978471397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faddc109c80 0x7faddc10dcd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fadc80099c0 tx=0x7fadc8009cd0 comp rx=0 tx=0).stop 2024-04-25T18:45:00.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.136+0000 7fade2a00700 1 -- 172.21.15.112:0/1978471397 shutdown_connections 2024-04-25T18:45:00.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.136+0000 7fade2a00700 1 --2- 172.21.15.112:0/1978471397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faddc10e3b0 0x7faddc115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:00.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.136+0000 7fade2a00700 1 --2- 172.21.15.112:0/1978471397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faddc1092d0 0x7faddc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:00.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.136+0000 7fade2a00700 1 --2- 172.21.15.112:0/1978471397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faddc109c80 0x7faddc10dcd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:00.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.136+0000 7fade2a00700 1 -- 172.21.15.112:0/1978471397 >> 172.21.15.112:0/1978471397 conn(0x7faddc076210 msgr2=0x7faddc107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:00.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.136+0000 7fade2a00700 1 -- 172.21.15.112:0/1978471397 shutdown_connections 2024-04-25T18:45:00.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.136+0000 7fade2a00700 1 -- 172.21.15.112:0/1978471397 wait complete. 2024-04-25T18:45:00.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.137+0000 7fade2a00700 1 Processor -- start 2024-04-25T18:45:00.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.137+0000 7fade2a00700 1 -- start start 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.138+0000 7fade2a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faddc1092d0 0x7faddc1a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.138+0000 7fade2a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faddc109c80 0x7faddc1a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.138+0000 7fade2a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faddc10e3b0 0x7faddc19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.138+0000 7fade2a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faddc119270 con 0x7faddc10e3b0 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.138+0000 7fade2a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1190f0 con 0x7faddc1092d0 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.138+0000 7fade2a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1193f0 con 0x7faddc109c80 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.138+0000 7fade0a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faddc10e3b0 0x7faddc19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.138+0000 7faddbe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faddc1092d0 0x7faddc1a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.138+0000 7fade0a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faddc10e3b0 0x7faddc19f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51060/0 (socket says 172.21.15.112:51060) 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.138+0000 7fade0a00700 1 -- 172.21.15.112:0/3876998231 learned_addr learned my addr 172.21.15.112:0/3876998231 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.138+0000 7faddb400700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faddc109c80 0x7faddc1a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.139+0000 7fade0a00700 1 -- 172.21.15.112:0/3876998231 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faddc109c80 msgr2=0x7faddc1a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.139+0000 7fade0a00700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faddc109c80 0x7faddc1a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.139+0000 7fade0a00700 1 -- 172.21.15.112:0/3876998231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faddc1092d0 msgr2=0x7faddc1a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:00.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.139+0000 7fade0a00700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faddc1092d0 0x7faddc1a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.139+0000 7fade0a00700 1 -- 172.21.15.112:0/3876998231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc19fd30 con 0x7faddc10e3b0 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.139+0000 7fade0a00700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faddc10e3b0 0x7faddc19f5d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fadd400ef30 tx=0x7fadd400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.139+0000 7fadd8c00700 1 -- 172.21.15.112:0/3876998231 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40092e0 con 0x7faddc10e3b0 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.139+0000 7fadd8c00700 1 -- 172.21.15.112:0/3876998231 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd4004bb0 con 0x7faddc10e3b0 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.139+0000 7fade2a00700 1 -- 172.21.15.112:0/3876998231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc19ffc0 con 0x7faddc10e3b0 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.139+0000 7fadd8c00700 1 -- 172.21.15.112:0/3876998231 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4004020 con 0x7faddc10e3b0 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.139+0000 7fade2a00700 1 -- 172.21.15.112:0/3876998231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1ac2e0 con 0x7faddc10e3b0 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.140+0000 7fade2a00700 1 -- 172.21.15.112:0/3876998231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faddc063150 con 0x7faddc10e3b0 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.142+0000 7fadd8c00700 1 -- 172.21.15.112:0/3876998231 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fadd401b030 con 0x7faddc10e3b0 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.144+0000 7fadd8c00700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fadc00a3cd0 0x7fadc00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.144+0000 7fadd8c00700 1 -- 172.21.15.112:0/3876998231 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fadd40d0970 con 0x7faddc10e3b0 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.144+0000 7faddbe00700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fadc00a3cd0 0x7fadc00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.144+0000 7fadd8c00700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535] conn(0x7fadc00ad910 0x7fadc00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:00.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.144+0000 7fadd8c00700 1 -- 172.21.15.112:0/3876998231 --> [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadc00b0480 con 0x7fadc00ad910 2024-04-25T18:45:00.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.144+0000 7fadd8c00700 1 -- 172.21.15.112:0/3876998231 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fadd40d0d20 con 0x7faddc10e3b0 2024-04-25T18:45:00.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.145+0000 7faddb400700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535] conn(0x7fadc00ad910 0x7fadc00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:00.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.145+0000 7faddbe00700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fadc00a3cd0 0x7fadc00a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009680 tx=0x7fadcc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:00.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.145+0000 7faddb400700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535] conn(0x7fadc00ad910 0x7fadc00afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:00.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.146+0000 7fadd8c00700 1 -- 172.21.15.112:0/3876998231 <== osd.0 v2:172.21.15.112:6802/232254535 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fadc00b0480 con 0x7fadc00ad910 2024-04-25T18:45:00.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.164+0000 7fade2a00700 1 -- 172.21.15.112:0/3876998231 --> [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faddc063550 con 0x7fadc00ad910 2024-04-25T18:45:00.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.164+0000 7fadd8c00700 1 -- 172.21.15.112:0/3876998231 <== osd.0 v2:172.21.15.112:6802/232254535 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faddc063550 con 0x7fadc00ad910 2024-04-25T18:45:00.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.165+0000 7fade2a00700 1 -- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535] conn(0x7fadc00ad910 msgr2=0x7fadc00afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:00.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.165+0000 7fade2a00700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535] conn(0x7fadc00ad910 0x7fadc00afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:00.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.165+0000 7fade2a00700 1 -- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fadc00a3cd0 msgr2=0x7fadc00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:00.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.165+0000 7fade2a00700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fadc00a3cd0 0x7fadc00a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009680 tx=0x7fadcc009260 comp rx=0 tx=0).stop 2024-04-25T18:45:00.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.165+0000 7fade2a00700 1 -- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faddc10e3b0 msgr2=0x7faddc19f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:00.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.165+0000 7fade2a00700 1 --2- 172.21.15.112:0/3876998231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faddc10e3b0 0x7faddc19f5d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fadd400ef30 tx=0x7fadd400c3a0 comp rx=0 tx=0).stop 2024-04-25T18:45:00.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.165+0000 7fade0a00700 1 -- 172.21.15.112:0/3876998231 reap_dead start 2024-04-25T18:45:00.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.165+0000 7fade2a00700 1 -- 172.21.15.112:0/3876998231 shutdown_connections 2024-04-25T18:45:00.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.165+0000 7fade2a00700 1 -- 172.21.15.112:0/3876998231 >> 172.21.15.112:0/3876998231 conn(0x7faddc076210 msgr2=0x7faddc0768d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:00.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.165+0000 7fade2a00700 1 -- 172.21.15.112:0/3876998231 shutdown_connections 2024-04-25T18:45:00.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.165+0000 7fade2a00700 1 -- 172.21.15.112:0/3876998231 wait complete. 2024-04-25T18:45:00.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:00 smithi171 ceph-mon[33266]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:00.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:00 smithi161 ceph-mon[32609]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:01.204 INFO:teuthology.orchestra.run.smithi112.stdout:77309411418 2024-04-25T18:45:01.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T18:45:01.559 INFO:teuthology.orchestra.run.smithi112.stdout:55834574943 2024-04-25T18:45:01.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T18:45:01.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.837+0000 7fe64a000700 1 -- 172.21.15.112:0/3819580775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe644103450 msgr2=0x7fe64410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:01.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.837+0000 7fe64a000700 1 --2- 172.21.15.112:0/3819580775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe644103450 0x7fe64410f8b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009f90 tx=0x7fe63c00b330 comp rx=0 tx=0).stop 2024-04-25T18:45:01.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.837+0000 7fe64a000700 1 -- 172.21.15.112:0/3819580775 shutdown_connections 2024-04-25T18:45:01.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.837+0000 7fe64a000700 1 --2- 172.21.15.112:0/3819580775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe644103450 0x7fe64410f8b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:01.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.837+0000 7fe64a000700 1 --2- 172.21.15.112:0/3819580775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe644108a50 0x7fe644108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:01.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.837+0000 7fe64a000700 1 --2- 172.21.15.112:0/3819580775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe644102a90 0x7fe644102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:01.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.837+0000 7fe64a000700 1 -- 172.21.15.112:0/3819580775 >> 172.21.15.112:0/3819580775 conn(0x7fe644079ba0 msgr2=0x7fe644079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.837+0000 7fe64a000700 1 -- 172.21.15.112:0/3819580775 shutdown_connections 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.837+0000 7fe64a000700 1 -- 172.21.15.112:0/3819580775 wait complete. 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.838+0000 7fe64a000700 1 Processor -- start 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.839+0000 7fe64a000700 1 -- start start 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.839+0000 7fe64a000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe644102a90 0x7fe644074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.839+0000 7fe64a000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe644103450 0x7fe644075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.839+0000 7fe64a000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe644108a50 0x7fe6440757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.839+0000 7fe64a000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe644112c50 con 0x7fe644103450 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.839+0000 7fe64a000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe644112ad0 con 0x7fe644108a50 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.839+0000 7fe64a000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe644112dd0 con 0x7fe644102a90 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.839+0000 7fe643400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe644102a90 0x7fe644074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.839+0000 7fe643400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe644102a90 0x7fe644074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:46438/0 (socket says 172.21.15.112:46438) 2024-04-25T18:45:01.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.839+0000 7fe643400700 1 -- 172.21.15.112:0/1835223341 learned_addr learned my addr 172.21.15.112:0/1835223341 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.840+0000 7fe643e00700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe644108a50 0x7fe6440757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.840+0000 7fe642a00700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe644103450 0x7fe644075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.840+0000 7fe643400700 1 -- 172.21.15.112:0/1835223341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe644108a50 msgr2=0x7fe6440757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.840+0000 7fe643400700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe644108a50 0x7fe6440757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.840+0000 7fe643400700 1 -- 172.21.15.112:0/1835223341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe644103450 msgr2=0x7fe644075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.840+0000 7fe643400700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe644103450 0x7fe644075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.840+0000 7fe643400700 1 -- 172.21.15.112:0/1835223341 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6441a76f0 con 0x7fe644102a90 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.840+0000 7fe643e00700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe644108a50 0x7fe6440757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.840+0000 7fe643400700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe644102a90 0x7fe644074d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe63400e9c0 tx=0x7fe63400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.840+0000 7fe63be00700 1 -- 172.21.15.112:0/1835223341 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63400cbd0 con 0x7fe644102a90 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.841+0000 7fe63be00700 1 -- 172.21.15.112:0/1835223341 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe63400cd70 con 0x7fe644102a90 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.841+0000 7fe64a000700 1 -- 172.21.15.112:0/1835223341 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6441a79e0 con 0x7fe644102a90 2024-04-25T18:45:01.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.841+0000 7fe63be00700 1 -- 172.21.15.112:0/1835223341 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe634005610 con 0x7fe644102a90 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.841+0000 7fe64a000700 1 -- 172.21.15.112:0/1835223341 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6441a7ed0 con 0x7fe644102a90 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.842+0000 7fe64a000700 1 -- 172.21.15.112:0/1835223341 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe644063110 con 0x7fe644102a90 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.844+0000 7fe63be00700 1 -- 172.21.15.112:0/1835223341 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe634004750 con 0x7fe644102a90 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.845+0000 7fe63be00700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6280a3cd0 0x7fe6280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.845+0000 7fe63be00700 1 -- 172.21.15.112:0/1835223341 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fe6340d0ce0 con 0x7fe644102a90 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.845+0000 7fe642a00700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6280a3cd0 0x7fe6280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.846+0000 7fe63be00700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306] conn(0x7fe6280ad910 0x7fe6280afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.846+0000 7fe63be00700 1 -- 172.21.15.112:0/1835223341 --> [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe63401ee50 con 0x7fe6280ad910 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.846+0000 7fe63be00700 1 -- 172.21.15.112:0/1835223341 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fe6340d1090 con 0x7fe644102a90 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.846+0000 7fe643e00700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306] conn(0x7fe6280ad910 0x7fe6280afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.846+0000 7fe642a00700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6280a3cd0 0x7fe6280a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe630009750 tx=0x7fe630009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.847+0000 7fe643e00700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306] conn(0x7fe6280ad910 0x7fe6280afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:01.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.848+0000 7fe63be00700 1 -- 172.21.15.112:0/1835223341 <== osd.4 v2:172.21.15.161:6808/373204306 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe63401ee50 con 0x7fe6280ad910 2024-04-25T18:45:01.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.865+0000 7fe64a000700 1 -- 172.21.15.112:0/1835223341 --> [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe644063510 con 0x7fe6280ad910 2024-04-25T18:45:01.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.866+0000 7fe63be00700 1 -- 172.21.15.112:0/1835223341 <== osd.4 v2:172.21.15.161:6808/373204306 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe644063510 con 0x7fe6280ad910 2024-04-25T18:45:01.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.866+0000 7fe64a000700 1 -- 172.21.15.112:0/1835223341 >> [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306] conn(0x7fe6280ad910 msgr2=0x7fe6280afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:01.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.866+0000 7fe64a000700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306] conn(0x7fe6280ad910 0x7fe6280afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:01.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.866+0000 7fe64a000700 1 -- 172.21.15.112:0/1835223341 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6280a3cd0 msgr2=0x7fe6280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:01.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.866+0000 7fe64a000700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6280a3cd0 0x7fe6280a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe630009750 tx=0x7fe630009310 comp rx=0 tx=0).stop 2024-04-25T18:45:01.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.866+0000 7fe64a000700 1 -- 172.21.15.112:0/1835223341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe644102a90 msgr2=0x7fe644074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:01.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.866+0000 7fe64a000700 1 --2- 172.21.15.112:0/1835223341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe644102a90 0x7fe644074d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe63400e9c0 tx=0x7fe63400ecd0 comp rx=0 tx=0).stop 2024-04-25T18:45:01.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.866+0000 7fe643e00700 1 -- 172.21.15.112:0/1835223341 reap_dead start 2024-04-25T18:45:01.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.867+0000 7fe64a000700 1 -- 172.21.15.112:0/1835223341 shutdown_connections 2024-04-25T18:45:01.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.867+0000 7fe64a000700 1 -- 172.21.15.112:0/1835223341 >> 172.21.15.112:0/1835223341 conn(0x7fe644079ba0 msgr2=0x7fe6441018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:01.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.867+0000 7fe64a000700 1 -- 172.21.15.112:0/1835223341 shutdown_connections 2024-04-25T18:45:01.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:55.867+0000 7fe64a000700 1 -- 172.21.15.112:0/1835223341 wait complete. 2024-04-25T18:45:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.637+0000 7f340ba00700 1 -- 172.21.15.112:0/573273937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3404108a50 msgr2=0x7f3404108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.637+0000 7f340ba00700 1 --2- 172.21.15.112:0/573273937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3404108a50 0x7f3404108e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009990 tx=0x7f33f8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.637+0000 7f340ba00700 1 -- 172.21.15.112:0/573273937 shutdown_connections 2024-04-25T18:45:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.637+0000 7f340ba00700 1 --2- 172.21.15.112:0/573273937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3404103450 0x7f340410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.637+0000 7f340ba00700 1 --2- 172.21.15.112:0/573273937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3404108a50 0x7f3404108e30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.637+0000 7f340ba00700 1 --2- 172.21.15.112:0/573273937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3404102a90 0x7f3404102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.637+0000 7f340ba00700 1 -- 172.21.15.112:0/573273937 >> 172.21.15.112:0/573273937 conn(0x7f3404079ba0 msgr2=0x7f3404079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:02.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.637+0000 7f340ba00700 1 -- 172.21.15.112:0/573273937 shutdown_connections 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.637+0000 7f340ba00700 1 -- 172.21.15.112:0/573273937 wait complete. 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.639+0000 7f340ba00700 1 Processor -- start 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.639+0000 7f340ba00700 1 -- start start 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.639+0000 7f340ba00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3404102a90 0x7f3404199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.639+0000 7f340ba00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3404103450 0x7f34041996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.639+0000 7f340ba00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3404108a50 0x7f340419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f340ba00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3404112c30 con 0x7f3404103450 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f340ba00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3404112ab0 con 0x7f3404108a50 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f340ba00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3404112db0 con 0x7f3404102a90 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f3409000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3404102a90 0x7f3404199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f3403e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3404103450 0x7f34041996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f3409000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3404102a90 0x7f3404199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:46458/0 (socket says 172.21.15.112:46458) 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f3409000700 1 -- 172.21.15.112:0/1791949962 learned_addr learned my addr 172.21.15.112:0/1791949962 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f3409a00700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3404108a50 0x7f340419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f3409000700 1 -- 172.21.15.112:0/1791949962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3404108a50 msgr2=0x7f340419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f3409000700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3404108a50 0x7f340419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f3409000700 1 -- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3404103450 msgr2=0x7f34041996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f3409000700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3404103450 0x7f34041996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.640+0000 7f3409000700 1 -- 172.21.15.112:0/1791949962 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340419e190 con 0x7f3404102a90 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.641+0000 7f3409000700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3404102a90 0x7f3404199180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33f8005910 tx=0x7f33f800b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.641+0000 7f3401600700 1 -- 172.21.15.112:0/1791949962 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8027070 con 0x7f3404102a90 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.641+0000 7f3401600700 1 -- 172.21.15.112:0/1791949962 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f8022470 con 0x7f3404102a90 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.641+0000 7f340ba00700 1 -- 172.21.15.112:0/1791949962 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340419e420 con 0x7f3404102a90 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.641+0000 7f3401600700 1 -- 172.21.15.112:0/1791949962 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8017650 con 0x7f3404102a90 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.641+0000 7f340ba00700 1 -- 172.21.15.112:0/1791949962 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34041a5d00 con 0x7f3404102a90 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.642+0000 7f340ba00700 1 -- 172.21.15.112:0/1791949962 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34040631d0 con 0x7f3404102a90 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.646+0000 7f3401600700 1 -- 172.21.15.112:0/1791949962 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f33f802a070 con 0x7f3404102a90 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.647+0000 7f3401600700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33ec0a3d50 0x7f33ec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.647+0000 7f3401600700 1 -- 172.21.15.112:0/1791949962 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f33f80d1610 con 0x7f3404102a90 2024-04-25T18:45:02.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.648+0000 7f3403e00700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33ec0a3d50 0x7f33ec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.648+0000 7f3401600700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528] conn(0x7f33ec0ad990 0x7f33ec0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.648+0000 7f3401600700 1 -- 172.21.15.112:0/1791949962 --> [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f33f8015ec0 con 0x7f33ec0ad990 2024-04-25T18:45:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.648+0000 7f3401600700 1 -- 172.21.15.112:0/1791949962 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f33f80d1a60 con 0x7f3404102a90 2024-04-25T18:45:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.648+0000 7f3409a00700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528] conn(0x7f33ec0ad990 0x7f33ec0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.648+0000 7f3403e00700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33ec0a3d50 0x7f33ec0a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f340407a2f0 tx=0x7f33f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.649+0000 7f3409a00700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528] conn(0x7f33ec0ad990 0x7f33ec0afe30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.650+0000 7f3401600700 1 -- 172.21.15.112:0/1791949962 <== osd.9 v2:172.21.15.112:6826/349758528 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f33f8015ec0 con 0x7f33ec0ad990 2024-04-25T18:45:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.677+0000 7f340ba00700 1 -- 172.21.15.112:0/1791949962 --> [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f34040635d0 con 0x7f33ec0ad990 2024-04-25T18:45:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.677+0000 7f3401600700 1 -- 172.21.15.112:0/1791949962 <== osd.9 v2:172.21.15.112:6826/349758528 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f34040635d0 con 0x7f33ec0ad990 2024-04-25T18:45:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.678+0000 7f340ba00700 1 -- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528] conn(0x7f33ec0ad990 msgr2=0x7f33ec0afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.678+0000 7f340ba00700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528] conn(0x7f33ec0ad990 0x7f33ec0afe30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.678+0000 7f340ba00700 1 -- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33ec0a3d50 msgr2=0x7f33ec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.678+0000 7f340ba00700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f33ec0a3d50 0x7f33ec0a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f340407a2f0 tx=0x7f33f0009260 comp rx=0 tx=0).stop 2024-04-25T18:45:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.678+0000 7f340ba00700 1 -- 172.21.15.112:0/1791949962 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3404102a90 msgr2=0x7f3404199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.678+0000 7f340ba00700 1 --2- 172.21.15.112:0/1791949962 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3404102a90 0x7f3404199180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33f8005910 tx=0x7f33f800b670 comp rx=0 tx=0).stop 2024-04-25T18:45:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.678+0000 7f3409a00700 1 -- 172.21.15.112:0/1791949962 reap_dead start 2024-04-25T18:45:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.678+0000 7f340ba00700 1 -- 172.21.15.112:0/1791949962 shutdown_connections 2024-04-25T18:45:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.678+0000 7f340ba00700 1 -- 172.21.15.112:0/1791949962 >> 172.21.15.112:0/1791949962 conn(0x7f3404079ba0 msgr2=0x7f3404101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.678+0000 7f340ba00700 1 -- 172.21.15.112:0/1791949962 shutdown_connections 2024-04-25T18:45:02.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:44:58.678+0000 7f340ba00700 1 -- 172.21.15.112:0/1791949962 wait complete. 2024-04-25T18:45:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:02 smithi112 ceph-mon[27864]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:45:02.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:02 smithi171 ceph-mon[33266]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:45:02.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:02 smithi161 ceph-mon[32609]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:45:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.978+0000 7fe75e600700 1 -- 172.21.15.112:0/248181113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe758105870 msgr2=0x7fe7581098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:02.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.978+0000 7fe75e600700 1 --2- 172.21.15.112:0/248181113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe758105870 0x7fe7581098c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe7440099c0 tx=0x7fe744009cd0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.979+0000 7fe75e600700 1 -- 172.21.15.112:0/248181113 shutdown_connections 2024-04-25T18:45:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.979+0000 7fe75e600700 1 --2- 172.21.15.112:0/248181113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe758109fa0 0x7fe758111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.979+0000 7fe75e600700 1 --2- 172.21.15.112:0/248181113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe758104ec0 0x7fe7581052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.979+0000 7fe75e600700 1 --2- 172.21.15.112:0/248181113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe758105870 0x7fe7581098c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.979+0000 7fe75e600700 1 -- 172.21.15.112:0/248181113 >> 172.21.15.112:0/248181113 conn(0x7fe758100750 msgr2=0x7fe758102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.979+0000 7fe75e600700 1 -- 172.21.15.112:0/248181113 shutdown_connections 2024-04-25T18:45:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.979+0000 7fe75e600700 1 -- 172.21.15.112:0/248181113 wait complete. 2024-04-25T18:45:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.980+0000 7fe75e600700 1 Processor -- start 2024-04-25T18:45:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.980+0000 7fe75e600700 1 -- start start 2024-04-25T18:45:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.980+0000 7fe75e600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe758104ec0 0x7fe75819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.980+0000 7fe75e600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe758105870 0x7fe75819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.980+0000 7fe75e600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe758109fa0 0x7fe75819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.980+0000 7fe75e600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe758114e60 con 0x7fe758109fa0 2024-04-25T18:45:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.980+0000 7fe75e600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe758114ce0 con 0x7fe758104ec0 2024-04-25T18:45:02.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.980+0000 7fe75e600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe758114fe0 con 0x7fe758105870 2024-04-25T18:45:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.980+0000 7fe757e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe758109fa0 0x7fe75819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.980+0000 7fe757400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe758104ec0 0x7fe75819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.980+0000 7fe757400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe758104ec0 0x7fe75819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:36912/0 (socket says 172.21.15.112:36912) 2024-04-25T18:45:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.981+0000 7fe757e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe758109fa0 0x7fe75819fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58104/0 (socket says 172.21.15.112:58104) 2024-04-25T18:45:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.981+0000 7fe757400700 1 -- 172.21.15.112:0/2877362464 learned_addr learned my addr 172.21.15.112:0/2877362464 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:02.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.981+0000 7fe756a00700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe758105870 0x7fe75819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.981+0000 7fe757400700 1 -- 172.21.15.112:0/2877362464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe758105870 msgr2=0x7fe75819b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.981+0000 7fe757400700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe758105870 0x7fe75819b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.981+0000 7fe757400700 1 -- 172.21.15.112:0/2877362464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe758109fa0 msgr2=0x7fe75819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.981+0000 7fe757400700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe758109fa0 0x7fe75819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.981+0000 7fe757400700 1 -- 172.21.15.112:0/2877362464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7581a0390 con 0x7fe758104ec0 2024-04-25T18:45:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.981+0000 7fe757e00700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe758109fa0 0x7fe75819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.981+0000 7fe757400700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe758104ec0 0x7fe75819b3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe74800eb30 tx=0x7fe74800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.982+0000 7fe74fe00700 1 -- 172.21.15.112:0/2877362464 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74800cbd0 con 0x7fe758104ec0 2024-04-25T18:45:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.982+0000 7fe74fe00700 1 -- 172.21.15.112:0/2877362464 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7480044d0 con 0x7fe758104ec0 2024-04-25T18:45:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.982+0000 7fe75e600700 1 -- 172.21.15.112:0/2877362464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7581a0680 con 0x7fe758104ec0 2024-04-25T18:45:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.982+0000 7fe74fe00700 1 -- 172.21.15.112:0/2877362464 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748003e30 con 0x7fe758104ec0 2024-04-25T18:45:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.982+0000 7fe75e600700 1 -- 172.21.15.112:0/2877362464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7581a7f00 con 0x7fe758104ec0 2024-04-25T18:45:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.983+0000 7fe75e600700 1 -- 172.21.15.112:0/2877362464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7580630d0 con 0x7fe758104ec0 2024-04-25T18:45:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.985+0000 7fe74fe00700 1 -- 172.21.15.112:0/2877362464 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe748004000 con 0x7fe758104ec0 2024-04-25T18:45:02.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.986+0000 7fe74fe00700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe73c0a3d50 0x7fe73c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:02.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.986+0000 7fe74fe00700 1 -- 172.21.15.112:0/2877362464 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fe7480d0d10 con 0x7fe758104ec0 2024-04-25T18:45:02.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.987+0000 7fe756a00700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe73c0a3d50 0x7fe73c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:02.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.987+0000 7fe74fe00700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571] conn(0x7fe73c0ad990 0x7fe73c0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:02.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.987+0000 7fe74fe00700 1 -- 172.21.15.112:0/2877362464 --> [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe748017e50 con 0x7fe73c0ad990 2024-04-25T18:45:02.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.987+0000 7fe74fe00700 1 -- 172.21.15.112:0/2877362464 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fe7480d10c0 con 0x7fe758104ec0 2024-04-25T18:45:02.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.987+0000 7fe757e00700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571] conn(0x7fe73c0ad990 0x7fe73c0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:02.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.987+0000 7fe756a00700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe73c0a3d50 0x7fe73c0a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe744000c00 tx=0x7fe744005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:02.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.988+0000 7fe757e00700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571] conn(0x7fe73c0ad990 0x7fe73c0afe30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:02.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:02.989+0000 7fe74fe00700 1 -- 172.21.15.112:0/2877362464 <== osd.8 v2:172.21.15.171:6816/835756571 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe748017e50 con 0x7fe73c0ad990 2024-04-25T18:45:03.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.007+0000 7fe75e600700 1 -- 172.21.15.112:0/2877362464 --> [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe758063550 con 0x7fe73c0ad990 2024-04-25T18:45:03.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.007+0000 7fe74fe00700 1 -- 172.21.15.112:0/2877362464 <== osd.8 v2:172.21.15.171:6816/835756571 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe758063550 con 0x7fe73c0ad990 2024-04-25T18:45:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.008+0000 7fe75e600700 1 -- 172.21.15.112:0/2877362464 >> [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571] conn(0x7fe73c0ad990 msgr2=0x7fe73c0afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.008+0000 7fe75e600700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571] conn(0x7fe73c0ad990 0x7fe73c0afe30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.008+0000 7fe75e600700 1 -- 172.21.15.112:0/2877362464 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe73c0a3d50 msgr2=0x7fe73c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.008+0000 7fe75e600700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe73c0a3d50 0x7fe73c0a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe744000c00 tx=0x7fe744005fb0 comp rx=0 tx=0).stop 2024-04-25T18:45:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.008+0000 7fe75e600700 1 -- 172.21.15.112:0/2877362464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe758104ec0 msgr2=0x7fe75819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:03.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.008+0000 7fe75e600700 1 --2- 172.21.15.112:0/2877362464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe758104ec0 0x7fe75819b3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe74800eb30 tx=0x7fe74800ee40 comp rx=0 tx=0).stop 2024-04-25T18:45:03.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.008+0000 7fe757e00700 1 -- 172.21.15.112:0/2877362464 reap_dead start 2024-04-25T18:45:03.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.008+0000 7fe75e600700 1 -- 172.21.15.112:0/2877362464 shutdown_connections 2024-04-25T18:45:03.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.008+0000 7fe75e600700 1 -- 172.21.15.112:0/2877362464 >> 172.21.15.112:0/2877362464 conn(0x7fe758100750 msgr2=0x7fe758101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:03.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.008+0000 7fe75e600700 1 -- 172.21.15.112:0/2877362464 shutdown_connections 2024-04-25T18:45:03.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:03.009+0000 7fe75e600700 1 -- 172.21.15.112:0/2877362464 wait complete. 2024-04-25T18:45:03.919 INFO:teuthology.orchestra.run.smithi112.stdout:360777252902 2024-04-25T18:45:03.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T18:45:04.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:04 smithi112 ceph-mon[27864]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:04.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:04 smithi171 ceph-mon[33266]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:04.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:04 smithi161 ceph-mon[32609]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:05.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.637+0000 7f1b7b200700 1 -- 172.21.15.112:0/3290447585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b74104dd0 msgr2=0x7f1b741051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.637+0000 7f1b7b200700 1 --2- 172.21.15.112:0/3290447585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b74104dd0 0x7f1b741051b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b680099c0 tx=0x7f1b68009cd0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.637+0000 7f1b7b200700 1 -- 172.21.15.112:0/3290447585 shutdown_connections 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.637+0000 7f1b7b200700 1 --2- 172.21.15.112:0/3290447585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b74109eb0 0x7f1b74111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.637+0000 7f1b7b200700 1 --2- 172.21.15.112:0/3290447585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b74104dd0 0x7f1b741051b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.637+0000 7f1b7b200700 1 --2- 172.21.15.112:0/3290447585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74105780 0x7f1b741097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.637+0000 7f1b7b200700 1 -- 172.21.15.112:0/3290447585 >> 172.21.15.112:0/3290447585 conn(0x7f1b74100660 msgr2=0x7f1b74102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.637+0000 7f1b7b200700 1 -- 172.21.15.112:0/3290447585 shutdown_connections 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.637+0000 7f1b7b200700 1 -- 172.21.15.112:0/3290447585 wait complete. 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.638+0000 7f1b7b200700 1 Processor -- start 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b7b200700 1 -- start start 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b7b200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74104dd0 0x7f1b7419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b7b200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b74105780 0x7f1b7419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b7b200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b74109eb0 0x7f1b7419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b7b200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74114d70 con 0x7f1b74109eb0 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b7b200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74114bf0 con 0x7f1b74105780 2024-04-25T18:45:05.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b7b200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74114ef0 con 0x7f1b74104dd0 2024-04-25T18:45:05.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b7a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b74109eb0 0x7f1b7419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:05.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b7a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b74109eb0 0x7f1b7419fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58124/0 (socket says 172.21.15.112:58124) 2024-04-25T18:45:05.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b79e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74104dd0 0x7f1b7419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:05.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b7a800700 1 -- 172.21.15.112:0/3390715431 learned_addr learned my addr 172.21.15.112:0/3390715431 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:05.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.639+0000 7f1b79400700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b74105780 0x7f1b7419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.640+0000 7f1b7a800700 1 -- 172.21.15.112:0/3390715431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74104dd0 msgr2=0x7f1b7419b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.640+0000 7f1b7a800700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74104dd0 0x7f1b7419b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.640+0000 7f1b7a800700 1 -- 172.21.15.112:0/3390715431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b74105780 msgr2=0x7f1b7419b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.640+0000 7f1b7a800700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b74105780 0x7f1b7419b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.640+0000 7f1b7a800700 1 -- 172.21.15.112:0/3390715431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b741a01f0 con 0x7f1b74109eb0 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.640+0000 7f1b79e00700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74104dd0 0x7f1b7419b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.640+0000 7f1b7a800700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b74109eb0 0x7f1b7419fab0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1b70007ee0 tx=0x7f1b7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.640+0000 7f1b6ea00700 1 -- 172.21.15.112:0/3390715431 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70004bb0 con 0x7f1b74109eb0 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.640+0000 7f1b6ea00700 1 -- 172.21.15.112:0/3390715431 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b70004d50 con 0x7f1b74109eb0 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.640+0000 7f1b7b200700 1 -- 172.21.15.112:0/3390715431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b741a04e0 con 0x7f1b74109eb0 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.640+0000 7f1b6ea00700 1 -- 172.21.15.112:0/3390715431 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70005920 con 0x7f1b74109eb0 2024-04-25T18:45:05.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.641+0000 7f1b7b200700 1 -- 172.21.15.112:0/3390715431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b741a7d60 con 0x7f1b74109eb0 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.641+0000 7f1b7b200700 1 -- 172.21.15.112:0/3390715431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b740631d0 con 0x7f1b74109eb0 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.643+0000 7f1b6ea00700 1 -- 172.21.15.112:0/3390715431 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1b7001a020 con 0x7f1b74109eb0 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.645+0000 7f1b6ea00700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1b5c0a3d50 0x7f1b5c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.645+0000 7f1b6ea00700 1 -- 172.21.15.112:0/3390715431 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f1b700d0300 con 0x7f1b74109eb0 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.645+0000 7f1b79e00700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1b5c0a3d50 0x7f1b5c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.645+0000 7f1b6ea00700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933] conn(0x7f1b5c0ad990 0x7f1b5c0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.645+0000 7f1b6ea00700 1 -- 172.21.15.112:0/3390715431 --> [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b5c0b0500 con 0x7f1b5c0ad990 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.645+0000 7f1b6ea00700 1 -- 172.21.15.112:0/3390715431 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f1b700d06f0 con 0x7f1b74109eb0 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.646+0000 7f1b79400700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933] conn(0x7f1b5c0ad990 0x7f1b5c0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.646+0000 7f1b79e00700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1b5c0a3d50 0x7f1b5c0a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1b68003760 tx=0x7f1b68005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.646+0000 7f1b79400700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933] conn(0x7f1b5c0ad990 0x7f1b5c0afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.647+0000 7f1b6ea00700 1 -- 172.21.15.112:0/3390715431 <== osd.6 v2:172.21.15.112:6818/1933654933 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b5c0b0500 con 0x7f1b5c0ad990 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.665+0000 7f1b7b200700 1 -- 172.21.15.112:0/3390715431 --> [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b74063650 con 0x7f1b5c0ad990 2024-04-25T18:45:05.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.666+0000 7f1b6ea00700 1 -- 172.21.15.112:0/3390715431 <== osd.6 v2:172.21.15.112:6818/1933654933 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1b74063650 con 0x7f1b5c0ad990 2024-04-25T18:45:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.666+0000 7f1b7b200700 1 -- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933] conn(0x7f1b5c0ad990 msgr2=0x7f1b5c0afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.666+0000 7f1b7b200700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933] conn(0x7f1b5c0ad990 0x7f1b5c0afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.667+0000 7f1b7b200700 1 -- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1b5c0a3d50 msgr2=0x7f1b5c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.667+0000 7f1b7b200700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1b5c0a3d50 0x7f1b5c0a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1b68003760 tx=0x7f1b68005fb0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.667+0000 7f1b7b200700 1 -- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b74109eb0 msgr2=0x7f1b7419fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.667+0000 7f1b7b200700 1 --2- 172.21.15.112:0/3390715431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b74109eb0 0x7f1b7419fab0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1b70007ee0 tx=0x7f1b7000d3b0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.667+0000 7f1b7a800700 1 -- 172.21.15.112:0/3390715431 reap_dead start 2024-04-25T18:45:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.667+0000 7f1b7b200700 1 -- 172.21.15.112:0/3390715431 shutdown_connections 2024-04-25T18:45:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.667+0000 7f1b7b200700 1 -- 172.21.15.112:0/3390715431 >> 172.21.15.112:0/3390715431 conn(0x7f1b74100660 msgr2=0x7f1b74102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.667+0000 7f1b7b200700 1 -- 172.21.15.112:0/3390715431 shutdown_connections 2024-04-25T18:45:05.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.667+0000 7f1b7b200700 1 -- 172.21.15.112:0/3390715431 wait complete. 2024-04-25T18:45:05.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.285+0000 7f7acfa00700 1 -- 172.21.15.112:0/2544218900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac810e3b0 msgr2=0x7f7ac8115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.285+0000 7f7acfa00700 1 --2- 172.21.15.112:0/2544218900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac810e3b0 0x7f7ac8115f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009920 tx=0x7f7ac0009c30 comp rx=0 tx=0).stop 2024-04-25T18:45:05.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.285+0000 7f7acfa00700 1 -- 172.21.15.112:0/2544218900 shutdown_connections 2024-04-25T18:45:05.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.285+0000 7f7acfa00700 1 --2- 172.21.15.112:0/2544218900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac810e3b0 0x7f7ac8115f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.285+0000 7f7acfa00700 1 --2- 172.21.15.112:0/2544218900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac81092d0 0x7f7ac81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.285+0000 7f7acfa00700 1 --2- 172.21.15.112:0/2544218900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ac8109c80 0x7f7ac810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.285+0000 7f7acfa00700 1 -- 172.21.15.112:0/2544218900 >> 172.21.15.112:0/2544218900 conn(0x7f7ac8076210 msgr2=0x7f7ac8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:05.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.285+0000 7f7acfa00700 1 -- 172.21.15.112:0/2544218900 shutdown_connections 2024-04-25T18:45:05.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.285+0000 7f7acfa00700 1 -- 172.21.15.112:0/2544218900 wait complete. 2024-04-25T18:45:05.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.286+0000 7f7acfa00700 1 Processor -- start 2024-04-25T18:45:05.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.286+0000 7f7acfa00700 1 -- start start 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acfa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ac81092d0 0x7f7ac81a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acfa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac8109c80 0x7f7ac81a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acfa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac810e3b0 0x7f7ac819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acfa00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81192a0 con 0x7f7ac810e3b0 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acfa00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8119120 con 0x7f7ac81092d0 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acfa00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8119420 con 0x7f7ac8109c80 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acda00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac810e3b0 0x7f7ac819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acda00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac810e3b0 0x7f7ac819f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58108/0 (socket says 172.21.15.112:58108) 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acda00700 1 -- 172.21.15.112:0/571416837 learned_addr learned my addr 172.21.15.112:0/571416837 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acd000700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ac81092d0 0x7f7ac81a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7ac7e00700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac8109c80 0x7f7ac81a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acda00700 1 -- 172.21.15.112:0/571416837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac8109c80 msgr2=0x7f7ac81a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acda00700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac8109c80 0x7f7ac81a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acda00700 1 -- 172.21.15.112:0/571416837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ac81092d0 msgr2=0x7f7ac81a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acda00700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ac81092d0 0x7f7ac81a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acda00700 1 -- 172.21.15.112:0/571416837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac819fd40 con 0x7f7ac810e3b0 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.287+0000 7f7acd000700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ac81092d0 0x7f7ac81a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.288+0000 7f7acda00700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac810e3b0 0x7f7ac819f5e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005950 tx=0x7f7ac000fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.288+0000 7f7ac5600700 1 -- 172.21.15.112:0/571416837 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0005040 con 0x7f7ac810e3b0 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.288+0000 7f7ac5600700 1 -- 172.21.15.112:0/571416837 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac0005690 con 0x7f7ac810e3b0 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.288+0000 7f7acfa00700 1 -- 172.21.15.112:0/571416837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac819ffd0 con 0x7f7ac810e3b0 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.288+0000 7f7ac5600700 1 -- 172.21.15.112:0/571416837 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0003a00 con 0x7f7ac810e3b0 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.288+0000 7f7acfa00700 1 -- 172.21.15.112:0/571416837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac81ac2f0 con 0x7f7ac810e3b0 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.289+0000 7f7acfa00700 1 -- 172.21.15.112:0/571416837 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ac80631d0 con 0x7f7ac810e3b0 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.291+0000 7f7ac5600700 1 -- 172.21.15.112:0/571416837 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7ac000b7f0 con 0x7f7ac810e3b0 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.293+0000 7f7ac5600700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7ab00a3d50 0x7f7ab00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.293+0000 7f7ac5600700 1 -- 172.21.15.112:0/571416837 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f7ac00d1920 con 0x7f7ac810e3b0 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.293+0000 7f7acd000700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7ab00a3d50 0x7f7ab00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.293+0000 7f7ac5600700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765] conn(0x7f7ab00ad860 0x7f7ab00afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.293+0000 7f7ac5600700 1 -- 172.21.15.112:0/571416837 --> [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ab00b03d0 con 0x7f7ab00ad860 2024-04-25T18:45:05.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.293+0000 7f7ac5600700 1 -- 172.21.15.112:0/571416837 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f7ac00d1cd0 con 0x7f7ac810e3b0 2024-04-25T18:45:05.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.293+0000 7f7ac7e00700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765] conn(0x7f7ab00ad860 0x7f7ab00afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:05.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.294+0000 7f7acd000700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7ab00a3d50 0x7f7ab00a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009cd0 tx=0x7f7abc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:05.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.294+0000 7f7ac7e00700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765] conn(0x7f7ab00ad860 0x7f7ab00afd00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:05.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.295+0000 7f7ac5600700 1 -- 172.21.15.112:0/571416837 <== osd.10 v2:172.21.15.161:6824/3013292765 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ab00b03d0 con 0x7f7ab00ad860 2024-04-25T18:45:05.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.313+0000 7f7acfa00700 1 -- 172.21.15.112:0/571416837 --> [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7ac8063650 con 0x7f7ab00ad860 2024-04-25T18:45:05.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.314+0000 7f7ac5600700 1 -- 172.21.15.112:0/571416837 <== osd.10 v2:172.21.15.161:6824/3013292765 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7ac8063650 con 0x7f7ab00ad860 2024-04-25T18:45:05.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.314+0000 7f7acfa00700 1 -- 172.21.15.112:0/571416837 >> [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765] conn(0x7f7ab00ad860 msgr2=0x7f7ab00afd00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.314+0000 7f7acfa00700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765] conn(0x7f7ab00ad860 0x7f7ab00afd00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:05.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.314+0000 7f7acfa00700 1 -- 172.21.15.112:0/571416837 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7ab00a3d50 msgr2=0x7f7ab00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.314+0000 7f7acfa00700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f7ab00a3d50 0x7f7ab00a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009cd0 tx=0x7f7abc00b040 comp rx=0 tx=0).stop 2024-04-25T18:45:05.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.314+0000 7f7acfa00700 1 -- 172.21.15.112:0/571416837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac810e3b0 msgr2=0x7f7ac819f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:05.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.314+0000 7f7acfa00700 1 --2- 172.21.15.112:0/571416837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac810e3b0 0x7f7ac819f5e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005950 tx=0x7f7ac000fa70 comp rx=0 tx=0).stop 2024-04-25T18:45:05.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.314+0000 7f7acda00700 1 -- 172.21.15.112:0/571416837 reap_dead start 2024-04-25T18:45:05.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.314+0000 7f7acfa00700 1 -- 172.21.15.112:0/571416837 shutdown_connections 2024-04-25T18:45:05.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.314+0000 7f7acfa00700 1 -- 172.21.15.112:0/571416837 >> 172.21.15.112:0/571416837 conn(0x7f7ac8076210 msgr2=0x7f7ac8076830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:05.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.315+0000 7f7acfa00700 1 -- 172.21.15.112:0/571416837 shutdown_connections 2024-04-25T18:45:05.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:04.315+0000 7f7acfa00700 1 -- 172.21.15.112:0/571416837 wait complete. 2024-04-25T18:45:05.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:05 smithi112 ceph-mon[27864]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:05.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:05 smithi171 ceph-mon[33266]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:05.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:05 smithi161 ceph-mon[32609]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:06.142 INFO:teuthology.orchestra.run.smithi112.stdout:150323855438 2024-04-25T18:45:06.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T18:45:07.431 INFO:teuthology.orchestra.run.smithi112.stdout:300647710767 2024-04-25T18:45:07.432 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T18:45:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.543+0000 7f540e400700 1 -- 172.21.15.112:0/3692851790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5408107990 msgr2=0x7f5408107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.543+0000 7f540e400700 1 --2- 172.21.15.112:0/3692851790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5408107990 0x7f5408107d70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009990 tx=0x7f53f8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.543+0000 7f540e400700 1 -- 172.21.15.112:0/3692851790 shutdown_connections 2024-04-25T18:45:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.543+0000 7f540e400700 1 --2- 172.21.15.112:0/3692851790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5408102300 0x7f540810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.543+0000 7f540e400700 1 --2- 172.21.15.112:0/3692851790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5408107990 0x7f5408107d70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.543+0000 7f540e400700 1 --2- 172.21.15.112:0/3692851790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5408101940 0x7f5408101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.543+0000 7f540e400700 1 -- 172.21.15.112:0/3692851790 >> 172.21.15.112:0/3692851790 conn(0x7f54080788f0 msgr2=0x7f5408078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.543+0000 7f540e400700 1 -- 172.21.15.112:0/3692851790 shutdown_connections 2024-04-25T18:45:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.543+0000 7f540e400700 1 -- 172.21.15.112:0/3692851790 wait complete. 2024-04-25T18:45:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.544+0000 7f540e400700 1 Processor -- start 2024-04-25T18:45:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.544+0000 7f540e400700 1 -- start start 2024-04-25T18:45:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.544+0000 7f540e400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5408101940 0x7f540819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.544+0000 7f540e400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5408102300 0x7f540819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.544+0000 7f540e400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5408107990 0x7f540819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.544+0000 7f540e400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5408111b50 con 0x7f5408101940 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.544+0000 7f540e400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f54081119d0 con 0x7f5408107990 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.544+0000 7f540e400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5408111cd0 con 0x7f5408102300 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.545+0000 7f5407400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5408101940 0x7f540819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.545+0000 7f5407400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5408101940 0x7f540819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58132/0 (socket says 172.21.15.112:58132) 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.545+0000 7f5407400700 1 -- 172.21.15.112:0/2341282318 learned_addr learned my addr 172.21.15.112:0/2341282318 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.545+0000 7f5407e00700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5408107990 0x7f540819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.545+0000 7f5406a00700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5408102300 0x7f540819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.545+0000 7f5407400700 1 -- 172.21.15.112:0/2341282318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5408102300 msgr2=0x7f540819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.545+0000 7f5407400700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5408102300 0x7f540819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.545+0000 7f5407400700 1 -- 172.21.15.112:0/2341282318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5408107990 msgr2=0x7f540819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.545+0000 7f5407400700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5408107990 0x7f540819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.545+0000 7f5407400700 1 -- 172.21.15.112:0/2341282318 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54081a0390 con 0x7f5408101940 2024-04-25T18:45:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.545+0000 7f5407e00700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5408107990 0x7f540819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:45:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.546+0000 7f5407400700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5408101940 0x7f540819b380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000c00 tx=0x7f53f8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.546+0000 7f53ffe00700 1 -- 172.21.15.112:0/2341282318 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8027070 con 0x7f5408101940 2024-04-25T18:45:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.546+0000 7f53ffe00700 1 -- 172.21.15.112:0/2341282318 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53f8022470 con 0x7f5408101940 2024-04-25T18:45:07.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.546+0000 7f540e400700 1 -- 172.21.15.112:0/2341282318 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54081a0620 con 0x7f5408101940 2024-04-25T18:45:07.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.546+0000 7f53ffe00700 1 -- 172.21.15.112:0/2341282318 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8017650 con 0x7f5408101940 2024-04-25T18:45:07.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.546+0000 7f540e400700 1 -- 172.21.15.112:0/2341282318 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54081a7f00 con 0x7f5408101940 2024-04-25T18:45:07.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.547+0000 7f540e400700 1 -- 172.21.15.112:0/2341282318 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f540806b140 con 0x7f5408101940 2024-04-25T18:45:07.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.550+0000 7f53ffe00700 1 -- 172.21.15.112:0/2341282318 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f53f802a070 con 0x7f5408101940 2024-04-25T18:45:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.552+0000 7f53ffe00700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f53ec0a3d50 0x7f53ec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.552+0000 7f53ffe00700 1 -- 172.21.15.112:0/2341282318 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f53f80d1820 con 0x7f5408101940 2024-04-25T18:45:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.552+0000 7f5406a00700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f53ec0a3d50 0x7f53ec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.553+0000 7f53ffe00700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156] conn(0x7f53ec0ad990 0x7f53ec0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.553+0000 7f53ffe00700 1 -- 172.21.15.112:0/2341282318 --> [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53f8024ec0 con 0x7f53ec0ad990 2024-04-25T18:45:07.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.553+0000 7f53ffe00700 1 -- 172.21.15.112:0/2341282318 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f53f80d1bd0 con 0x7f5408101940 2024-04-25T18:45:07.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.553+0000 7f5406a00700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f53ec0a3d50 0x7f53ec0a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5408078d10 tx=0x7f53f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:07.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.553+0000 7f5407e00700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156] conn(0x7f53ec0ad990 0x7f53ec0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:07.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.554+0000 7f5407e00700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156] conn(0x7f53ec0ad990 0x7f53ec0afe30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:07.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.555+0000 7f53ffe00700 1 -- 172.21.15.112:0/2341282318 <== osd.3 v2:172.21.15.112:6810/1472407156 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53f8024ec0 con 0x7f53ec0ad990 2024-04-25T18:45:07.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.573+0000 7f540e400700 1 -- 172.21.15.112:0/2341282318 --> [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5408103060 con 0x7f53ec0ad990 2024-04-25T18:45:07.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.574+0000 7f53ffe00700 1 -- 172.21.15.112:0/2341282318 <== osd.3 v2:172.21.15.112:6810/1472407156 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5408103060 con 0x7f53ec0ad990 2024-04-25T18:45:07.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.575+0000 7f540e400700 1 -- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156] conn(0x7f53ec0ad990 msgr2=0x7f53ec0afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:07.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.575+0000 7f540e400700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156] conn(0x7f53ec0ad990 0x7f53ec0afe30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:07.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.575+0000 7f540e400700 1 -- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f53ec0a3d50 msgr2=0x7f53ec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:07.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.575+0000 7f540e400700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f53ec0a3d50 0x7f53ec0a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5408078d10 tx=0x7f53f4009260 comp rx=0 tx=0).stop 2024-04-25T18:45:07.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.575+0000 7f540e400700 1 -- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5408101940 msgr2=0x7f540819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:07.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.575+0000 7f540e400700 1 --2- 172.21.15.112:0/2341282318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5408101940 0x7f540819b380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000c00 tx=0x7f53f8009510 comp rx=0 tx=0).stop 2024-04-25T18:45:07.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.575+0000 7f5407e00700 1 -- 172.21.15.112:0/2341282318 reap_dead start 2024-04-25T18:45:07.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.575+0000 7f540e400700 1 -- 172.21.15.112:0/2341282318 shutdown_connections 2024-04-25T18:45:07.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.575+0000 7f540e400700 1 -- 172.21.15.112:0/2341282318 >> 172.21.15.112:0/2341282318 conn(0x7f54080788f0 msgr2=0x7f540806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:07.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.575+0000 7f540e400700 1 -- 172.21.15.112:0/2341282318 shutdown_connections 2024-04-25T18:45:07.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:07.575+0000 7f540e400700 1 -- 172.21.15.112:0/2341282318 wait complete. 2024-04-25T18:45:08.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.467+0000 7f3439a00700 1 -- 172.21.15.112:0/2277140107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3434108a30 msgr2=0x7f3434108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.467+0000 7f3439a00700 1 --2- 172.21.15.112:0/2277140107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3434108a30 0x7f3434108e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3424009990 tx=0x7f3424009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.467+0000 7f3439a00700 1 -- 172.21.15.112:0/2277140107 shutdown_connections 2024-04-25T18:45:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.467+0000 7f3439a00700 1 --2- 172.21.15.112:0/2277140107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f343406b0f0 0x7f343410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.467+0000 7f3439a00700 1 --2- 172.21.15.112:0/2277140107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3434108a30 0x7f3434108e10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.467+0000 7f3439a00700 1 --2- 172.21.15.112:0/2277140107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f343406a750 0x7f343406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.467+0000 7f3439a00700 1 -- 172.21.15.112:0/2277140107 >> 172.21.15.112:0/2277140107 conn(0x7f34340788f0 msgr2=0x7f3434078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.467+0000 7f3439a00700 1 -- 172.21.15.112:0/2277140107 shutdown_connections 2024-04-25T18:45:08.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.467+0000 7f3439a00700 1 -- 172.21.15.112:0/2277140107 wait complete. 2024-04-25T18:45:08.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.468+0000 7f3439a00700 1 Processor -- start 2024-04-25T18:45:08.470 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:08 smithi112 ceph-mon[27864]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T18:45:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3439a00700 1 -- start start 2024-04-25T18:45:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3439a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f343406a750 0x7f343419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3439a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f343406b0f0 0x7f343419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3439a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3434108a30 0x7f343419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3439a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3434112bd0 con 0x7f343406b0f0 2024-04-25T18:45:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3439a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3434112a50 con 0x7f343406a750 2024-04-25T18:45:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3439a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3434112d50 con 0x7f3434108a30 2024-04-25T18:45:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3432a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f343406a750 0x7f343419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3432000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f343406b0f0 0x7f343419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3432a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f343406a750 0x7f343419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:37002/0 (socket says 172.21.15.112:37002) 2024-04-25T18:45:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3432a00700 1 -- 172.21.15.112:0/3552996516 learned_addr learned my addr 172.21.15.112:0/3552996516 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.469+0000 7f3433400700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3434108a30 0x7f343419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.470+0000 7f3432a00700 1 -- 172.21.15.112:0/3552996516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3434108a30 msgr2=0x7f343419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.470+0000 7f3432a00700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3434108a30 0x7f343419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.470+0000 7f3432a00700 1 -- 172.21.15.112:0/3552996516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f343406b0f0 msgr2=0x7f343419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.470+0000 7f3432a00700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f343406b0f0 0x7f343419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.470+0000 7f3432a00700 1 -- 172.21.15.112:0/3552996516 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34341a0390 con 0x7f343406a750 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.470+0000 7f3432a00700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f343406a750 0x7f343419b380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3424000c00 tx=0x7f342400bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.470+0000 7f342b400700 1 -- 172.21.15.112:0/3552996516 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424027070 con 0x7f343406a750 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.470+0000 7f342b400700 1 -- 172.21.15.112:0/3552996516 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3424022470 con 0x7f343406a750 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.471+0000 7f342b400700 1 -- 172.21.15.112:0/3552996516 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424017650 con 0x7f343406a750 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.471+0000 7f3439a00700 1 -- 172.21.15.112:0/3552996516 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34341a0620 con 0x7f343406a750 2024-04-25T18:45:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.471+0000 7f3439a00700 1 -- 172.21.15.112:0/3552996516 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34341a7f00 con 0x7f343406a750 2024-04-25T18:45:08.474 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:08 smithi161 ceph-mon[32609]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T18:45:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.472+0000 7f3439a00700 1 -- 172.21.15.112:0/3552996516 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34341a0ab0 con 0x7f343406a750 2024-04-25T18:45:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.474+0000 7f342b400700 1 -- 172.21.15.112:0/3552996516 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f342402a070 con 0x7f343406a750 2024-04-25T18:45:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.475+0000 7f342b400700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f34180a3d50 0x7f34180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.475+0000 7f342b400700 1 -- 172.21.15.112:0/3552996516 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f34240d1680 con 0x7f343406a750 2024-04-25T18:45:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.475+0000 7f3432000700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f34180a3d50 0x7f34180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.476+0000 7f342b400700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313] conn(0x7f34180ad990 0x7f34180afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.476+0000 7f342b400700 1 -- 172.21.15.112:0/3552996516 --> [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34240095d0 con 0x7f34180ad990 2024-04-25T18:45:08.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.476+0000 7f342b400700 1 -- 172.21.15.112:0/3552996516 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f34240d1a30 con 0x7f343406a750 2024-04-25T18:45:08.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.476+0000 7f3433400700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313] conn(0x7f34180ad990 0x7f34180afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:08.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.476+0000 7f3432000700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f34180a3d50 0x7f34180a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3434078ff0 tx=0x7f3420009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:08.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.477+0000 7f3433400700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313] conn(0x7f34180ad990 0x7f34180afe30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:08.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.478+0000 7f342b400700 1 -- 172.21.15.112:0/3552996516 <== osd.5 v2:172.21.15.171:6808/2643512313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34240095d0 con 0x7f34180ad990 2024-04-25T18:45:08.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.495+0000 7f3439a00700 1 -- 172.21.15.112:0/3552996516 --> [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f34340633f0 con 0x7f34180ad990 2024-04-25T18:45:08.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.496+0000 7f342b400700 1 -- 172.21.15.112:0/3552996516 <== osd.5 v2:172.21.15.171:6808/2643512313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f34340633f0 con 0x7f34180ad990 2024-04-25T18:45:08.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.496+0000 7f3439a00700 1 -- 172.21.15.112:0/3552996516 >> [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313] conn(0x7f34180ad990 msgr2=0x7f34180afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:08.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.496+0000 7f3439a00700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313] conn(0x7f34180ad990 0x7f34180afe30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.497+0000 7f3439a00700 1 -- 172.21.15.112:0/3552996516 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f34180a3d50 msgr2=0x7f34180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.497+0000 7f3439a00700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f34180a3d50 0x7f34180a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3434078ff0 tx=0x7f3420009260 comp rx=0 tx=0).stop 2024-04-25T18:45:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.497+0000 7f3439a00700 1 -- 172.21.15.112:0/3552996516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f343406a750 msgr2=0x7f343419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.497+0000 7f3439a00700 1 --2- 172.21.15.112:0/3552996516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f343406a750 0x7f343419b380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3424000c00 tx=0x7f342400bfd0 comp rx=0 tx=0).stop 2024-04-25T18:45:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.497+0000 7f3433400700 1 -- 172.21.15.112:0/3552996516 reap_dead start 2024-04-25T18:45:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.497+0000 7f3439a00700 1 -- 172.21.15.112:0/3552996516 shutdown_connections 2024-04-25T18:45:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.497+0000 7f3439a00700 1 -- 172.21.15.112:0/3552996516 >> 172.21.15.112:0/3552996516 conn(0x7f34340788f0 msgr2=0x7f343410df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.497+0000 7f3439a00700 1 -- 172.21.15.112:0/3552996516 shutdown_connections 2024-04-25T18:45:08.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:08.497+0000 7f3439a00700 1 -- 172.21.15.112:0/3552996516 wait complete. 2024-04-25T18:45:08.662 INFO:teuthology.orchestra.run.smithi112.stdout:231928234045 2024-04-25T18:45:08.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T18:45:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:08 smithi171 ceph-mon[33266]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T18:45:09.040 INFO:teuthology.orchestra.run.smithi112.stdout:34359738466 2024-04-25T18:45:09.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T18:45:09.797 INFO:teuthology.orchestra.run.smithi112.stdout:266287972407 2024-04-25T18:45:09.798 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T18:45:10.163 INFO:teuthology.orchestra.run.smithi112.stdout:171798691914 2024-04-25T18:45:10.164 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T18:45:10.368 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:10 smithi112 ceph-mon[27864]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:10.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:10 smithi161 ceph-mon[32609]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:10.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:10 smithi171 ceph-mon[33266]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:10.782 INFO:teuthology.orchestra.run.smithi112.stdout:206158430275 2024-04-25T18:45:10.782 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T18:45:11.217 INFO:teuthology.orchestra.run.smithi112.stdout:326417514541 2024-04-25T18:45:11.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T18:45:11.801 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:11.853 INFO:teuthology.orchestra.run.smithi112.stdout:115964117077 2024-04-25T18:45:11.853 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T18:45:11.870 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:12.284 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:12.284 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:12.285 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:12.341 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:12 smithi112 ceph-mon[27864]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:12.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:12 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:45:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:12 smithi161 ceph-mon[32609]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:12 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:45:12.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:12 smithi171 ceph-mon[33266]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:12.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:12 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:45:12.925 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:14.265 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:14 smithi112 ceph-mon[27864]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:14.266 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.266 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.266 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.266 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.266 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.266 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:14 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:14 smithi161 ceph-mon[32609]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:14 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:14 smithi171 ceph-mon[33266]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:14.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:14.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:14 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:15.687 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:15.688 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:16.364 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:16 smithi112 ceph-mon[27864]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:16.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:16 smithi171 ceph-mon[33266]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:16.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:16 smithi161 ceph-mon[32609]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:18.467 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:18.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:18 smithi161 ceph-mon[32609]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T18:45:18.553 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:18 smithi112 ceph-mon[27864]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T18:45:18.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:18 smithi171 ceph-mon[33266]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T18:45:19.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.944+0000 7f7908e00700 1 -- 172.21.15.112:0/4249774025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7904105870 msgr2=0x7f79041098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:19.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.944+0000 7f7908e00700 1 --2- 172.21.15.112:0/4249774025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7904105870 0x7f79041098c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f78f00099c0 tx=0x7f78f0009cd0 comp rx=0 tx=0).stop 2024-04-25T18:45:19.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.944+0000 7f7908e00700 1 -- 172.21.15.112:0/4249774025 shutdown_connections 2024-04-25T18:45:19.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.944+0000 7f7908e00700 1 --2- 172.21.15.112:0/4249774025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7904109fa0 0x7f7904111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:19.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.944+0000 7f7908e00700 1 --2- 172.21.15.112:0/4249774025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7904104ec0 0x7f79041052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:19.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.944+0000 7f7908e00700 1 --2- 172.21.15.112:0/4249774025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7904105870 0x7f79041098c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:19.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.944+0000 7f7908e00700 1 -- 172.21.15.112:0/4249774025 >> 172.21.15.112:0/4249774025 conn(0x7f7904100750 msgr2=0x7f7904102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:19.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.944+0000 7f7908e00700 1 -- 172.21.15.112:0/4249774025 shutdown_connections 2024-04-25T18:45:19.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.944+0000 7f7908e00700 1 -- 172.21.15.112:0/4249774025 wait complete. 2024-04-25T18:45:19.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.946+0000 7f7908e00700 1 Processor -- start 2024-04-25T18:45:19.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.946+0000 7f7908e00700 1 -- start start 2024-04-25T18:45:19.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.946+0000 7f7908e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7904104ec0 0x7f790419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:19.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.946+0000 7f7908e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7904105870 0x7f790419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:19.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.946+0000 7f7908e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7904109fa0 0x7f790419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:19.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.946+0000 7f7908e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7904114e10 con 0x7f7904109fa0 2024-04-25T18:45:19.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.946+0000 7f7908e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7904114c90 con 0x7f7904104ec0 2024-04-25T18:45:19.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.946+0000 7f7908e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7904114f90 con 0x7f7904105870 2024-04-25T18:45:19.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.947+0000 7f7902800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7904109fa0 0x7f790419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:19.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.947+0000 7f7902800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7904109fa0 0x7f790419fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58366/0 (socket says 172.21.15.112:58366) 2024-04-25T18:45:19.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.947+0000 7f7901e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7904104ec0 0x7f790419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:19.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.947+0000 7f7902800700 1 -- 172.21.15.112:0/797501600 learned_addr learned my addr 172.21.15.112:0/797501600 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:19.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.947+0000 7f7901400700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7904105870 0x7f790419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:19.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.947+0000 7f7902800700 1 -- 172.21.15.112:0/797501600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7904105870 msgr2=0x7f790419b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:19.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.947+0000 7f7902800700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7904105870 0x7f790419b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:19.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.947+0000 7f7902800700 1 -- 172.21.15.112:0/797501600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7904104ec0 msgr2=0x7f790419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:19.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.947+0000 7f7902800700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7904104ec0 0x7f790419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:19.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.947+0000 7f7902800700 1 -- 172.21.15.112:0/797501600 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79041a02e0 con 0x7f7904109fa0 2024-04-25T18:45:19.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.947+0000 7f7902800700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7904109fa0 0x7f790419fba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f78fc007ee0 tx=0x7f78fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:19.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.948+0000 7f78faa00700 1 -- 172.21.15.112:0/797501600 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc004bb0 con 0x7f7904109fa0 2024-04-25T18:45:19.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.948+0000 7f78faa00700 1 -- 172.21.15.112:0/797501600 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78fc004d50 con 0x7f7904109fa0 2024-04-25T18:45:19.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.948+0000 7f7908e00700 1 -- 172.21.15.112:0/797501600 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79041a05d0 con 0x7f7904109fa0 2024-04-25T18:45:19.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.948+0000 7f78faa00700 1 -- 172.21.15.112:0/797501600 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc005920 con 0x7f7904109fa0 2024-04-25T18:45:19.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.948+0000 7f7908e00700 1 -- 172.21.15.112:0/797501600 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79041a7e50 con 0x7f7904109fa0 2024-04-25T18:45:19.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.949+0000 7f7908e00700 1 -- 172.21.15.112:0/797501600 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79041063a0 con 0x7f7904109fa0 2024-04-25T18:45:19.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.951+0000 7f78faa00700 1 -- 172.21.15.112:0/797501600 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f78fc01a020 con 0x7f7904109fa0 2024-04-25T18:45:19.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.953+0000 7f78faa00700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78e80a3cd0 0x7f78e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:19.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.953+0000 7f78faa00700 1 -- 172.21.15.112:0/797501600 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f78fc0cfbe0 con 0x7f7904109fa0 2024-04-25T18:45:19.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.953+0000 7f7901e00700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78e80a3cd0 0x7f78e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:19.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.954+0000 7f7901e00700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78e80a3cd0 0x7f78e80a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009c60 tx=0x7f78f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:19.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:19.958+0000 7f78faa00700 1 -- 172.21.15.112:0/797501600 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78fc010070 con 0x7f7904109fa0 2024-04-25T18:45:20.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.186+0000 7f7908e00700 1 -- 172.21.15.112:0/797501600 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f790402df80 con 0x7f7904109fa0 2024-04-25T18:45:20.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.187+0000 7f78faa00700 1 -- 172.21.15.112:0/797501600 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f78fc09c3e0 con 0x7f7904109fa0 2024-04-25T18:45:20.189 INFO:teuthology.orchestra.run.smithi112.stdout:150323855442 2024-04-25T18:45:20.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 -- 172.21.15.112:0/797501600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78e80a3cd0 msgr2=0x7f78e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:20.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78e80a3cd0 0x7f78e80a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009c60 tx=0x7f78f4009260 comp rx=0 tx=0).stop 2024-04-25T18:45:20.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 -- 172.21.15.112:0/797501600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7904109fa0 msgr2=0x7f790419fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:20.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7904109fa0 0x7f790419fba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f78fc007ee0 tx=0x7f78fc00d3b0 comp rx=0 tx=0).stop 2024-04-25T18:45:20.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 -- 172.21.15.112:0/797501600 shutdown_connections 2024-04-25T18:45:20.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f78e80a3cd0 0x7f78e80a6190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:20.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7904109fa0 0x7f790419fba0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:20.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7904104ec0 0x7f790419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:20.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 --2- 172.21.15.112:0/797501600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7904105870 0x7f790419b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:20.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 -- 172.21.15.112:0/797501600 >> 172.21.15.112:0/797501600 conn(0x7f7904100750 msgr2=0x7f7904102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:20.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 -- 172.21.15.112:0/797501600 shutdown_connections 2024-04-25T18:45:20.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:20.190+0000 7f7908e00700 1 -- 172.21.15.112:0/797501600 wait complete. 2024-04-25T18:45:20.458 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:20 smithi112 ceph-mon[27864]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T18:45:20.458 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:20 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/797501600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T18:45:20.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:20 smithi171 ceph-mon[33266]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T18:45:20.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:20 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/797501600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T18:45:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:20 smithi161 ceph-mon[32609]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T18:45:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:20 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/797501600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T18:45:22.583 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:22 smithi112 ceph-mon[27864]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:45:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:22 smithi171 ceph-mon[33266]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:45:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:22 smithi161 ceph-mon[32609]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:45:22.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.754+0000 7fe47e800700 1 -- 172.21.15.112:0/2637153723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe478074c70 msgr2=0x7fe478075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:22.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.754+0000 7fe47e800700 1 --2- 172.21.15.112:0/2637153723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe478074c70 0x7fe478075050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe468009990 tx=0x7fe468009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.754+0000 7fe47e800700 1 -- 172.21.15.112:0/2637153723 shutdown_connections 2024-04-25T18:45:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.754+0000 7fe47e800700 1 --2- 172.21.15.112:0/2637153723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe47810f4a0 0x7fe478111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.754+0000 7fe47e800700 1 --2- 172.21.15.112:0/2637153723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe478074c70 0x7fe478075050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.754+0000 7fe47e800700 1 --2- 172.21.15.112:0/2637153723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe478075590 0x7fe47810ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.754+0000 7fe47e800700 1 -- 172.21.15.112:0/2637153723 >> 172.21.15.112:0/2637153723 conn(0x7fe478079ba0 msgr2=0x7fe478079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.754+0000 7fe47e800700 1 -- 172.21.15.112:0/2637153723 shutdown_connections 2024-04-25T18:45:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.754+0000 7fe47e800700 1 -- 172.21.15.112:0/2637153723 wait complete. 2024-04-25T18:45:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.755+0000 7fe47e800700 1 Processor -- start 2024-04-25T18:45:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.755+0000 7fe47e800700 1 -- start start 2024-04-25T18:45:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.756+0000 7fe47e800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe478074c70 0x7fe47819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:22.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.756+0000 7fe47e800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe478075590 0x7fe47819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.756+0000 7fe47e800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe47810f4a0 0x7fe47819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.756+0000 7fe47e800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe478114ea0 con 0x7fe478074c70 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.756+0000 7fe47e800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe478114d20 con 0x7fe478075590 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.756+0000 7fe47e800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe478115020 con 0x7fe47810f4a0 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.756+0000 7fe477400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe478074c70 0x7fe47819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.756+0000 7fe477400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe478074c70 0x7fe47819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58370/0 (socket says 172.21.15.112:58370) 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.756+0000 7fe477400700 1 -- 172.21.15.112:0/1661051403 learned_addr learned my addr 172.21.15.112:0/1661051403 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.756+0000 7fe477e00700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe47810f4a0 0x7fe47819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.756+0000 7fe46fe00700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe478075590 0x7fe47819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.757+0000 7fe477400700 1 -- 172.21.15.112:0/1661051403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe47810f4a0 msgr2=0x7fe47819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.757+0000 7fe477400700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe47810f4a0 0x7fe47819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.757+0000 7fe477400700 1 -- 172.21.15.112:0/1661051403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe478075590 msgr2=0x7fe47819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:22.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.757+0000 7fe477400700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe478075590 0x7fe47819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.757+0000 7fe477400700 1 -- 172.21.15.112:0/1661051403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4781a0320 con 0x7fe478074c70 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.757+0000 7fe477e00700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe47810f4a0 0x7fe47819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.757+0000 7fe46fe00700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe478075590 0x7fe47819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.757+0000 7fe477400700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe478074c70 0x7fe47819b310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe468000c00 tx=0x7fe4680057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.757+0000 7fe474c00700 1 -- 172.21.15.112:0/1661051403 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468027070 con 0x7fe478074c70 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.757+0000 7fe474c00700 1 -- 172.21.15.112:0/1661051403 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe468003680 con 0x7fe478074c70 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.757+0000 7fe47e800700 1 -- 172.21.15.112:0/1661051403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4781a05b0 con 0x7fe478074c70 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.758+0000 7fe474c00700 1 -- 172.21.15.112:0/1661051403 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468022aa0 con 0x7fe478074c70 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.758+0000 7fe47e800700 1 -- 172.21.15.112:0/1661051403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4781a7e90 con 0x7fe478074c70 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.759+0000 7fe47e800700 1 -- 172.21.15.112:0/1661051403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe478069730 con 0x7fe478074c70 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.761+0000 7fe474c00700 1 -- 172.21.15.112:0/1661051403 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe46802a070 con 0x7fe478074c70 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.763+0000 7fe474c00700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe45c0a3d50 0x7fe45c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:22.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.763+0000 7fe474c00700 1 -- 172.21.15.112:0/1661051403 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fe4680d1550 con 0x7fe478074c70 2024-04-25T18:45:22.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.764+0000 7fe46fe00700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe45c0a3d50 0x7fe45c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:22.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.764+0000 7fe46fe00700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe45c0a3d50 0x7fe45c0a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe478111130 tx=0x7fe46000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:22.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.768+0000 7fe474c00700 1 -- 172.21.15.112:0/1661051403 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe46805eff0 con 0x7fe478074c70 2024-04-25T18:45:23.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.998+0000 7fe47e800700 1 -- 172.21.15.112:0/1661051403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe47802df80 con 0x7fe478074c70 2024-04-25T18:45:23.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:22.999+0000 7fe474c00700 1 -- 172.21.15.112:0/1661051403 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe468013070 con 0x7fe478074c70 2024-04-25T18:45:23.001 INFO:teuthology.orchestra.run.smithi112.stdout:77309411423 2024-04-25T18:45:23.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.002+0000 7fe47e800700 1 -- 172.21.15.112:0/1661051403 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe45c0a3d50 msgr2=0x7fe45c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:23.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.002+0000 7fe47e800700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe45c0a3d50 0x7fe45c0a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe478111130 tx=0x7fe46000b410 comp rx=0 tx=0).stop 2024-04-25T18:45:23.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.002+0000 7fe47e800700 1 -- 172.21.15.112:0/1661051403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe478074c70 msgr2=0x7fe47819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:23.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.002+0000 7fe47e800700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe478074c70 0x7fe47819b310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe468000c00 tx=0x7fe4680057a0 comp rx=0 tx=0).stop 2024-04-25T18:45:23.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.002+0000 7fe47e800700 1 -- 172.21.15.112:0/1661051403 shutdown_connections 2024-04-25T18:45:23.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.002+0000 7fe47e800700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe45c0a3d50 0x7fe45c0a6210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:23.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.002+0000 7fe47e800700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe47810f4a0 0x7fe47819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:23.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.002+0000 7fe47e800700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe478074c70 0x7fe47819b310 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:23.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.002+0000 7fe47e800700 1 --2- 172.21.15.112:0/1661051403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe478075590 0x7fe47819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:23.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.002+0000 7fe47e800700 1 -- 172.21.15.112:0/1661051403 >> 172.21.15.112:0/1661051403 conn(0x7fe478079ba0 msgr2=0x7fe4780ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:23.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.002+0000 7fe47e800700 1 -- 172.21.15.112:0/1661051403 shutdown_connections 2024-04-25T18:45:23.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:23.003+0000 7fe47e800700 1 -- 172.21.15.112:0/1661051403 wait complete. 2024-04-25T18:45:23.460 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:23 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1661051403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T18:45:23.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:23 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1661051403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T18:45:23.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:23 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1661051403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T18:45:24.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:24 smithi112 ceph-mon[27864]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:24.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:24 smithi171 ceph-mon[33266]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:24.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:24 smithi161 ceph-mon[32609]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:25.604 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:25 smithi112 ceph-mon[27864]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:25.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:25 smithi171 ceph-mon[33266]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:25.686 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855438 got 150323855442 for osd.4 2024-04-25T18:45:25.686 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:25.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:25 smithi161 ceph-mon[32609]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:27.348 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:27 smithi112 ceph-mon[27864]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-25T18:45:27.348 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:27 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/432585465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T18:45:27.348 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:27 smithi112 ceph-mon[27864]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:45:27.351 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411418 got 77309411423 for osd.2 2024-04-25T18:45:27.351 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:27.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:27 smithi171 ceph-mon[33266]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-25T18:45:27.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:27 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/432585465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T18:45:27.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:27 smithi171 ceph-mon[33266]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:45:27.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.111+0000 7fa461000700 1 -- 172.21.15.112:0/2318700017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45c102a90 msgr2=0x7fa45c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.111+0000 7fa461000700 1 --2- 172.21.15.112:0/2318700017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45c102a90 0x7fa45c102f10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa448009990 tx=0x7fa448009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.112+0000 7fa461000700 1 -- 172.21.15.112:0/2318700017 shutdown_connections 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.112+0000 7fa461000700 1 --2- 172.21.15.112:0/2318700017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa45c103450 0x7fa45c10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.112+0000 7fa461000700 1 --2- 172.21.15.112:0/2318700017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c108a80 0x7fa45c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.112+0000 7fa461000700 1 --2- 172.21.15.112:0/2318700017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45c102a90 0x7fa45c102f10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.112+0000 7fa461000700 1 -- 172.21.15.112:0/2318700017 >> 172.21.15.112:0/2318700017 conn(0x7fa45c079bf0 msgr2=0x7fa45c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.112+0000 7fa461000700 1 -- 172.21.15.112:0/2318700017 shutdown_connections 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.112+0000 7fa461000700 1 -- 172.21.15.112:0/2318700017 wait complete. 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.113+0000 7fa461000700 1 Processor -- start 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.113+0000 7fa461000700 1 -- start start 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.113+0000 7fa461000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c102a90 0x7fa45c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.113+0000 7fa461000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45c103450 0x7fa45c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.113+0000 7fa461000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa45c108a80 0x7fa45c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.113+0000 7fa461000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c112c30 con 0x7fa45c103450 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.113+0000 7fa461000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c112ab0 con 0x7fa45c102a90 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.113+0000 7fa461000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c112db0 con 0x7fa45c108a80 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa459e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c102a90 0x7fa45c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa459400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45c103450 0x7fa45c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa459e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c102a90 0x7fa45c19b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:41740/0 (socket says 172.21.15.112:41740) 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa459e00700 1 -- 172.21.15.112:0/814122513 learned_addr learned my addr 172.21.15.112:0/814122513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa45a800700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa45c108a80 0x7fa45c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa459e00700 1 -- 172.21.15.112:0/814122513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa45c108a80 msgr2=0x7fa45c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa459e00700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa45c108a80 0x7fa45c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa459e00700 1 -- 172.21.15.112:0/814122513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45c103450 msgr2=0x7fa45c19b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa459e00700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45c103450 0x7fa45c19b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:27.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa459e00700 1 -- 172.21.15.112:0/814122513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45c1a03b0 con 0x7fa45c102a90 2024-04-25T18:45:27.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa459400700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45c103450 0x7fa45c19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:27.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa459e00700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c102a90 0x7fa45c19b3a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00e9a0 tx=0x7fa44c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:27.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.114+0000 7fa452a00700 1 -- 172.21.15.112:0/814122513 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00ca10 con 0x7fa45c102a90 2024-04-25T18:45:27.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.115+0000 7fa461000700 1 -- 172.21.15.112:0/814122513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45c1a06a0 con 0x7fa45c102a90 2024-04-25T18:45:27.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.115+0000 7fa452a00700 1 -- 172.21.15.112:0/814122513 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa44c00cbb0 con 0x7fa45c102a90 2024-04-25T18:45:27.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.115+0000 7fa452a00700 1 -- 172.21.15.112:0/814122513 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c018680 con 0x7fa45c102a90 2024-04-25T18:45:27.687 INFO:teuthology.orchestra.run.smithi112.stdout:231928234051 2024-04-25T18:45:27.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.115+0000 7fa461000700 1 -- 172.21.15.112:0/814122513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45c1a7f20 con 0x7fa45c102a90 2024-04-25T18:45:27.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.116+0000 7fa461000700 1 -- 172.21.15.112:0/814122513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa45c069730 con 0x7fa45c102a90 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.118+0000 7fa452a00700 1 -- 172.21.15.112:0/814122513 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fa44c010430 con 0x7fa45c102a90 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.119+0000 7fa452a00700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4400a3c40 0x7fa4400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.119+0000 7fa452a00700 1 -- 172.21.15.112:0/814122513 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa44c0d0580 con 0x7fa45c102a90 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.119+0000 7fa459400700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4400a3c40 0x7fa4400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.120+0000 7fa459400700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4400a3c40 0x7fa4400a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa44800b2e0 tx=0x7fa44801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.123+0000 7fa452a00700 1 -- 172.21.15.112:0/814122513 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa44c09cee0 con 0x7fa45c102a90 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.359+0000 7fa461000700 1 -- 172.21.15.112:0/814122513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa45c1a80f0 con 0x7fa45c102a90 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.359+0000 7fa452a00700 1 -- 172.21.15.112:0/814122513 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa44c09c7e0 con 0x7fa45c102a90 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 -- 172.21.15.112:0/814122513 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4400a3c40 msgr2=0x7fa4400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4400a3c40 0x7fa4400a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa44800b2e0 tx=0x7fa44801a040 comp rx=0 tx=0).stop 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 -- 172.21.15.112:0/814122513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c102a90 msgr2=0x7fa45c19b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c102a90 0x7fa45c19b3a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00e9a0 tx=0x7fa44c00ecb0 comp rx=0 tx=0).stop 2024-04-25T18:45:27.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 -- 172.21.15.112:0/814122513 shutdown_connections 2024-04-25T18:45:27.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa4400a3c40 0x7fa4400a6100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:27.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa45c108a80 0x7fa45c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:27.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c102a90 0x7fa45c19b3a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:27.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 --2- 172.21.15.112:0/814122513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa45c103450 0x7fa45c19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:27.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 -- 172.21.15.112:0/814122513 >> 172.21.15.112:0/814122513 conn(0x7fa45c079bf0 msgr2=0x7fa45c10df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:27.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 -- 172.21.15.112:0/814122513 shutdown_connections 2024-04-25T18:45:27.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:27.362+0000 7fa461000700 1 -- 172.21.15.112:0/814122513 wait complete. 2024-04-25T18:45:27.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:27 smithi161 ceph-mon[32609]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-25T18:45:27.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:27 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/432585465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T18:45:27.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:27 smithi161 ceph-mon[32609]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:45:28.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.159+0000 7f34aaa00700 1 -- 172.21.15.112:0/2870959849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34a406b070 msgr2=0x7f34a410efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:28.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.159+0000 7f34aaa00700 1 --2- 172.21.15.112:0/2870959849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34a406b070 0x7f34a410efb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3490009990 tx=0x7f3490009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:28.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.159+0000 7f34aaa00700 1 -- 172.21.15.112:0/2870959849 shutdown_connections 2024-04-25T18:45:28.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.159+0000 7f34aaa00700 1 --2- 172.21.15.112:0/2870959849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a410f4f0 0x7f34a41118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:28.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.159+0000 7f34aaa00700 1 --2- 172.21.15.112:0/2870959849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a406a750 0x7f34a406ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.159+0000 7f34aaa00700 1 --2- 172.21.15.112:0/2870959849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34a406b070 0x7f34a410efb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.159+0000 7f34aaa00700 1 -- 172.21.15.112:0/2870959849 >> 172.21.15.112:0/2870959849 conn(0x7f34a40788f0 msgr2=0x7f34a4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.160+0000 7f34aaa00700 1 -- 172.21.15.112:0/2870959849 shutdown_connections 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.160+0000 7f34aaa00700 1 -- 172.21.15.112:0/2870959849 wait complete. 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.161+0000 7f34aaa00700 1 Processor -- start 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.161+0000 7f34aaa00700 1 -- start start 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.161+0000 7f34aaa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a406a750 0x7f34a419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.161+0000 7f34aaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a406b070 0x7f34a419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.161+0000 7f34aaa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34a410f4f0 0x7f34a419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.161+0000 7f34aaa00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4114e50 con 0x7f34a406b070 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.161+0000 7f34aaa00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4114cd0 con 0x7f34a406a750 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.161+0000 7f34aaa00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4114fd0 con 0x7f34a410f4f0 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a406b070 0x7f34a419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a406a750 0x7f34a419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:28.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a406a750 0x7f34a419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:41730/0 (socket says 172.21.15.112:41730) 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a8a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34a410f4f0 0x7f34a419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a406b070 0x7f34a419b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58388/0 (socket says 172.21.15.112:58388) 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3e00700 1 -- 172.21.15.112:0/432585465 learned_addr learned my addr 172.21.15.112:0/432585465 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3e00700 1 -- 172.21.15.112:0/432585465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34a410f4f0 msgr2=0x7f34a419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3e00700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34a410f4f0 0x7f34a419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3e00700 1 -- 172.21.15.112:0/432585465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a406b070 msgr2=0x7f34a419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3e00700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a406b070 0x7f34a419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3e00700 1 -- 172.21.15.112:0/432585465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a41a03a0 con 0x7f34a406a750 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3400700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a406b070 0x7f34a419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a8a00700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34a410f4f0 0x7f34a419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.162+0000 7f34a3e00700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a406a750 0x7f34a419b3e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f349400eb30 tx=0x7f349400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.163+0000 7f34a0c00700 1 -- 172.21.15.112:0/432585465 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349400cbd0 con 0x7f34a406a750 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.163+0000 7f34aaa00700 1 -- 172.21.15.112:0/432585465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a41a0690 con 0x7f34a406a750 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.163+0000 7f34a0c00700 1 -- 172.21.15.112:0/432585465 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34940044d0 con 0x7f34a406a750 2024-04-25T18:45:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.163+0000 7f34a0c00700 1 -- 172.21.15.112:0/432585465 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494003e30 con 0x7f34a406a750 2024-04-25T18:45:28.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.163+0000 7f34aaa00700 1 -- 172.21.15.112:0/432585465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a41a7f10 con 0x7f34a406a750 2024-04-25T18:45:28.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.164+0000 7f34aaa00700 1 -- 172.21.15.112:0/432585465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a4069730 con 0x7f34a406a750 2024-04-25T18:45:28.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.166+0000 7f34a0c00700 1 -- 172.21.15.112:0/432585465 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3494004000 con 0x7f34a406a750 2024-04-25T18:45:28.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.167+0000 7f34a0c00700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f34880a3cd0 0x7f34880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:28.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.168+0000 7f34a0c00700 1 -- 172.21.15.112:0/432585465 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f34940d0b50 con 0x7f34a406a750 2024-04-25T18:45:28.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.168+0000 7f34a3400700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f34880a3cd0 0x7f34880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:28.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.168+0000 7f34a3400700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f34880a3cd0 0x7f34880a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f34a4102ea0 tx=0x7f349000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:28.174 INFO:teuthology.orchestra.run.smithi112.stdout:300647710773 2024-04-25T18:45:28.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.170+0000 7f34a0c00700 1 -- 172.21.15.112:0/432585465 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3494053ff0 con 0x7f34a406a750 2024-04-25T18:45:28.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.280+0000 7f34a0c00700 1 -- 172.21.15.112:0/432585465 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f34940992c0 con 0x7f34a406a750 2024-04-25T18:45:28.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.411+0000 7f34aaa00700 1 -- 172.21.15.112:0/432585465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f34a402df80 con 0x7f34a406a750 2024-04-25T18:45:28.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.412+0000 7f34a0c00700 1 -- 172.21.15.112:0/432585465 <== mon.2 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3494020390 con 0x7f34a406a750 2024-04-25T18:45:28.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.414+0000 7f34aaa00700 1 -- 172.21.15.112:0/432585465 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f34880a3cd0 msgr2=0x7f34880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:28.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.414+0000 7f34aaa00700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f34880a3cd0 0x7f34880a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f34a4102ea0 tx=0x7f349000b260 comp rx=0 tx=0).stop 2024-04-25T18:45:28.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.414+0000 7f34aaa00700 1 -- 172.21.15.112:0/432585465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a406a750 msgr2=0x7f34a419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:28.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.414+0000 7f34aaa00700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a406a750 0x7f34a419b3e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f349400eb30 tx=0x7f349400ee40 comp rx=0 tx=0).stop 2024-04-25T18:45:28.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.415+0000 7f34aaa00700 1 -- 172.21.15.112:0/432585465 shutdown_connections 2024-04-25T18:45:28.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.415+0000 7f34aaa00700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f34880a3cd0 0x7f34880a6190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.415+0000 7f34aaa00700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34a410f4f0 0x7f34a419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.415+0000 7f34aaa00700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a406a750 0x7f34a419b3e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.415+0000 7f34aaa00700 1 --2- 172.21.15.112:0/432585465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a406b070 0x7f34a419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.415+0000 7f34aaa00700 1 -- 172.21.15.112:0/432585465 >> 172.21.15.112:0/432585465 conn(0x7f34a40788f0 msgr2=0x7f34a4110a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.415+0000 7f34aaa00700 1 -- 172.21.15.112:0/432585465 shutdown_connections 2024-04-25T18:45:28.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:26.415+0000 7f34aaa00700 1 -- 172.21.15.112:0/432585465 wait complete. 2024-04-25T18:45:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:28 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/814122513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T18:45:28.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:28 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/814122513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T18:45:28.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:28 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/814122513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T18:45:29.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:29 smithi112 ceph-mon[27864]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T18:45:29.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:29 smithi171 ceph-mon[33266]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T18:45:29.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:29 smithi161 ceph-mon[32609]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T18:45:29.885 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710767 got 300647710773 for osd.9 2024-04-25T18:45:29.885 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:30.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.516+0000 7f5a71400700 1 -- 172.21.15.112:0/3213875723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c103460 msgr2=0x7f5a6c10f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.516+0000 7f5a71400700 1 --2- 172.21.15.112:0/3213875723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c103460 0x7f5a6c10f940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400b0a0 tx=0x7f5a6400b3b0 comp rx=0 tx=0).stop 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.516+0000 7f5a71400700 1 -- 172.21.15.112:0/3213875723 shutdown_connections 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.516+0000 7f5a71400700 1 --2- 172.21.15.112:0/3213875723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c103460 0x7f5a6c10f940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.516+0000 7f5a71400700 1 --2- 172.21.15.112:0/3213875723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a6c108a70 0x7f5a6c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.516+0000 7f5a71400700 1 --2- 172.21.15.112:0/3213875723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a6c102ac0 0x7f5a6c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.516+0000 7f5a71400700 1 -- 172.21.15.112:0/3213875723 >> 172.21.15.112:0/3213875723 conn(0x7f5a6c079b10 msgr2=0x7f5a6c079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.517+0000 7f5a71400700 1 -- 172.21.15.112:0/3213875723 shutdown_connections 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.517+0000 7f5a71400700 1 -- 172.21.15.112:0/3213875723 wait complete. 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a71400700 1 Processor -- start 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a71400700 1 -- start start 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a71400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a6c102ac0 0x7f5a6c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a71400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c103460 0x7f5a6c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a71400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a6c108a70 0x7f5a6c19d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a71400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c112c80 con 0x7f5a6c103460 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a71400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c112b00 con 0x7f5a6c108a70 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a71400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c112e00 con 0x7f5a6c102ac0 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a6a800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a6c102ac0 0x7f5a6c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a69e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c103460 0x7f5a6c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a6a800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a6c102ac0 0x7f5a6c199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:48472/0 (socket says 172.21.15.112:48472) 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a6a800700 1 -- 172.21.15.112:0/273520233 learned_addr learned my addr 172.21.15.112:0/273520233 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.518+0000 7f5a6b200700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a6c108a70 0x7f5a6c19d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.519+0000 7f5a6a800700 1 -- 172.21.15.112:0/273520233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a6c108a70 msgr2=0x7f5a6c19d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.519+0000 7f5a6a800700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a6c108a70 0x7f5a6c19d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.519+0000 7f5a6a800700 1 -- 172.21.15.112:0/273520233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c103460 msgr2=0x7f5a6c1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.519+0000 7f5a6a800700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c103460 0x7f5a6c1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.519+0000 7f5a6a800700 1 -- 172.21.15.112:0/273520233 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c19e130 con 0x7f5a6c102ac0 2024-04-25T18:45:30.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.519+0000 7f5a6a800700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a6c102ac0 0x7f5a6c199170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00e9a0 tx=0x7f5a5c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:30.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.519+0000 7f5a63400700 1 -- 172.21.15.112:0/273520233 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00ca10 con 0x7f5a6c102ac0 2024-04-25T18:45:30.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.520+0000 7f5a63400700 1 -- 172.21.15.112:0/273520233 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a5c00cbb0 con 0x7f5a6c102ac0 2024-04-25T18:45:30.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.520+0000 7f5a71400700 1 -- 172.21.15.112:0/273520233 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c19e420 con 0x7f5a6c102ac0 2024-04-25T18:45:30.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.520+0000 7f5a63400700 1 -- 172.21.15.112:0/273520233 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c018680 con 0x7f5a6c102ac0 2024-04-25T18:45:30.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.520+0000 7f5a71400700 1 -- 172.21.15.112:0/273520233 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c1a5ca0 con 0x7f5a6c102ac0 2024-04-25T18:45:30.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.521+0000 7f5a71400700 1 -- 172.21.15.112:0/273520233 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6c069730 con 0x7f5a6c102ac0 2024-04-25T18:45:30.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.523+0000 7f5a63400700 1 -- 172.21.15.112:0/273520233 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f5a5c010430 con 0x7f5a6c102ac0 2024-04-25T18:45:30.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.524+0000 7f5a63400700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5a500a3cc0 0x7f5a500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:30.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.525+0000 7f5a63400700 1 -- 172.21.15.112:0/273520233 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f5a5c0d0590 con 0x7f5a6c102ac0 2024-04-25T18:45:30.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.525+0000 7f5a69e00700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5a500a3cc0 0x7f5a500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:30.198 INFO:teuthology.orchestra.run.smithi112.stdout:55834574949 2024-04-25T18:45:30.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.525+0000 7f5a69e00700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5a500a3cc0 0x7f5a500a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c1028f0 tx=0x7f5a58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:30.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.527+0000 7f5a63400700 1 -- 172.21.15.112:0/273520233 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a5c09cef0 con 0x7f5a6c102ac0 2024-04-25T18:45:30.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.764+0000 7f5a71400700 1 -- 172.21.15.112:0/273520233 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5a6c02df80 con 0x7f5a6c102ac0 2024-04-25T18:45:30.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.765+0000 7f5a63400700 1 -- 172.21.15.112:0/273520233 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5a5c09c7f0 con 0x7f5a6c102ac0 2024-04-25T18:45:30.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 -- 172.21.15.112:0/273520233 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5a500a3cc0 msgr2=0x7f5a500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:30.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5a500a3cc0 0x7f5a500a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c1028f0 tx=0x7f5a58009260 comp rx=0 tx=0).stop 2024-04-25T18:45:30.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 -- 172.21.15.112:0/273520233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a6c102ac0 msgr2=0x7f5a6c199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:30.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a6c102ac0 0x7f5a6c199170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00e9a0 tx=0x7f5a5c00ecb0 comp rx=0 tx=0).stop 2024-04-25T18:45:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 -- 172.21.15.112:0/273520233 shutdown_connections 2024-04-25T18:45:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f5a500a3cc0 0x7f5a500a6180 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a6c108a70 0x7f5a6c19d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a6c102ac0 0x7f5a6c199170 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 --2- 172.21.15.112:0/273520233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a6c103460 0x7f5a6c1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 -- 172.21.15.112:0/273520233 >> 172.21.15.112:0/273520233 conn(0x7f5a6c079b10 msgr2=0x7f5a6c100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 -- 172.21.15.112:0/273520233 shutdown_connections 2024-04-25T18:45:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:29.768+0000 7f5a71400700 1 -- 172.21.15.112:0/273520233 wait complete. 2024-04-25T18:45:30.406 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:30 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/273520233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T18:45:30.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:30 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/273520233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T18:45:30.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:30 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/273520233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T18:45:31.239 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574943 got 55834574949 for osd.1 2024-04-25T18:45:31.239 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:31.622 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:31 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2203259390' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T18:45:31.622 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:31 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1743924120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T18:45:31.622 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:31 smithi112 ceph-mon[27864]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T18:45:31.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:31 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2203259390' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T18:45:31.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:31 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1743924120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T18:45:31.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:31 smithi171 ceph-mon[33266]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T18:45:31.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:31 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2203259390' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T18:45:31.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:31 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1743924120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T18:45:31.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:31 smithi161 ceph-mon[32609]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T18:45:31.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.566+0000 7efc2c400700 1 -- 172.21.15.112:0/3744624139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc24074c70 msgr2=0x7efc24075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.566+0000 7efc2c400700 1 --2- 172.21.15.112:0/3744624139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc24074c70 0x7efc24075050 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc18009990 tx=0x7efc18009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.566+0000 7efc2c400700 1 -- 172.21.15.112:0/3744624139 shutdown_connections 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.566+0000 7efc2c400700 1 --2- 172.21.15.112:0/3744624139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc2410f440 0x7efc24111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.566+0000 7efc2c400700 1 --2- 172.21.15.112:0/3744624139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc24074c70 0x7efc24075050 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.566+0000 7efc2c400700 1 --2- 172.21.15.112:0/3744624139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc24075590 0x7efc2410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.566+0000 7efc2c400700 1 -- 172.21.15.112:0/3744624139 >> 172.21.15.112:0/3744624139 conn(0x7efc24079bf0 msgr2=0x7efc2407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.566+0000 7efc2c400700 1 -- 172.21.15.112:0/3744624139 shutdown_connections 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.566+0000 7efc2c400700 1 -- 172.21.15.112:0/3744624139 wait complete. 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.567+0000 7efc2c400700 1 Processor -- start 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc2c400700 1 -- start start 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc2c400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc24074c70 0x7efc2419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:31.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc2c400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc24075590 0x7efc2419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc2c400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc2410f440 0x7efc2419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc2c400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc24114e40 con 0x7efc2410f440 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc2c400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efc24114cc0 con 0x7efc24074c70 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc2c400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efc24114fc0 con 0x7efc24075590 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc2a400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc2410f440 0x7efc2419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc29a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc24074c70 0x7efc2419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc29a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc24074c70 0x7efc2419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:42968/0 (socket says 172.21.15.112:42968) 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc2a400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc2410f440 0x7efc2419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56328/0 (socket says 172.21.15.112:56328) 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc29a00700 1 -- 172.21.15.112:0/2203259390 learned_addr learned my addr 172.21.15.112:0/2203259390 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.568+0000 7efc29000700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc24075590 0x7efc2419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.569+0000 7efc29a00700 1 -- 172.21.15.112:0/2203259390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc24075590 msgr2=0x7efc2419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.569+0000 7efc29a00700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc24075590 0x7efc2419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.569+0000 7efc29a00700 1 -- 172.21.15.112:0/2203259390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc2410f440 msgr2=0x7efc2419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.569+0000 7efc29a00700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc2410f440 0x7efc2419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:31.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.569+0000 7efc29a00700 1 -- 172.21.15.112:0/2203259390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc241a0370 con 0x7efc24074c70 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.569+0000 7efc29000700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc24075590 0x7efc2419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.569+0000 7efc2a400700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc2410f440 0x7efc2419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.569+0000 7efc29a00700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc24074c70 0x7efc2419b3b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc180057f0 tx=0x7efc1800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.569+0000 7efc22000700 1 -- 172.21.15.112:0/2203259390 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18026070 con 0x7efc24074c70 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.570+0000 7efc22000700 1 -- 172.21.15.112:0/2203259390 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc18021410 con 0x7efc24074c70 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.570+0000 7efc22000700 1 -- 172.21.15.112:0/2203259390 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18017650 con 0x7efc24074c70 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.570+0000 7efc2c400700 1 -- 172.21.15.112:0/2203259390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc241a0600 con 0x7efc24074c70 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.570+0000 7efc2c400700 1 -- 172.21.15.112:0/2203259390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc241a7ee0 con 0x7efc24074c70 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.571+0000 7efc2c400700 1 -- 172.21.15.112:0/2203259390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc24076390 con 0x7efc24074c70 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.573+0000 7efc22000700 1 -- 172.21.15.112:0/2203259390 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7efc18029070 con 0x7efc24074c70 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.574+0000 7efc22000700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7efc0c0a3cc0 0x7efc0c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.574+0000 7efc22000700 1 -- 172.21.15.112:0/2203259390 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7efc180d1160 con 0x7efc24074c70 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.574+0000 7efc29000700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7efc0c0a3cc0 0x7efc0c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.575+0000 7efc29000700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7efc0c0a3cc0 0x7efc0c0a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efc24102640 tx=0x7efc10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:31.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.578+0000 7efc22000700 1 -- 172.21.15.112:0/2203259390 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc1805dff0 con 0x7efc24074c70 2024-04-25T18:45:31.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.825+0000 7efc2c400700 1 -- 172.21.15.112:0/2203259390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7efc2402df80 con 0x7efc24074c70 2024-04-25T18:45:31.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.825+0000 7efc22000700 1 -- 172.21.15.112:0/2203259390 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc18013070 con 0x7efc24074c70 2024-04-25T18:45:31.774 INFO:teuthology.orchestra.run.smithi112.stdout:34359738473 2024-04-25T18:45:31.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 -- 172.21.15.112:0/2203259390 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7efc0c0a3cc0 msgr2=0x7efc0c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:31.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7efc0c0a3cc0 0x7efc0c0a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efc24102640 tx=0x7efc10009260 comp rx=0 tx=0).stop 2024-04-25T18:45:31.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 -- 172.21.15.112:0/2203259390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc24074c70 msgr2=0x7efc2419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:31.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc24074c70 0x7efc2419b3b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc180057f0 tx=0x7efc1800bfd0 comp rx=0 tx=0).stop 2024-04-25T18:45:31.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 -- 172.21.15.112:0/2203259390 shutdown_connections 2024-04-25T18:45:31.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7efc0c0a3cc0 0x7efc0c0a6180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:31.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc2410f440 0x7efc2419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:31.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc24074c70 0x7efc2419b3b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:31.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 --2- 172.21.15.112:0/2203259390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc24075590 0x7efc2419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:31.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 -- 172.21.15.112:0/2203259390 >> 172.21.15.112:0/2203259390 conn(0x7efc24079bf0 msgr2=0x7efc240ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:31.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 -- 172.21.15.112:0/2203259390 shutdown_connections 2024-04-25T18:45:31.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.828+0000 7efc2c400700 1 -- 172.21.15.112:0/2203259390 wait complete. 2024-04-25T18:45:32.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.682+0000 7fd5ca000700 1 -- 172.21.15.112:0/2819541861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5c406ac90 msgr2=0x7fd5c406b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:32.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.682+0000 7fd5ca000700 1 --2- 172.21.15.112:0/2819541861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5c406ac90 0x7fd5c406b0f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009990 tx=0x7fd5b0009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:32.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.682+0000 7fd5ca000700 1 -- 172.21.15.112:0/2819541861 shutdown_connections 2024-04-25T18:45:32.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.682+0000 7fd5ca000700 1 --2- 172.21.15.112:0/2819541861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c4105480 0x7fd5c4105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:32.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.682+0000 7fd5ca000700 1 --2- 172.21.15.112:0/2819541861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c4107970 0x7fd5c406a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:32.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.682+0000 7fd5ca000700 1 --2- 172.21.15.112:0/2819541861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5c406ac90 0x7fd5c406b0f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:32.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.682+0000 7fd5ca000700 1 -- 172.21.15.112:0/2819541861 >> 172.21.15.112:0/2819541861 conn(0x7fd5c40788f0 msgr2=0x7fd5c4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:32.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.682+0000 7fd5ca000700 1 -- 172.21.15.112:0/2819541861 shutdown_connections 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.682+0000 7fd5ca000700 1 -- 172.21.15.112:0/2819541861 wait complete. 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.684+0000 7fd5ca000700 1 Processor -- start 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.684+0000 7fd5ca000700 1 -- start start 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.685+0000 7fd5ca000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c406ac90 0x7fd5c419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.685+0000 7fd5ca000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c4105480 0x7fd5c419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.685+0000 7fd5ca000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5c4107970 0x7fd5c41a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.685+0000 7fd5ca000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c4119260 con 0x7fd5c406ac90 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.685+0000 7fd5ca000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41190e0 con 0x7fd5c4107970 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.685+0000 7fd5ca000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41193e0 con 0x7fd5c4105480 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.685+0000 7fd5c2a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c4105480 0x7fd5c419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.685+0000 7fd5c2a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c4105480 0x7fd5c419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:49872/0 (socket says 172.21.15.112:49872) 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.686+0000 7fd5c3e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5c4107970 0x7fd5c41a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.686+0000 7fd5c2a00700 1 -- 172.21.15.112:0/1743924120 learned_addr learned my addr 172.21.15.112:0/1743924120 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.686+0000 7fd5c3e00700 1 -- 172.21.15.112:0/1743924120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c4105480 msgr2=0x7fd5c419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.686+0000 7fd5c3e00700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c4105480 0x7fd5c419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.686+0000 7fd5c3e00700 1 -- 172.21.15.112:0/1743924120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c406ac90 msgr2=0x7fd5c419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.686+0000 7fd5c3e00700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c406ac90 0x7fd5c419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.686+0000 7fd5c3e00700 1 -- 172.21.15.112:0/1743924120 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c41a4760 con 0x7fd5c4107970 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.686+0000 7fd5c3e00700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5c4107970 0x7fd5c41a4020 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00efc0 tx=0x7fd5bc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.687+0000 7fd5bbe00700 1 -- 172.21.15.112:0/1743924120 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0092e0 con 0x7fd5c4107970 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.687+0000 7fd5bbe00700 1 -- 172.21.15.112:0/1743924120 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5bc004bb0 con 0x7fd5c4107970 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.687+0000 7fd5ca000700 1 -- 172.21.15.112:0/1743924120 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c41a49f0 con 0x7fd5c4107970 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.687+0000 7fd5bbe00700 1 -- 172.21.15.112:0/1743924120 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc004080 con 0x7fd5c4107970 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.687+0000 7fd5ca000700 1 -- 172.21.15.112:0/1743924120 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c41ac2d0 con 0x7fd5c4107970 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.689+0000 7fd5ca000700 1 -- 172.21.15.112:0/1743924120 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c4069730 con 0x7fd5c4107970 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.691+0000 7fd5bbe00700 1 -- 172.21.15.112:0/1743924120 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fd5bc01b030 con 0x7fd5c4107970 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.692+0000 7fd5bbe00700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fd5a80a3d10 0x7fd5a80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.692+0000 7fd5bbe00700 1 -- 172.21.15.112:0/1743924120 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fd5bc0cffa0 con 0x7fd5c4107970 2024-04-25T18:45:32.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.693+0000 7fd5c3400700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fd5a80a3d10 0x7fd5a80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:32.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.693+0000 7fd5c3400700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fd5a80a3d10 0x7fd5a80a61d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009b80 tx=0x7fd5b4009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:32.159 INFO:teuthology.orchestra.run.smithi112.stdout:171798691918 2024-04-25T18:45:32.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.697+0000 7fd5bbe00700 1 -- 172.21.15.112:0/1743924120 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5bc022020 con 0x7fd5c4107970 2024-04-25T18:45:32.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.955+0000 7fd5ca000700 1 -- 172.21.15.112:0/1743924120 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd5c41ac4a0 con 0x7fd5c4107970 2024-04-25T18:45:32.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.956+0000 7fd5bbe00700 1 -- 172.21.15.112:0/1743924120 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd5bc010040 con 0x7fd5c4107970 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.959+0000 7fd5ca000700 1 -- 172.21.15.112:0/1743924120 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fd5a80a3d10 msgr2=0x7fd5a80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.959+0000 7fd5ca000700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fd5a80a3d10 0x7fd5a80a61d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009b80 tx=0x7fd5b4009580 comp rx=0 tx=0).stop 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.959+0000 7fd5ca000700 1 -- 172.21.15.112:0/1743924120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5c4107970 msgr2=0x7fd5c41a4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.959+0000 7fd5ca000700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5c4107970 0x7fd5c41a4020 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00efc0 tx=0x7fd5bc00c3a0 comp rx=0 tx=0).stop 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.959+0000 7fd5ca000700 1 -- 172.21.15.112:0/1743924120 shutdown_connections 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.959+0000 7fd5ca000700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fd5a80a3d10 0x7fd5a80a61d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.959+0000 7fd5ca000700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5c4107970 0x7fd5c41a4020 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.959+0000 7fd5ca000700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c406ac90 0x7fd5c419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.959+0000 7fd5ca000700 1 --2- 172.21.15.112:0/1743924120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c4105480 0x7fd5c419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.959+0000 7fd5ca000700 1 -- 172.21.15.112:0/1743924120 >> 172.21.15.112:0/1743924120 conn(0x7fd5c40788f0 msgr2=0x7fd5c4104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.959+0000 7fd5ca000700 1 -- 172.21.15.112:0/1743924120 shutdown_connections 2024-04-25T18:45:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:30.960+0000 7fd5ca000700 1 -- 172.21.15.112:0/1743924120 wait complete. 2024-04-25T18:45:33.338 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691914 got 171798691918 for osd.5 2024-04-25T18:45:33.338 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:33.936 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738466 got 34359738473 for osd.0 2024-04-25T18:45:33.937 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:34.462 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:34 smithi112 ceph-mon[27864]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T18:45:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:34 smithi161 ceph-mon[32609]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T18:45:34.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:34 smithi171 ceph-mon[33266]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T18:45:35.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.119+0000 7fa698400700 1 -- 172.21.15.112:0/3192032985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa690108a80 msgr2=0x7fa690108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:35.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.119+0000 7fa698400700 1 --2- 172.21.15.112:0/3192032985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa690108a80 0x7fa690108e60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa684009990 tx=0x7fa684009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.120+0000 7fa698400700 1 -- 172.21.15.112:0/3192032985 shutdown_connections 2024-04-25T18:45:35.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.120+0000 7fa698400700 1 --2- 172.21.15.112:0/3192032985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa690103450 0x7fa69010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.120+0000 7fa698400700 1 --2- 172.21.15.112:0/3192032985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa690108a80 0x7fa690108e60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.120+0000 7fa698400700 1 --2- 172.21.15.112:0/3192032985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa690102a90 0x7fa690102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.120+0000 7fa698400700 1 -- 172.21.15.112:0/3192032985 >> 172.21.15.112:0/3192032985 conn(0x7fa690079bf0 msgr2=0x7fa69007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:35.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.120+0000 7fa698400700 1 -- 172.21.15.112:0/3192032985 shutdown_connections 2024-04-25T18:45:35.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.120+0000 7fa698400700 1 -- 172.21.15.112:0/3192032985 wait complete. 2024-04-25T18:45:35.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.121+0000 7fa698400700 1 Processor -- start 2024-04-25T18:45:35.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.121+0000 7fa698400700 1 -- start start 2024-04-25T18:45:35.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.121+0000 7fa698400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa690102a90 0x7fa690199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:35.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.122+0000 7fa698400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa690103450 0x7fa6901996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:35.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.122+0000 7fa698400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa690108a80 0x7fa69019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:35.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.122+0000 7fa698400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa690112c10 con 0x7fa690103450 2024-04-25T18:45:35.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.122+0000 7fa698400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa690112a90 con 0x7fa690102a90 2024-04-25T18:45:35.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.122+0000 7fa698400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa690112d90 con 0x7fa690108a80 2024-04-25T18:45:35.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.122+0000 7fa695000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa690103450 0x7fa6901996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:35.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.122+0000 7fa695a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa690102a90 0x7fa690199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:35.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.122+0000 7fa695a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa690102a90 0x7fa690199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:43026/0 (socket says 172.21.15.112:43026) 2024-04-25T18:45:35.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.122+0000 7fa695a00700 1 -- 172.21.15.112:0/44742398 learned_addr learned my addr 172.21.15.112:0/44742398 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:35.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.122+0000 7fa696400700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa690108a80 0x7fa69019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:35.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.123+0000 7fa695000700 1 -- 172.21.15.112:0/44742398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa690108a80 msgr2=0x7fa69019da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:35.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.123+0000 7fa695000700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa690108a80 0x7fa69019da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.123+0000 7fa695000700 1 -- 172.21.15.112:0/44742398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa690102a90 msgr2=0x7fa690199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:35.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.123+0000 7fa695000700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa690102a90 0x7fa690199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.123+0000 7fa695000700 1 -- 172.21.15.112:0/44742398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69019e170 con 0x7fa690103450 2024-04-25T18:45:35.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.123+0000 7fa695000700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa690103450 0x7fa6901996a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00e9f0 tx=0x7fa67c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:35.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.123+0000 7fa68e000700 1 -- 172.21.15.112:0/44742398 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00cbd0 con 0x7fa690103450 2024-04-25T18:45:35.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.124+0000 7fa698400700 1 -- 172.21.15.112:0/44742398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69019e460 con 0x7fa690103450 2024-04-25T18:45:35.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.124+0000 7fa698400700 1 -- 172.21.15.112:0/44742398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa690074c70 con 0x7fa690103450 2024-04-25T18:45:35.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.124+0000 7fa68e000700 1 -- 172.21.15.112:0/44742398 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa67c00cd70 con 0x7fa690103450 2024-04-25T18:45:35.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.124+0000 7fa68e000700 1 -- 172.21.15.112:0/44742398 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c018680 con 0x7fa690103450 2024-04-25T18:45:35.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.126+0000 7fa698400700 1 -- 172.21.15.112:0/44742398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa670005330 con 0x7fa690103450 2024-04-25T18:45:35.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.128+0000 7fa68e000700 1 -- 172.21.15.112:0/44742398 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fa67c010430 con 0x7fa690103450 2024-04-25T18:45:35.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.131+0000 7fa68e000700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6780ac4d0 0x7fa6780ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:35.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.131+0000 7fa68e000700 1 -- 172.21.15.112:0/44742398 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa67c0d04a0 con 0x7fa690103450 2024-04-25T18:45:35.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.132+0000 7fa695a00700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6780ac4d0 0x7fa6780ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:35.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.133+0000 7fa695a00700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6780ac4d0 0x7fa6780ae990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa6840059d0 tx=0x7fa68401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:35.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.136+0000 7fa68e000700 1 -- 172.21.15.112:0/44742398 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa67c09ce60 con 0x7fa690103450 2024-04-25T18:45:35.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.384+0000 7fa698400700 1 -- 172.21.15.112:0/44742398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa670005620 con 0x7fa690103450 2024-04-25T18:45:35.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.384+0000 7fa68e000700 1 -- 172.21.15.112:0/44742398 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa67c09c760 con 0x7fa690103450 2024-04-25T18:45:35.386 INFO:teuthology.orchestra.run.smithi112.stdout:206158430281 2024-04-25T18:45:35.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 -- 172.21.15.112:0/44742398 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6780ac4d0 msgr2=0x7fa6780ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:35.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6780ac4d0 0x7fa6780ae990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa6840059d0 tx=0x7fa68401a040 comp rx=0 tx=0).stop 2024-04-25T18:45:35.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 -- 172.21.15.112:0/44742398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa690103450 msgr2=0x7fa6901996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:35.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa690103450 0x7fa6901996a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00e9f0 tx=0x7fa67c00ed00 comp rx=0 tx=0).stop 2024-04-25T18:45:35.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 -- 172.21.15.112:0/44742398 shutdown_connections 2024-04-25T18:45:35.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa6780ac4d0 0x7fa6780ae990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa690108a80 0x7fa69019da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa690102a90 0x7fa690199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 --2- 172.21.15.112:0/44742398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa690103450 0x7fa6901996a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 -- 172.21.15.112:0/44742398 >> 172.21.15.112:0/44742398 conn(0x7fa690079bf0 msgr2=0x7fa690101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:35.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 -- 172.21.15.112:0/44742398 shutdown_connections 2024-04-25T18:45:35.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.387+0000 7fa698400700 1 -- 172.21.15.112:0/44742398 wait complete. 2024-04-25T18:45:35.798 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234045 got 231928234051 for osd.7 2024-04-25T18:45:35.798 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.978+0000 7f4857000700 1 -- 172.21.15.112:0/1551999115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f485006b070 msgr2=0x7f485010efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.978+0000 7f4857000700 1 --2- 172.21.15.112:0/1551999115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f485006b070 0x7f485010efb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f483c0098f0 tx=0x7f483c009c00 comp rx=0 tx=0).stop 2024-04-25T18:45:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.978+0000 7f4857000700 1 -- 172.21.15.112:0/1551999115 shutdown_connections 2024-04-25T18:45:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.978+0000 7f4857000700 1 --2- 172.21.15.112:0/1551999115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485010f4f0 0x7f48501118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.978+0000 7f4857000700 1 --2- 172.21.15.112:0/1551999115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f485006a750 0x7f485006ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.978+0000 7f4857000700 1 --2- 172.21.15.112:0/1551999115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f485006b070 0x7f485010efb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.978+0000 7f4857000700 1 -- 172.21.15.112:0/1551999115 >> 172.21.15.112:0/1551999115 conn(0x7f48500788f0 msgr2=0x7f4850078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:35.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.978+0000 7f4857000700 1 -- 172.21.15.112:0/1551999115 shutdown_connections 2024-04-25T18:45:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.978+0000 7f4857000700 1 -- 172.21.15.112:0/1551999115 wait complete. 2024-04-25T18:45:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.979+0000 7f4857000700 1 Processor -- start 2024-04-25T18:45:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.979+0000 7f4857000700 1 -- start start 2024-04-25T18:45:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f4857000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485006a750 0x7f485019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f4857000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f485006b070 0x7f485019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f4857000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f485010f4f0 0x7f485019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f4857000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4850114e00 con 0x7f485006a750 2024-04-25T18:45:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f4857000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4850114c80 con 0x7f485006b070 2024-04-25T18:45:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f4857000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4850114f80 con 0x7f485010f4f0 2024-04-25T18:45:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f484fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485006a750 0x7f485019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:35.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f4855000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f485010f4f0 0x7f485019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f4855000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f485010f4f0 0x7f485019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:49894/0 (socket says 172.21.15.112:49894) 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f4855000700 1 -- 172.21.15.112:0/3918665419 learned_addr learned my addr 172.21.15.112:0/3918665419 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f484fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485006a750 0x7f485019b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56376/0 (socket says 172.21.15.112:56376) 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.980+0000 7f484f400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f485006b070 0x7f485019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.981+0000 7f4855000700 1 -- 172.21.15.112:0/3918665419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f485006b070 msgr2=0x7f485019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.981+0000 7f4855000700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f485006b070 0x7f485019b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.981+0000 7f4855000700 1 -- 172.21.15.112:0/3918665419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485006a750 msgr2=0x7f485019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.981+0000 7f4855000700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485006a750 0x7f485019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.981+0000 7f4855000700 1 -- 172.21.15.112:0/3918665419 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48501a0410 con 0x7f485010f4f0 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.981+0000 7f484fe00700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485006a750 0x7f485019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.981+0000 7f4855000700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f485010f4f0 0x7f485019fcd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f484800ef90 tx=0x7f484800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:35.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.981+0000 7f484cc00700 1 -- 172.21.15.112:0/3918665419 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48480092e0 con 0x7f485010f4f0 2024-04-25T18:45:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.981+0000 7f484cc00700 1 -- 172.21.15.112:0/3918665419 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4848004bb0 con 0x7f485010f4f0 2024-04-25T18:45:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.981+0000 7f4857000700 1 -- 172.21.15.112:0/3918665419 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48501a06a0 con 0x7f485010f4f0 2024-04-25T18:45:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.982+0000 7f484cc00700 1 -- 172.21.15.112:0/3918665419 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848004280 con 0x7f485010f4f0 2024-04-25T18:45:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.982+0000 7f4857000700 1 -- 172.21.15.112:0/3918665419 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48501a7f80 con 0x7f485010f4f0 2024-04-25T18:45:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.983+0000 7f4857000700 1 -- 172.21.15.112:0/3918665419 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4850101f60 con 0x7f485010f4f0 2024-04-25T18:45:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.985+0000 7f484cc00700 1 -- 172.21.15.112:0/3918665419 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f484801b030 con 0x7f485010f4f0 2024-04-25T18:45:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.986+0000 7f484cc00700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48340a3c40 0x7f48340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.986+0000 7f484cc00700 1 -- 172.21.15.112:0/3918665419 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f48480cff20 con 0x7f485010f4f0 2024-04-25T18:45:35.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.986+0000 7f484fe00700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48340a3c40 0x7f48340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:35.865 INFO:teuthology.orchestra.run.smithi112.stdout:360777252910 2024-04-25T18:45:35.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.987+0000 7f484fe00700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48340a3c40 0x7f48340a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4840009750 tx=0x7f4840009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:35.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:34.991+0000 7f484cc00700 1 -- 172.21.15.112:0/3918665419 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4848022020 con 0x7f485010f4f0 2024-04-25T18:45:35.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.239+0000 7f4857000700 1 -- 172.21.15.112:0/3918665419 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f485002df80 con 0x7f485010f4f0 2024-04-25T18:45:35.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.240+0000 7f484cc00700 1 -- 172.21.15.112:0/3918665419 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4848010040 con 0x7f485010f4f0 2024-04-25T18:45:35.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.242+0000 7f4857000700 1 -- 172.21.15.112:0/3918665419 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48340a3c40 msgr2=0x7f48340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:35.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.242+0000 7f4857000700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48340a3c40 0x7f48340a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4840009750 tx=0x7f4840009310 comp rx=0 tx=0).stop 2024-04-25T18:45:35.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.242+0000 7f4857000700 1 -- 172.21.15.112:0/3918665419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f485010f4f0 msgr2=0x7f485019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:35.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.242+0000 7f4857000700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f485010f4f0 0x7f485019fcd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f484800ef90 tx=0x7f484800c3a0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.242+0000 7f4857000700 1 -- 172.21.15.112:0/3918665419 shutdown_connections 2024-04-25T18:45:35.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.242+0000 7f4857000700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f48340a3c40 0x7f48340a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.243+0000 7f4857000700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f485010f4f0 0x7f485019fcd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.243+0000 7f4857000700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485006a750 0x7f485019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.243+0000 7f4857000700 1 --2- 172.21.15.112:0/3918665419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f485006b070 0x7f485019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:35.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.243+0000 7f4857000700 1 -- 172.21.15.112:0/3918665419 >> 172.21.15.112:0/3918665419 conn(0x7f48500788f0 msgr2=0x7f4850110b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:35.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.243+0000 7f4857000700 1 -- 172.21.15.112:0/3918665419 shutdown_connections 2024-04-25T18:45:35.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:35.243+0000 7f4857000700 1 -- 172.21.15.112:0/3918665419 wait complete. 2024-04-25T18:45:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:36 smithi161 ceph-mon[32609]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T18:45:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:36 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3918665419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T18:45:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:36 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/44742398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T18:45:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:36 smithi112 ceph-mon[27864]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T18:45:36.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:36 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3918665419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T18:45:36.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:36 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/44742398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T18:45:36.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:36 smithi171 ceph-mon[33266]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T18:45:36.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:36 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3918665419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T18:45:36.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:36 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/44742398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T18:45:36.772 INFO:tasks.cephadm.ceph_manager.ceph:need seq 360777252902 got 360777252910 for osd.11 2024-04-25T18:45:36.772 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:36.794 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:38 smithi161 ceph-mon[32609]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:45:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:38 smithi112 ceph-mon[27864]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:45:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:38 smithi171 ceph-mon[33266]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:45:39.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.107+0000 7f9ddca00700 1 -- 172.21.15.112:0/3861566720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dd8109f90 msgr2=0x7f9dd8111b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:39.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.107+0000 7f9ddca00700 1 --2- 172.21.15.112:0/3861566720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dd8109f90 0x7f9dd8111b60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00099c0 tx=0x7f9dd0009cd0 comp rx=0 tx=0).stop 2024-04-25T18:45:39.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.107+0000 7f9ddca00700 1 -- 172.21.15.112:0/3861566720 shutdown_connections 2024-04-25T18:45:39.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.107+0000 7f9ddca00700 1 --2- 172.21.15.112:0/3861566720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dd8109f90 0x7f9dd8111b60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:39.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.107+0000 7f9ddca00700 1 --2- 172.21.15.112:0/3861566720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd8104eb0 0x7f9dd8105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:39.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.107+0000 7f9ddca00700 1 --2- 172.21.15.112:0/3861566720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd8105860 0x7f9dd81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:39.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.107+0000 7f9ddca00700 1 -- 172.21.15.112:0/3861566720 >> 172.21.15.112:0/3861566720 conn(0x7f9dd8100740 msgr2=0x7f9dd8102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:39.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.107+0000 7f9ddca00700 1 -- 172.21.15.112:0/3861566720 shutdown_connections 2024-04-25T18:45:39.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.108+0000 7f9ddca00700 1 -- 172.21.15.112:0/3861566720 wait complete. 2024-04-25T18:45:39.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9ddca00700 1 Processor -- start 2024-04-25T18:45:39.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9ddca00700 1 -- start start 2024-04-25T18:45:39.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9ddca00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dd8104eb0 0x7f9dd81119c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:39.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9ddca00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd8105860 0x7f9dd810eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:39.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9ddca00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd8109f90 0x7f9dd810f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9ddca00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd8114e20 con 0x7f9dd8104eb0 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9ddca00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd8114ca0 con 0x7f9dd8105860 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9ddca00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd8114fa0 con 0x7f9dd8109f90 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9dd5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dd8104eb0 0x7f9dd81119c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9dd6800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd8109f90 0x7f9dd810f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9dd5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dd8104eb0 0x7f9dd81119c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56418/0 (socket says 172.21.15.112:56418) 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9dd5e00700 1 -- 172.21.15.112:0/3480633708 learned_addr learned my addr 172.21.15.112:0/3480633708 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.109+0000 7f9dd5400700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd8105860 0x7f9dd810eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.110+0000 7f9dd5e00700 1 -- 172.21.15.112:0/3480633708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd8109f90 msgr2=0x7f9dd810f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.110+0000 7f9dd5e00700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd8109f90 0x7f9dd810f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.110+0000 7f9dd5e00700 1 -- 172.21.15.112:0/3480633708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd8105860 msgr2=0x7f9dd810eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.110+0000 7f9dd5e00700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd8105860 0x7f9dd810eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.110+0000 7f9dd5e00700 1 -- 172.21.15.112:0/3480633708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd810f8d0 con 0x7f9dd8104eb0 2024-04-25T18:45:39.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.110+0000 7f9dd5400700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd8105860 0x7f9dd810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:39.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.110+0000 7f9dd5e00700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dd8104eb0 0x7f9dd81119c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800ef10 tx=0x7f9dc800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:39.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.110+0000 7f9dcea00700 1 -- 172.21.15.112:0/3480633708 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc800c9c0 con 0x7f9dd8104eb0 2024-04-25T18:45:39.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.110+0000 7f9dcea00700 1 -- 172.21.15.112:0/3480633708 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc8019070 con 0x7f9dd8104eb0 2024-04-25T18:45:39.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.110+0000 7f9ddca00700 1 -- 172.21.15.112:0/3480633708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd81a1400 con 0x7f9dd8104eb0 2024-04-25T18:45:39.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.111+0000 7f9dcea00700 1 -- 172.21.15.112:0/3480633708 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8014480 con 0x7f9dd8104eb0 2024-04-25T18:45:39.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.111+0000 7f9ddca00700 1 -- 172.21.15.112:0/3480633708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd81a1910 con 0x7f9dd8104eb0 2024-04-25T18:45:39.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.112+0000 7f9ddca00700 1 -- 172.21.15.112:0/3480633708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd8106390 con 0x7f9dd8104eb0 2024-04-25T18:45:39.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.114+0000 7f9dcea00700 1 -- 172.21.15.112:0/3480633708 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f9dc8012050 con 0x7f9dd8104eb0 2024-04-25T18:45:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.116+0000 7f9dcea00700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dbc0a3c40 0x7f9dbc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.116+0000 7f9dcea00700 1 -- 172.21.15.112:0/3480633708 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f9dc80d0330 con 0x7f9dd8104eb0 2024-04-25T18:45:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.117+0000 7f9dd5400700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dbc0a3c40 0x7f9dbc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:39.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.117+0000 7f9dd5400700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dbc0a3c40 0x7f9dbc0a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8110230 tx=0x7f9dc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:39.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.122+0000 7f9dcea00700 1 -- 172.21.15.112:0/3480633708 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9dc8010040 con 0x7f9dd8104eb0 2024-04-25T18:45:39.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.354+0000 7f9ddca00700 1 -- 172.21.15.112:0/3480633708 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f9dd81a23e0 con 0x7f9dd8104eb0 2024-04-25T18:45:39.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.355+0000 7f9dcea00700 1 -- 172.21.15.112:0/3480633708 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9dc809ccf0 con 0x7f9dd8104eb0 2024-04-25T18:45:39.357 INFO:teuthology.orchestra.run.smithi112.stdout:266287972414 2024-04-25T18:45:39.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 -- 172.21.15.112:0/3480633708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dbc0a3c40 msgr2=0x7f9dbc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:39.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dbc0a3c40 0x7f9dbc0a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8110230 tx=0x7f9dc4009260 comp rx=0 tx=0).stop 2024-04-25T18:45:39.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 -- 172.21.15.112:0/3480633708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dd8104eb0 msgr2=0x7f9dd81119c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:39.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dd8104eb0 0x7f9dd81119c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800ef10 tx=0x7f9dc800c3a0 comp rx=0 tx=0).stop 2024-04-25T18:45:39.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 -- 172.21.15.112:0/3480633708 shutdown_connections 2024-04-25T18:45:39.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dbc0a3c40 0x7f9dbc0a6100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:39.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd8109f90 0x7f9dd810f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:39.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dd8104eb0 0x7f9dd81119c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:39.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 --2- 172.21.15.112:0/3480633708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd8105860 0x7f9dd810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:39.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 -- 172.21.15.112:0/3480633708 >> 172.21.15.112:0/3480633708 conn(0x7f9dd8100740 msgr2=0x7f9dd81012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:39.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 -- 172.21.15.112:0/3480633708 shutdown_connections 2024-04-25T18:45:39.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:39.358+0000 7f9ddca00700 1 -- 172.21.15.112:0/3480633708 wait complete. 2024-04-25T18:45:39.453 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430275 got 206158430281 for osd.6 2024-04-25T18:45:39.453 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:39.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:39 smithi171 ceph-mon[33266]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T18:45:39.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:39 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3480633708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T18:45:39.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:39 smithi112 ceph-mon[27864]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T18:45:39.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:39 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3480633708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T18:45:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:39 smithi161 ceph-mon[32609]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T18:45:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:39 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3480633708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T18:45:41.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.682+0000 7f762a000700 1 -- 172.21.15.112:0/52307018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7624108a30 msgr2=0x7f7624108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:41.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.682+0000 7f762a000700 1 --2- 172.21.15.112:0/52307018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7624108a30 0x7f7624108e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7614009990 tx=0x7f7614009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:41.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.682+0000 7f762a000700 1 -- 172.21.15.112:0/52307018 shutdown_connections 2024-04-25T18:45:41.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.682+0000 7f762a000700 1 --2- 172.21.15.112:0/52307018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f762406b0f0 0x7f762410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:41.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.682+0000 7f762a000700 1 --2- 172.21.15.112:0/52307018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7624108a30 0x7f7624108e10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:41.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.682+0000 7f762a000700 1 --2- 172.21.15.112:0/52307018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f762406a750 0x7f762406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:41.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.682+0000 7f762a000700 1 -- 172.21.15.112:0/52307018 >> 172.21.15.112:0/52307018 conn(0x7f76240788f0 msgr2=0x7f7624078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:41.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.683+0000 7f762a000700 1 -- 172.21.15.112:0/52307018 shutdown_connections 2024-04-25T18:45:41.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.683+0000 7f762a000700 1 -- 172.21.15.112:0/52307018 wait complete. 2024-04-25T18:45:41.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.684+0000 7f762a000700 1 Processor -- start 2024-04-25T18:45:41.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.684+0000 7f762a000700 1 -- start start 2024-04-25T18:45:41.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.685+0000 7f762a000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f762406a750 0x7f76241991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:41.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.685+0000 7f762a000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f762406b0f0 0x7f76241996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:41.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.685+0000 7f762a000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7624108a30 0x7f762419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:41.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.685+0000 7f762a000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7624112c20 con 0x7f7624108a30 2024-04-25T18:45:41.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.685+0000 7f762a000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7624112aa0 con 0x7f762406b0f0 2024-04-25T18:45:41.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.685+0000 7f762a000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7624112da0 con 0x7f762406a750 2024-04-25T18:45:41.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.685+0000 7f7623400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f762406a750 0x7f76241991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:41.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.685+0000 7f761be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f762406b0f0 0x7f76241996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:41.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.685+0000 7f7623400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f762406a750 0x7f76241991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:33492/0 (socket says 172.21.15.112:33492) 2024-04-25T18:45:41.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.685+0000 7f7623400700 1 -- 172.21.15.112:0/602423255 learned_addr learned my addr 172.21.15.112:0/602423255 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:41.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.685+0000 7f7623e00700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7624108a30 0x7f762419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:41.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.686+0000 7f7623400700 1 -- 172.21.15.112:0/602423255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f762406b0f0 msgr2=0x7f76241996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:41.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.686+0000 7f7623400700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f762406b0f0 0x7f76241996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:41.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.686+0000 7f7623400700 1 -- 172.21.15.112:0/602423255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7624108a30 msgr2=0x7f762419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:41.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.686+0000 7f7623400700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7624108a30 0x7f762419da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:41.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.686+0000 7f7623400700 1 -- 172.21.15.112:0/602423255 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762419e1c0 con 0x7f762406a750 2024-04-25T18:45:41.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.686+0000 7f7623400700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f762406a750 0x7f76241991b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7614000c00 tx=0x7f76140057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:41.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.686+0000 7f7620c00700 1 -- 172.21.15.112:0/602423255 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614027070 con 0x7f762406a750 2024-04-25T18:45:41.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.686+0000 7f7620c00700 1 -- 172.21.15.112:0/602423255 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7614022470 con 0x7f762406a750 2024-04-25T18:45:41.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.686+0000 7f7620c00700 1 -- 172.21.15.112:0/602423255 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614017650 con 0x7f762406a750 2024-04-25T18:45:41.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.687+0000 7f762a000700 1 -- 172.21.15.112:0/602423255 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762419e450 con 0x7f762406a750 2024-04-25T18:45:41.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.687+0000 7f762a000700 1 -- 172.21.15.112:0/602423255 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76241a5d30 con 0x7f762406a750 2024-04-25T18:45:41.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.688+0000 7f762a000700 1 -- 172.21.15.112:0/602423255 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7624104170 con 0x7f762406a750 2024-04-25T18:45:41.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.690+0000 7f7620c00700 1 -- 172.21.15.112:0/602423255 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f761402a070 con 0x7f762406a750 2024-04-25T18:45:41.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.691+0000 7f7620c00700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76080a3cc0 0x7f76080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:41.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.691+0000 7f7620c00700 1 -- 172.21.15.112:0/602423255 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f76140d1130 con 0x7f762406a750 2024-04-25T18:45:41.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.692+0000 7f761be00700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76080a3cc0 0x7f76080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:41.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.692+0000 7f761be00700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76080a3cc0 0x7f76080a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7624079010 tx=0x7f760c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:41.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.696+0000 7f7620c00700 1 -- 172.21.15.112:0/602423255 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7614013070 con 0x7f762406a750 2024-04-25T18:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.926+0000 7f762a000700 1 -- 172.21.15.112:0/602423255 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f762419a510 con 0x7f762406a750 2024-04-25T18:45:41.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.927+0000 7f7620c00700 1 -- 172.21.15.112:0/602423255 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f761409db20 con 0x7f762406a750 2024-04-25T18:45:41.929 INFO:teuthology.orchestra.run.smithi112.stdout:326417514548 2024-04-25T18:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 -- 172.21.15.112:0/602423255 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76080a3cc0 msgr2=0x7f76080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76080a3cc0 0x7f76080a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7624079010 tx=0x7f760c009260 comp rx=0 tx=0).stop 2024-04-25T18:45:41.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 -- 172.21.15.112:0/602423255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f762406a750 msgr2=0x7f76241991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f762406a750 0x7f76241991b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7614000c00 tx=0x7f76140057c0 comp rx=0 tx=0).stop 2024-04-25T18:45:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 -- 172.21.15.112:0/602423255 shutdown_connections 2024-04-25T18:45:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f76080a3cc0 0x7f76080a6180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7624108a30 0x7f762419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f762406a750 0x7f76241991b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 --2- 172.21.15.112:0/602423255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f762406b0f0 0x7f76241996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:41.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 -- 172.21.15.112:0/602423255 >> 172.21.15.112:0/602423255 conn(0x7f76240788f0 msgr2=0x7f76240ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:41.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 -- 172.21.15.112:0/602423255 shutdown_connections 2024-04-25T18:45:41.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:41.931+0000 7f762a000700 1 -- 172.21.15.112:0/602423255 wait complete. 2024-04-25T18:45:42.379 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972407 got 266287972414 for osd.8 2024-04-25T18:45:42.380 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:42.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:42 smithi161 ceph-mon[32609]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T18:45:42.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:42 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/602423255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T18:45:42.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:42 smithi112 ceph-mon[27864]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T18:45:42.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:42 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/602423255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T18:45:42.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:42 smithi171 ceph-mon[33266]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T18:45:42.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:42 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/602423255' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T18:45:42.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.644+0000 7f9dd3800700 1 -- 172.21.15.112:0/4089760206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dcc074c70 msgr2=0x7f9dcc075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:42.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.644+0000 7f9dd3800700 1 --2- 172.21.15.112:0/4089760206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dcc074c70 0x7f9dcc075050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009990 tx=0x7f9dbc009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:42.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.644+0000 7f9dd3800700 1 -- 172.21.15.112:0/4089760206 shutdown_connections 2024-04-25T18:45:42.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.644+0000 7f9dd3800700 1 --2- 172.21.15.112:0/4089760206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dcc10f440 0x7f9dcc111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.644+0000 7f9dd3800700 1 --2- 172.21.15.112:0/4089760206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dcc074c70 0x7f9dcc075050 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.644+0000 7f9dd3800700 1 --2- 172.21.15.112:0/4089760206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dcc075590 0x7f9dcc10ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.644+0000 7f9dd3800700 1 -- 172.21.15.112:0/4089760206 >> 172.21.15.112:0/4089760206 conn(0x7f9dcc079ba0 msgr2=0x7f9dcc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.644+0000 7f9dd3800700 1 -- 172.21.15.112:0/4089760206 shutdown_connections 2024-04-25T18:45:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.644+0000 7f9dd3800700 1 -- 172.21.15.112:0/4089760206 wait complete. 2024-04-25T18:45:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.645+0000 7f9dd3800700 1 Processor -- start 2024-04-25T18:45:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.645+0000 7f9dd3800700 1 -- start start 2024-04-25T18:45:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.646+0000 7f9dd3800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dcc074c70 0x7f9dcc19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.646+0000 7f9dd3800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dcc075590 0x7f9dcc19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.646+0000 7f9dd3800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dcc10f440 0x7f9dcc19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.646+0000 7f9dd3800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc114e10 con 0x7f9dcc10f440 2024-04-25T18:45:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.646+0000 7f9dd3800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc114c90 con 0x7f9dcc075590 2024-04-25T18:45:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.646+0000 7f9dd3800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc114f90 con 0x7f9dcc074c70 2024-04-25T18:45:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.646+0000 7f9dd0e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dcc074c70 0x7f9dcc19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.646+0000 7f9dd0e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dcc074c70 0x7f9dcc19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:33504/0 (socket says 172.21.15.112:33504) 2024-04-25T18:45:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.646+0000 7f9dd1800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dcc10f440 0x7f9dcc19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.646+0000 7f9dd0e00700 1 -- 172.21.15.112:0/1859330453 learned_addr learned my addr 172.21.15.112:0/1859330453 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.646+0000 7f9dc3e00700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dcc075590 0x7f9dcc19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.647+0000 7f9dd0e00700 1 -- 172.21.15.112:0/1859330453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dcc075590 msgr2=0x7f9dcc19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.647+0000 7f9dd0e00700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dcc075590 0x7f9dcc19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.647+0000 7f9dd0e00700 1 -- 172.21.15.112:0/1859330453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dcc10f440 msgr2=0x7f9dcc19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.647+0000 7f9dd0e00700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dcc10f440 0x7f9dcc19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.647+0000 7f9dd0e00700 1 -- 172.21.15.112:0/1859330453 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc1a0340 con 0x7f9dcc074c70 2024-04-25T18:45:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.647+0000 7f9dd1800700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dcc10f440 0x7f9dcc19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.647+0000 7f9dd0e00700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dcc074c70 0x7f9dcc19b380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0057f0 tx=0x7f9dbc00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.647+0000 7f9dca000700 1 -- 172.21.15.112:0/1859330453 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc026070 con 0x7f9dcc074c70 2024-04-25T18:45:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.647+0000 7f9dd3800700 1 -- 172.21.15.112:0/1859330453 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc1a05d0 con 0x7f9dcc074c70 2024-04-25T18:45:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.647+0000 7f9dca000700 1 -- 172.21.15.112:0/1859330453 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dbc021410 con 0x7f9dcc074c70 2024-04-25T18:45:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.647+0000 7f9dca000700 1 -- 172.21.15.112:0/1859330453 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc017650 con 0x7f9dcc074c70 2024-04-25T18:45:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.648+0000 7f9dd3800700 1 -- 172.21.15.112:0/1859330453 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc1a7eb0 con 0x7f9dcc074c70 2024-04-25T18:45:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.649+0000 7f9dd3800700 1 -- 172.21.15.112:0/1859330453 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dcc069730 con 0x7f9dcc074c70 2024-04-25T18:45:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.650+0000 7f9dca000700 1 -- 172.21.15.112:0/1859330453 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f9dbc029070 con 0x7f9dcc074c70 2024-04-25T18:45:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.652+0000 7f9dca000700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dac0a3cc0 0x7f9dac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.652+0000 7f9dca000700 1 -- 172.21.15.112:0/1859330453 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f9dbc0d0fe0 con 0x7f9dcc074c70 2024-04-25T18:45:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.652+0000 7f9dc3e00700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dac0a3cc0 0x7f9dac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.653+0000 7f9dc3e00700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dac0a3cc0 0x7f9dac0a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc1110d0 tx=0x7f9db4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.656+0000 7f9dca000700 1 -- 172.21.15.112:0/1859330453 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9dbc05dff0 con 0x7f9dcc074c70 2024-04-25T18:45:42.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.891+0000 7f9dd3800700 1 -- 172.21.15.112:0/1859330453 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9dcc02df80 con 0x7f9dcc074c70 2024-04-25T18:45:42.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.892+0000 7f9dca000700 1 -- 172.21.15.112:0/1859330453 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9dbc013070 con 0x7f9dcc074c70 2024-04-25T18:45:42.893 INFO:teuthology.orchestra.run.smithi112.stdout:115964117084 2024-04-25T18:45:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.895+0000 7f9dd3800700 1 -- 172.21.15.112:0/1859330453 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dac0a3cc0 msgr2=0x7f9dac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.895+0000 7f9dd3800700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dac0a3cc0 0x7f9dac0a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc1110d0 tx=0x7f9db4009260 comp rx=0 tx=0).stop 2024-04-25T18:45:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.895+0000 7f9dd3800700 1 -- 172.21.15.112:0/1859330453 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dcc074c70 msgr2=0x7f9dcc19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.895+0000 7f9dd3800700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dcc074c70 0x7f9dcc19b380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0057f0 tx=0x7f9dbc00bfd0 comp rx=0 tx=0).stop 2024-04-25T18:45:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.895+0000 7f9dd3800700 1 -- 172.21.15.112:0/1859330453 shutdown_connections 2024-04-25T18:45:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.895+0000 7f9dd3800700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f9dac0a3cc0 0x7f9dac0a6180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.895+0000 7f9dd3800700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dcc10f440 0x7f9dcc19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.895+0000 7f9dd3800700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dcc074c70 0x7f9dcc19b380 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:42.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.895+0000 7f9dd3800700 1 --2- 172.21.15.112:0/1859330453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dcc075590 0x7f9dcc19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:42.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.895+0000 7f9dd3800700 1 -- 172.21.15.112:0/1859330453 >> 172.21.15.112:0/1859330453 conn(0x7f9dcc079ba0 msgr2=0x7f9dcc100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:42.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.896+0000 7f9dd3800700 1 -- 172.21.15.112:0/1859330453 shutdown_connections 2024-04-25T18:45:42.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:42.896+0000 7f9dd3800700 1 -- 172.21.15.112:0/1859330453 wait complete. 2024-04-25T18:45:43.521 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:43 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1859330453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T18:45:43.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:43 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1859330453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T18:45:43.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:43 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1859330453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T18:45:43.839 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117077 got 115964117084 for osd.3 2024-04-25T18:45:43.839 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:44.205 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514541 got 326417514548 for osd.10 2024-04-25T18:45:44.205 DEBUG:teuthology.parallel:result is None 2024-04-25T18:45:44.205 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T18:45:44.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:45:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:44 smithi112 ceph-mon[27864]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:44.584 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:44 smithi171 ceph-mon[33266]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:44.641 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:44.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:44 smithi161 ceph-mon[32609]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:45.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:45 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:45.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:45 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:45.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:45 smithi171 ceph-mon[33266]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:45:45.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:45 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:45.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:45 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:45.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:45 smithi161 ceph-mon[32609]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:45:45.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:45 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:45.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:45 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:45.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:45 smithi112 ceph-mon[27864]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:45:46.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.867+0000 7f19efc00700 1 -- 172.21.15.112:0/3814932673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19e810f4e0 msgr2=0x7f19e81118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:46.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.867+0000 7f19efc00700 1 --2- 172.21.15.112:0/3814932673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19e810f4e0 0x7f19e81118d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009f90 tx=0x7f19e000b330 comp rx=0 tx=0).stop 2024-04-25T18:45:46.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.867+0000 7f19efc00700 1 -- 172.21.15.112:0/3814932673 shutdown_connections 2024-04-25T18:45:46.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.867+0000 7f19efc00700 1 --2- 172.21.15.112:0/3814932673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19e810f4e0 0x7f19e81118d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:46.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.867+0000 7f19efc00700 1 --2- 172.21.15.112:0/3814932673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e806afe0 0x7f19e806b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:46.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.867+0000 7f19efc00700 1 --2- 172.21.15.112:0/3814932673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19e806b900 0x7f19e810efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:46.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.867+0000 7f19efc00700 1 -- 172.21.15.112:0/3814932673 >> 172.21.15.112:0/3814932673 conn(0x7f19e80788b0 msgr2=0x7f19e8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:46.869 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:46 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:45:46.869 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:46 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:45:46.869 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:46 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:46.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.869+0000 7f19efc00700 1 -- 172.21.15.112:0/3814932673 shutdown_connections 2024-04-25T18:45:46.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.869+0000 7f19efc00700 1 -- 172.21.15.112:0/3814932673 wait complete. 2024-04-25T18:45:46.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.870+0000 7f19efc00700 1 Processor -- start 2024-04-25T18:45:46.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.870+0000 7f19efc00700 1 -- start start 2024-04-25T18:45:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.870+0000 7f19efc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19e806afe0 0x7f19e819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.870+0000 7f19efc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e806b900 0x7f19e819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:46.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.870+0000 7f19efc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19e810f4e0 0x7f19e819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.870+0000 7f19efc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8114e30 con 0x7f19e806b900 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.870+0000 7f19efc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8114cb0 con 0x7f19e806afe0 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.870+0000 7f19efc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8114fb0 con 0x7f19e810f4e0 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.870+0000 7f19e7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e806b900 0x7f19e819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.870+0000 7f19edc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19e810f4e0 0x7f19e819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.871+0000 7f19ed200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19e806afe0 0x7f19e819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.871+0000 7f19edc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19e810f4e0 0x7f19e819fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:33526/0 (socket says 172.21.15.112:33526) 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.871+0000 7f19e7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e806b900 0x7f19e819b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58476/0 (socket says 172.21.15.112:58476) 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.871+0000 7f19edc00700 1 -- 172.21.15.112:0/2674233657 learned_addr learned my addr 172.21.15.112:0/2674233657 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.871+0000 7f19edc00700 1 -- 172.21.15.112:0/2674233657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19e806afe0 msgr2=0x7f19e819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.871+0000 7f19edc00700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19e806afe0 0x7f19e819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.871+0000 7f19edc00700 1 -- 172.21.15.112:0/2674233657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e806b900 msgr2=0x7f19e819b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:46.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.871+0000 7f19edc00700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e806b900 0x7f19e819b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.871+0000 7f19edc00700 1 -- 172.21.15.112:0/2674233657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e81a0310 con 0x7f19e810f4e0 2024-04-25T18:45:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.871+0000 7f19e7e00700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e806b900 0x7f19e819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.871+0000 7f19edc00700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19e810f4e0 0x7f19e819fbd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009f60 tx=0x7f19e0009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.872+0000 7f19e5600700 1 -- 172.21.15.112:0/2674233657 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0021070 con 0x7f19e810f4e0 2024-04-25T18:45:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.872+0000 7f19efc00700 1 -- 172.21.15.112:0/2674233657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e81a05a0 con 0x7f19e810f4e0 2024-04-25T18:45:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.872+0000 7f19e5600700 1 -- 172.21.15.112:0/2674233657 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19e0007ac0 con 0x7f19e810f4e0 2024-04-25T18:45:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.872+0000 7f19e5600700 1 -- 172.21.15.112:0/2674233657 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0005040 con 0x7f19e810f4e0 2024-04-25T18:45:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.872+0000 7f19efc00700 1 -- 172.21.15.112:0/2674233657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e81a7e80 con 0x7f19e810f4e0 2024-04-25T18:45:46.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.873+0000 7f19efc00700 1 -- 172.21.15.112:0/2674233657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e8069730 con 0x7f19e810f4e0 2024-04-25T18:45:46.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.876+0000 7f19e5600700 1 -- 172.21.15.112:0/2674233657 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f19e0024070 con 0x7f19e810f4e0 2024-04-25T18:45:46.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.877+0000 7f19e5600700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f19d00a3c40 0x7f19d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:46.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.877+0000 7f19e5600700 1 -- 172.21.15.112:0/2674233657 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f19e000f020 con 0x7f19e810f4e0 2024-04-25T18:45:46.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.877+0000 7f19ed200700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f19d00a3c40 0x7f19d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:46.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.878+0000 7f19ed200700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f19d00a3c40 0x7f19d00a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19dc005fd0 tx=0x7f19dc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:46.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:46.879+0000 7f19e5600700 1 -- 172.21.15.112:0/2674233657 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19e002b020 con 0x7f19e810f4e0 2024-04-25T18:45:46.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:46 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:45:46.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:46 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:45:46.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:46 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:46.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:46 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:45:46.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:46 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:45:46.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:46 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:45:47.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.100+0000 7f19efc00700 1 -- 172.21.15.112:0/2674233657 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19e819c660 con 0x7f19d00a3c40 2024-04-25T18:45:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.114+0000 7f19e5600700 1 -- 172.21.15.112:0/2674233657 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258853 (secure 0 0 0) 0x7f19e819c660 con 0x7f19d00a3c40 2024-04-25T18:45:47.120 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:45:47.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 -- 172.21.15.112:0/2674233657 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f19d00a3c40 msgr2=0x7f19d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:47.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f19d00a3c40 0x7f19d00a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19dc005fd0 tx=0x7f19dc009310 comp rx=0 tx=0).stop 2024-04-25T18:45:47.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 -- 172.21.15.112:0/2674233657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19e810f4e0 msgr2=0x7f19e819fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:47.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19e810f4e0 0x7f19e819fbd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009f60 tx=0x7f19e0009510 comp rx=0 tx=0).stop 2024-04-25T18:45:47.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 -- 172.21.15.112:0/2674233657 shutdown_connections 2024-04-25T18:45:47.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f19d00a3c40 0x7f19d00a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:47.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f19e810f4e0 0x7f19e819fbd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:47.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19e806afe0 0x7f19e819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:47.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 --2- 172.21.15.112:0/2674233657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19e806b900 0x7f19e819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:47.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 -- 172.21.15.112:0/2674233657 >> 172.21.15.112:0/2674233657 conn(0x7f19e80788b0 msgr2=0x7f19e8110990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:47.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 -- 172.21.15.112:0/2674233657 shutdown_connections 2024-04-25T18:45:47.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:47.120+0000 7f19efc00700 1 -- 172.21.15.112:0/2674233657 wait complete. 2024-04-25T18:45:47.124 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T18:45:47.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:47 smithi171 ceph-mon[33266]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:45:47.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:47 smithi171 ceph-mon[33266]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T18:45:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:47 smithi161 ceph-mon[32609]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:45:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:47 smithi161 ceph-mon[32609]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T18:45:47.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:47 smithi112 ceph-mon[27864]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:45:47.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:47 smithi112 ceph-mon[27864]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T18:45:48.137 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-04-25T18:45:45.156272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659717,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":897,"num_read_kb":571,"num_write":357,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181318,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379976,"kb_used_data":14184,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561848,"statfs":{"total":1151940427776,"available":1151551332352,"internally_reserved":0,"allocated":14524416,"data_stored":7957251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":34,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004834"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.122166+0000","last_change":"2024-04-25T18:43:16.262618+0000","last_active":"2024-04-25T18:45:25.122166+0000","last_peered":"2024-04-25T18:45:25.122166+0000","last_clean":"2024-04-25T18:45:25.122166+0000","last_became_active":"2024-04-25T18:43:16.262215+0000","last_became_peered":"2024-04-25T18:43:16.262215+0000","last_unstale":"2024-04-25T18:45:25.122166+0000","last_undegraded":"2024-04-25T18:45:25.122166+0000","last_fullsized":"2024-04-25T18:45:25.122166+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:40.319102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.123754+0000","last_change":"2024-04-25T18:43:16.254085+0000","last_active":"2024-04-25T18:45:25.123754+0000","last_peered":"2024-04-25T18:45:25.123754+0000","last_clean":"2024-04-25T18:45:25.123754+0000","last_became_active":"2024-04-25T18:43:16.248943+0000","last_became_peered":"2024-04-25T18:43:16.248943+0000","last_unstale":"2024-04-25T18:45:25.123754+0000","last_undegraded":"2024-04-25T18:45:25.123754+0000","last_fullsized":"2024-04-25T18:45:25.123754+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:53.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.123304+0000","last_change":"2024-04-25T18:43:16.252684+0000","last_active":"2024-04-25T18:45:25.123304+0000","last_peered":"2024-04-25T18:45:25.123304+0000","last_clean":"2024-04-25T18:45:25.123304+0000","last_became_active":"2024-04-25T18:43:16.252318+0000","last_became_peered":"2024-04-25T18:43:16.252318+0000","last_unstale":"2024-04-25T18:45:25.123304+0000","last_undegraded":"2024-04-25T18:45:25.123304+0000","last_fullsized":"2024-04-25T18:45:25.123304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:58.904865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.092379+0000","last_change":"2024-04-25T18:43:16.247059+0000","last_active":"2024-04-25T18:45:25.092379+0000","last_peered":"2024-04-25T18:45:25.092379+0000","last_clean":"2024-04-25T18:45:25.092379+0000","last_became_active":"2024-04-25T18:43:16.246655+0000","last_became_peered":"2024-04-25T18:43:16.246655+0000","last_unstale":"2024-04-25T18:45:25.092379+0000","last_undegraded":"2024-04-25T18:45:25.092379+0000","last_fullsized":"2024-04-25T18:45:25.092379+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:31.026579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.098879+0000","last_change":"2024-04-25T18:43:16.250033+0000","last_active":"2024-04-25T18:45:25.098879+0000","last_peered":"2024-04-25T18:45:25.098879+0000","last_clean":"2024-04-25T18:45:25.098879+0000","last_became_active":"2024-04-25T18:43:16.249017+0000","last_became_peered":"2024-04-25T18:43:16.249017+0000","last_unstale":"2024-04-25T18:45:25.098879+0000","last_undegraded":"2024-04-25T18:45:25.098879+0000","last_fullsized":"2024-04-25T18:45:25.098879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:34.304263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.110285+0000","last_change":"2024-04-25T18:43:56.642410+0000","last_active":"2024-04-25T18:45:25.110285+0000","last_peered":"2024-04-25T18:45:25.110285+0000","last_clean":"2024-04-25T18:45:25.110285+0000","last_became_active":"2024-04-25T18:43:56.642009+0000","last_became_peered":"2024-04-25T18:43:56.642009+0000","last_unstale":"2024-04-25T18:45:25.110285+0000","last_undegraded":"2024-04-25T18:45:25.110285+0000","last_fullsized":"2024-04-25T18:45:25.110285+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:19.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.109029+0000","last_change":"2024-04-25T18:43:56.647158+0000","last_active":"2024-04-25T18:45:25.109029+0000","last_peered":"2024-04-25T18:45:25.109029+0000","last_clean":"2024-04-25T18:45:25.109029+0000","last_became_active":"2024-04-25T18:43:56.646779+0000","last_became_peered":"2024-04-25T18:43:56.646779+0000","last_unstale":"2024-04-25T18:45:25.109029+0000","last_undegraded":"2024-04-25T18:45:25.109029+0000","last_fullsized":"2024-04-25T18:45:25.109029+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:04:14.055762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.108821+0000","last_change":"2024-04-25T18:43:16.247607+0000","last_active":"2024-04-25T18:45:25.108821+0000","last_peered":"2024-04-25T18:45:25.108821+0000","last_clean":"2024-04-25T18:45:25.108821+0000","last_became_active":"2024-04-25T18:43:16.247234+0000","last_became_peered":"2024-04-25T18:43:16.247234+0000","last_unstale":"2024-04-25T18:45:25.108821+0000","last_undegraded":"2024-04-25T18:45:25.108821+0000","last_fullsized":"2024-04-25T18:45:25.108821+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:57.117900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.106758+0000","last_change":"2024-04-25T18:43:16.247909+0000","last_active":"2024-04-25T18:45:25.106758+0000","last_peered":"2024-04-25T18:45:25.106758+0000","last_clean":"2024-04-25T18:45:25.106758+0000","last_became_active":"2024-04-25T18:43:16.247533+0000","last_became_peered":"2024-04-25T18:43:16.247533+0000","last_unstale":"2024-04-25T18:45:25.106758+0000","last_undegraded":"2024-04-25T18:45:25.106758+0000","last_fullsized":"2024-04-25T18:45:25.106758+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:02.257832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.105378+0000","last_change":"2024-04-25T18:43:16.252088+0000","last_active":"2024-04-25T18:45:25.105378+0000","last_peered":"2024-04-25T18:45:25.105378+0000","last_clean":"2024-04-25T18:45:25.105378+0000","last_became_active":"2024-04-25T18:43:16.251580+0000","last_became_peered":"2024-04-25T18:43:16.251580+0000","last_unstale":"2024-04-25T18:45:25.105378+0000","last_undegraded":"2024-04-25T18:45:25.105378+0000","last_fullsized":"2024-04-25T18:45:25.105378+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:22.992087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.109045+0000","last_change":"2024-04-25T18:43:16.239696+0000","last_active":"2024-04-25T18:45:25.109045+0000","last_peered":"2024-04-25T18:45:25.109045+0000","last_clean":"2024-04-25T18:45:25.109045+0000","last_became_active":"2024-04-25T18:43:16.239354+0000","last_became_peered":"2024-04-25T18:43:16.239354+0000","last_unstale":"2024-04-25T18:45:25.109045+0000","last_undegraded":"2024-04-25T18:45:25.109045+0000","last_fullsized":"2024-04-25T18:45:25.109045+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:39.682138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.100506+0000","last_change":"2024-04-25T18:43:16.242487+0000","last_active":"2024-04-25T18:45:25.100506+0000","last_peered":"2024-04-25T18:45:25.100506+0000","last_clean":"2024-04-25T18:45:25.100506+0000","last_became_active":"2024-04-25T18:43:16.242158+0000","last_became_peered":"2024-04-25T18:43:16.242158+0000","last_unstale":"2024-04-25T18:45:25.100506+0000","last_undegraded":"2024-04-25T18:45:25.100506+0000","last_fullsized":"2024-04-25T18:45:25.100506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:36.477440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.118327+0000","last_change":"2024-04-25T18:43:16.244154+0000","last_active":"2024-04-25T18:45:25.118327+0000","last_peered":"2024-04-25T18:45:25.118327+0000","last_clean":"2024-04-25T18:45:25.118327+0000","last_became_active":"2024-04-25T18:43:16.243829+0000","last_became_peered":"2024-04-25T18:43:16.243829+0000","last_unstale":"2024-04-25T18:45:25.118327+0000","last_undegraded":"2024-04-25T18:45:25.118327+0000","last_fullsized":"2024-04-25T18:45:25.118327+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:43.262751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.124140+0000","last_change":"2024-04-25T18:43:16.261227+0000","last_active":"2024-04-25T18:45:25.124140+0000","last_peered":"2024-04-25T18:45:25.124140+0000","last_clean":"2024-04-25T18:45:25.124140+0000","last_became_active":"2024-04-25T18:43:16.250468+0000","last_became_peered":"2024-04-25T18:43:16.250468+0000","last_unstale":"2024-04-25T18:45:25.124140+0000","last_undegraded":"2024-04-25T18:45:25.124140+0000","last_fullsized":"2024-04-25T18:45:25.124140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:44.805643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.123276+0000","last_change":"2024-04-25T18:43:16.257956+0000","last_active":"2024-04-25T18:45:25.123276+0000","last_peered":"2024-04-25T18:45:25.123276+0000","last_clean":"2024-04-25T18:45:25.123276+0000","last_became_active":"2024-04-25T18:43:16.255126+0000","last_became_peered":"2024-04-25T18:43:16.255126+0000","last_unstale":"2024-04-25T18:45:25.123276+0000","last_undegraded":"2024-04-25T18:45:25.123276+0000","last_fullsized":"2024-04-25T18:45:25.123276+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:30.047165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.123810+0000","last_change":"2024-04-25T18:43:16.256208+0000","last_active":"2024-04-25T18:45:25.123810+0000","last_peered":"2024-04-25T18:45:25.123810+0000","last_clean":"2024-04-25T18:45:25.123810+0000","last_became_active":"2024-04-25T18:43:16.255614+0000","last_became_peered":"2024-04-25T18:43:16.255614+0000","last_unstale":"2024-04-25T18:45:25.123810+0000","last_undegraded":"2024-04-25T18:45:25.123810+0000","last_fullsized":"2024-04-25T18:45:25.123810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:46.700209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.126947+0000","last_change":"2024-04-25T18:43:16.262730+0000","last_active":"2024-04-25T18:45:25.126947+0000","last_peered":"2024-04-25T18:45:25.126947+0000","last_clean":"2024-04-25T18:45:25.126947+0000","last_became_active":"2024-04-25T18:43:16.262192+0000","last_became_peered":"2024-04-25T18:43:16.262192+0000","last_unstale":"2024-04-25T18:45:25.126947+0000","last_undegraded":"2024-04-25T18:45:25.126947+0000","last_fullsized":"2024-04-25T18:45:25.126947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:29.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.110843+0000","last_change":"2024-04-25T18:43:16.261161+0000","last_active":"2024-04-25T18:45:25.110843+0000","last_peered":"2024-04-25T18:45:25.110843+0000","last_clean":"2024-04-25T18:45:25.110843+0000","last_became_active":"2024-04-25T18:43:16.258217+0000","last_became_peered":"2024-04-25T18:43:16.258217+0000","last_unstale":"2024-04-25T18:45:25.110843+0000","last_undegraded":"2024-04-25T18:45:25.110843+0000","last_fullsized":"2024-04-25T18:45:25.110843+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.564609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.106213+0000","last_change":"2024-04-25T18:43:16.266660+0000","last_active":"2024-04-25T18:45:25.106213+0000","last_peered":"2024-04-25T18:45:25.106213+0000","last_clean":"2024-04-25T18:45:25.106213+0000","last_became_active":"2024-04-25T18:43:16.266295+0000","last_became_peered":"2024-04-25T18:43:16.266295+0000","last_unstale":"2024-04-25T18:45:25.106213+0000","last_undegraded":"2024-04-25T18:45:25.106213+0000","last_fullsized":"2024-04-25T18:45:25.106213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:42.406585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.643467+0000","last_change":"2024-04-25T18:43:12.200172+0000","last_active":"2024-04-25T18:43:56.643467+0000","last_peered":"2024-04-25T18:43:56.643467+0000","last_clean":"2024-04-25T18:43:56.643467+0000","last_became_active":"2024-04-25T18:43:12.199760+0000","last_became_peered":"2024-04-25T18:43:12.199760+0000","last_unstale":"2024-04-25T18:43:56.643467+0000","last_undegraded":"2024-04-25T18:43:56.643467+0000","last_fullsized":"2024-04-25T18:43:56.643467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:57.460274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.106430+0000","last_change":"2024-04-25T18:43:16.255989+0000","last_active":"2024-04-25T18:45:25.106430+0000","last_peered":"2024-04-25T18:45:25.106430+0000","last_clean":"2024-04-25T18:45:25.106430+0000","last_became_active":"2024-04-25T18:43:16.255114+0000","last_became_peered":"2024-04-25T18:43:16.255114+0000","last_unstale":"2024-04-25T18:45:25.106430+0000","last_undegraded":"2024-04-25T18:45:25.106430+0000","last_fullsized":"2024-04-25T18:45:25.106430+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:52.888876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.639724+0000","last_change":"2024-04-25T18:43:12.208194+0000","last_active":"2024-04-25T18:43:56.639724+0000","last_peered":"2024-04-25T18:43:56.639724+0000","last_clean":"2024-04-25T18:43:56.639724+0000","last_became_active":"2024-04-25T18:43:12.207801+0000","last_became_peered":"2024-04-25T18:43:12.207801+0000","last_unstale":"2024-04-25T18:43:56.639724+0000","last_undegraded":"2024-04-25T18:43:56.639724+0000","last_fullsized":"2024-04-25T18:43:56.639724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:24.742486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.128382+0000","last_change":"2024-04-25T18:43:16.268960+0000","last_active":"2024-04-25T18:45:25.128382+0000","last_peered":"2024-04-25T18:45:25.128382+0000","last_clean":"2024-04-25T18:45:25.128382+0000","last_became_active":"2024-04-25T18:43:16.263626+0000","last_became_peered":"2024-04-25T18:43:16.263626+0000","last_unstale":"2024-04-25T18:45:25.128382+0000","last_undegraded":"2024-04-25T18:45:25.128382+0000","last_fullsized":"2024-04-25T18:45:25.128382+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:57.721918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.110665+0000","last_change":"2024-04-25T18:43:57.110208+0000","last_active":"2024-04-25T18:43:57.110665+0000","last_peered":"2024-04-25T18:43:57.110665+0000","last_clean":"2024-04-25T18:43:57.110665+0000","last_became_active":"2024-04-25T18:43:57.106027+0000","last_became_peered":"2024-04-25T18:43:57.106027+0000","last_unstale":"2024-04-25T18:43:57.110665+0000","last_undegraded":"2024-04-25T18:43:57.110665+0000","last_fullsized":"2024-04-25T18:43:57.110665+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:20.819961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.118169+0000","last_change":"2024-04-25T18:43:16.250414+0000","last_active":"2024-04-25T18:45:25.118169+0000","last_peered":"2024-04-25T18:45:25.118169+0000","last_clean":"2024-04-25T18:45:25.118169+0000","last_became_active":"2024-04-25T18:43:16.249832+0000","last_became_peered":"2024-04-25T18:43:16.249832+0000","last_unstale":"2024-04-25T18:45:25.118169+0000","last_undegraded":"2024-04-25T18:45:25.118169+0000","last_fullsized":"2024-04-25T18:45:25.118169+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.577854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628858+0000","last_change":"2024-04-25T18:43:12.195099+0000","last_active":"2024-04-25T18:43:56.628858+0000","last_peered":"2024-04-25T18:43:56.628858+0000","last_clean":"2024-04-25T18:43:56.628858+0000","last_became_active":"2024-04-25T18:43:12.194726+0000","last_became_peered":"2024-04-25T18:43:12.194726+0000","last_unstale":"2024-04-25T18:43:56.628858+0000","last_undegraded":"2024-04-25T18:43:56.628858+0000","last_fullsized":"2024-04-25T18:43:56.628858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.866905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.107531+0000","last_change":"2024-04-25T18:43:16.245208+0000","last_active":"2024-04-25T18:45:25.107531+0000","last_peered":"2024-04-25T18:45:25.107531+0000","last_clean":"2024-04-25T18:45:25.107531+0000","last_became_active":"2024-04-25T18:43:16.244836+0000","last_became_peered":"2024-04-25T18:43:16.244836+0000","last_unstale":"2024-04-25T18:45:25.107531+0000","last_undegraded":"2024-04-25T18:45:25.107531+0000","last_fullsized":"2024-04-25T18:45:25.107531+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:35.917060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":160,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:42.462758+0000","last_change":"2024-04-25T18:43:12.198252+0000","last_active":"2024-04-25T18:45:42.462758+0000","last_peered":"2024-04-25T18:45:42.462758+0000","last_clean":"2024-04-25T18:45:42.462758+0000","last_became_active":"2024-04-25T18:43:12.197691+0000","last_became_peered":"2024-04-25T18:43:12.197691+0000","last_unstale":"2024-04-25T18:45:42.462758+0000","last_undegraded":"2024-04-25T18:45:42.462758+0000","last_fullsized":"2024-04-25T18:45:42.462758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.900012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.101212+0000","last_change":"2024-04-25T18:43:16.255576+0000","last_active":"2024-04-25T18:45:25.101212+0000","last_peered":"2024-04-25T18:45:25.101212+0000","last_clean":"2024-04-25T18:45:25.101212+0000","last_became_active":"2024-04-25T18:43:16.255195+0000","last_became_peered":"2024-04-25T18:43:16.255195+0000","last_unstale":"2024-04-25T18:45:25.101212+0000","last_undegraded":"2024-04-25T18:45:25.101212+0000","last_fullsized":"2024-04-25T18:45:25.101212+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:41.510872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626060+0000","last_change":"2024-04-25T18:43:12.200786+0000","last_active":"2024-04-25T18:43:56.626060+0000","last_peered":"2024-04-25T18:43:56.626060+0000","last_clean":"2024-04-25T18:43:56.626060+0000","last_became_active":"2024-04-25T18:43:12.200409+0000","last_became_peered":"2024-04-25T18:43:12.200409+0000","last_unstale":"2024-04-25T18:43:56.626060+0000","last_undegraded":"2024-04-25T18:43:56.626060+0000","last_fullsized":"2024-04-25T18:43:56.626060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:52.177787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.105680+0000","last_change":"2024-04-25T18:43:16.255563+0000","last_active":"2024-04-25T18:45:25.105680+0000","last_peered":"2024-04-25T18:45:25.105680+0000","last_clean":"2024-04-25T18:45:25.105680+0000","last_became_active":"2024-04-25T18:43:16.255174+0000","last_became_peered":"2024-04-25T18:43:16.255174+0000","last_unstale":"2024-04-25T18:45:25.105680+0000","last_undegraded":"2024-04-25T18:45:25.105680+0000","last_fullsized":"2024-04-25T18:45:25.105680+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:32:15.735051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628920+0000","last_change":"2024-04-25T18:43:12.210242+0000","last_active":"2024-04-25T18:43:56.628920+0000","last_peered":"2024-04-25T18:43:56.628920+0000","last_clean":"2024-04-25T18:43:56.628920+0000","last_became_active":"2024-04-25T18:43:12.209600+0000","last_became_peered":"2024-04-25T18:43:12.209600+0000","last_unstale":"2024-04-25T18:43:56.628920+0000","last_undegraded":"2024-04-25T18:43:56.628920+0000","last_fullsized":"2024-04-25T18:43:56.628920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.886712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.105558+0000","last_change":"2024-04-25T18:43:56.649102+0000","last_active":"2024-04-25T18:45:25.105558+0000","last_peered":"2024-04-25T18:45:25.105558+0000","last_clean":"2024-04-25T18:45:25.105558+0000","last_became_active":"2024-04-25T18:43:56.647127+0000","last_became_peered":"2024-04-25T18:43:56.647127+0000","last_unstale":"2024-04-25T18:45:25.105558+0000","last_undegraded":"2024-04-25T18:45:25.105558+0000","last_fullsized":"2024-04-25T18:45:25.105558+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:43.111539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632572+0000","last_change":"2024-04-25T18:43:12.198957+0000","last_active":"2024-04-25T18:43:56.632572+0000","last_peered":"2024-04-25T18:43:56.632572+0000","last_clean":"2024-04-25T18:43:56.632572+0000","last_became_active":"2024-04-25T18:43:12.198605+0000","last_became_peered":"2024-04-25T18:43:12.198605+0000","last_unstale":"2024-04-25T18:43:56.632572+0000","last_undegraded":"2024-04-25T18:43:56.632572+0000","last_fullsized":"2024-04-25T18:43:56.632572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:11.759430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.108032+0000","last_change":"2024-04-25T18:43:16.266829+0000","last_active":"2024-04-25T18:45:25.108032+0000","last_peered":"2024-04-25T18:45:25.108032+0000","last_clean":"2024-04-25T18:45:25.108032+0000","last_became_active":"2024-04-25T18:43:16.266256+0000","last_became_peered":"2024-04-25T18:43:16.266256+0000","last_unstale":"2024-04-25T18:45:25.108032+0000","last_undegraded":"2024-04-25T18:45:25.108032+0000","last_fullsized":"2024-04-25T18:45:25.108032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:43.343613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":152,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:40.458500+0000","last_change":"2024-04-25T18:43:12.201641+0000","last_active":"2024-04-25T18:45:40.458500+0000","last_peered":"2024-04-25T18:45:40.458500+0000","last_clean":"2024-04-25T18:45:40.458500+0000","last_became_active":"2024-04-25T18:43:12.201274+0000","last_became_peered":"2024-04-25T18:43:12.201274+0000","last_unstale":"2024-04-25T18:45:40.458500+0000","last_undegraded":"2024-04-25T18:45:40.458500+0000","last_fullsized":"2024-04-25T18:45:40.458500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:47.683850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.111637+0000","last_change":"2024-04-25T18:43:16.259551+0000","last_active":"2024-04-25T18:45:25.111637+0000","last_peered":"2024-04-25T18:45:25.111637+0000","last_clean":"2024-04-25T18:45:25.111637+0000","last_became_active":"2024-04-25T18:43:16.259165+0000","last_became_peered":"2024-04-25T18:43:16.259165+0000","last_unstale":"2024-04-25T18:45:25.111637+0000","last_undegraded":"2024-04-25T18:45:25.111637+0000","last_fullsized":"2024-04-25T18:45:25.111637+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:23.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630339+0000","last_change":"2024-04-25T18:43:12.197534+0000","last_active":"2024-04-25T18:43:56.630339+0000","last_peered":"2024-04-25T18:43:56.630339+0000","last_clean":"2024-04-25T18:43:56.630339+0000","last_became_active":"2024-04-25T18:43:12.197106+0000","last_became_peered":"2024-04-25T18:43:12.197106+0000","last_unstale":"2024-04-25T18:43:56.630339+0000","last_undegraded":"2024-04-25T18:43:56.630339+0000","last_fullsized":"2024-04-25T18:43:56.630339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:35.399044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.127461+0000","last_change":"2024-04-25T18:43:16.257668+0000","last_active":"2024-04-25T18:45:25.127461+0000","last_peered":"2024-04-25T18:45:25.127461+0000","last_clean":"2024-04-25T18:45:25.127461+0000","last_became_active":"2024-04-25T18:43:16.253609+0000","last_became_peered":"2024-04-25T18:43:16.253609+0000","last_unstale":"2024-04-25T18:45:25.127461+0000","last_undegraded":"2024-04-25T18:45:25.127461+0000","last_fullsized":"2024-04-25T18:45:25.127461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:36.048386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.633120+0000","last_change":"2024-04-25T18:43:12.207322+0000","last_active":"2024-04-25T18:43:56.633120+0000","last_peered":"2024-04-25T18:43:56.633120+0000","last_clean":"2024-04-25T18:43:56.633120+0000","last_became_active":"2024-04-25T18:43:12.206947+0000","last_became_peered":"2024-04-25T18:43:12.206947+0000","last_unstale":"2024-04-25T18:43:56.633120+0000","last_undegraded":"2024-04-25T18:43:56.633120+0000","last_fullsized":"2024-04-25T18:43:56.633120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:20.536598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.110098+0000","last_change":"2024-04-25T18:43:16.239575+0000","last_active":"2024-04-25T18:45:25.110098+0000","last_peered":"2024-04-25T18:45:25.110098+0000","last_clean":"2024-04-25T18:45:25.110098+0000","last_became_active":"2024-04-25T18:43:16.239208+0000","last_became_peered":"2024-04-25T18:43:16.239208+0000","last_unstale":"2024-04-25T18:45:25.110098+0000","last_undegraded":"2024-04-25T18:45:25.110098+0000","last_fullsized":"2024-04-25T18:45:25.110098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:58.195855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":314,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:41.532158+0000","last_change":"2024-04-25T18:43:12.197199+0000","last_active":"2024-04-25T18:45:41.532158+0000","last_peered":"2024-04-25T18:45:41.532158+0000","last_clean":"2024-04-25T18:45:41.532158+0000","last_became_active":"2024-04-25T18:43:12.196776+0000","last_became_peered":"2024-04-25T18:43:12.196776+0000","last_unstale":"2024-04-25T18:45:41.532158+0000","last_undegraded":"2024-04-25T18:45:41.532158+0000","last_fullsized":"2024-04-25T18:45:41.532158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:54.149782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64973,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":252,"num_read_kb":189,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.087909+0000","last_change":"2024-04-25T18:43:16.250993+0000","last_active":"2024-04-25T18:45:25.087909+0000","last_peered":"2024-04-25T18:45:25.087909+0000","last_clean":"2024-04-25T18:45:25.087909+0000","last_became_active":"2024-04-25T18:43:16.250216+0000","last_became_peered":"2024-04-25T18:43:16.250216+0000","last_unstale":"2024-04-25T18:45:25.087909+0000","last_undegraded":"2024-04-25T18:45:25.087909+0000","last_fullsized":"2024-04-25T18:45:25.087909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:37.996463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626183+0000","last_change":"2024-04-25T18:43:12.204606+0000","last_active":"2024-04-25T18:43:56.626183+0000","last_peered":"2024-04-25T18:43:56.626183+0000","last_clean":"2024-04-25T18:43:56.626183+0000","last_became_active":"2024-04-25T18:43:12.204306+0000","last_became_peered":"2024-04-25T18:43:12.204306+0000","last_unstale":"2024-04-25T18:43:56.626183+0000","last_undegraded":"2024-04-25T18:43:56.626183+0000","last_fullsized":"2024-04-25T18:43:56.626183+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:29.903775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.115780+0000","last_change":"2024-04-25T18:43:16.250093+0000","last_active":"2024-04-25T18:45:25.115780+0000","last_peered":"2024-04-25T18:45:25.115780+0000","last_clean":"2024-04-25T18:45:25.115780+0000","last_became_active":"2024-04-25T18:43:16.249567+0000","last_became_peered":"2024-04-25T18:43:16.249567+0000","last_unstale":"2024-04-25T18:45:25.115780+0000","last_undegraded":"2024-04-25T18:45:25.115780+0000","last_fullsized":"2024-04-25T18:45:25.115780+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:35.498641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.652890+0000","last_change":"2024-04-25T18:43:56.652488+0000","last_active":"2024-04-25T18:43:56.652890+0000","last_peered":"2024-04-25T18:43:56.652890+0000","last_clean":"2024-04-25T18:43:56.652890+0000","last_became_active":"2024-04-25T18:43:56.647745+0000","last_became_peered":"2024-04-25T18:43:56.647745+0000","last_unstale":"2024-04-25T18:43:56.652890+0000","last_undegraded":"2024-04-25T18:43:56.652890+0000","last_fullsized":"2024-04-25T18:43:56.652890+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:33.474470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.117963+0000","last_change":"2024-04-25T18:43:56.642673+0000","last_active":"2024-04-25T18:45:25.117963+0000","last_peered":"2024-04-25T18:45:25.117963+0000","last_clean":"2024-04-25T18:45:25.117963+0000","last_became_active":"2024-04-25T18:43:56.642313+0000","last_became_peered":"2024-04-25T18:43:56.642313+0000","last_unstale":"2024-04-25T18:45:25.117963+0000","last_undegraded":"2024-04-25T18:45:25.117963+0000","last_fullsized":"2024-04-25T18:45:25.117963+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:45.805738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:43.496725+0000","last_change":"2024-04-25T18:43:12.197341+0000","last_active":"2024-04-25T18:45:43.496725+0000","last_peered":"2024-04-25T18:45:43.496725+0000","last_clean":"2024-04-25T18:45:43.496725+0000","last_became_active":"2024-04-25T18:43:12.196952+0000","last_became_peered":"2024-04-25T18:43:12.196952+0000","last_unstale":"2024-04-25T18:45:43.496725+0000","last_undegraded":"2024-04-25T18:45:43.496725+0000","last_fullsized":"2024-04-25T18:45:43.496725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:52.902579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.106340+0000","last_change":"2024-04-25T18:43:16.245436+0000","last_active":"2024-04-25T18:45:25.106340+0000","last_peered":"2024-04-25T18:45:25.106340+0000","last_clean":"2024-04-25T18:45:25.106340+0000","last_became_active":"2024-04-25T18:43:16.244943+0000","last_became_peered":"2024-04-25T18:43:16.244943+0000","last_unstale":"2024-04-25T18:45:25.106340+0000","last_undegraded":"2024-04-25T18:45:25.106340+0000","last_fullsized":"2024-04-25T18:45:25.106340+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:03.402999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628148+0000","last_change":"2024-04-25T18:43:12.197642+0000","last_active":"2024-04-25T18:43:56.628148+0000","last_peered":"2024-04-25T18:43:56.628148+0000","last_clean":"2024-04-25T18:43:56.628148+0000","last_became_active":"2024-04-25T18:43:12.196395+0000","last_became_peered":"2024-04-25T18:43:12.196395+0000","last_unstale":"2024-04-25T18:43:56.628148+0000","last_undegraded":"2024-04-25T18:43:56.628148+0000","last_fullsized":"2024-04-25T18:43:56.628148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:11.783422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.107052+0000","last_change":"2024-04-25T18:43:16.245344+0000","last_active":"2024-04-25T18:45:25.107052+0000","last_peered":"2024-04-25T18:45:25.107052+0000","last_clean":"2024-04-25T18:45:25.107052+0000","last_became_active":"2024-04-25T18:43:16.244984+0000","last_became_peered":"2024-04-25T18:43:16.244984+0000","last_unstale":"2024-04-25T18:45:25.107052+0000","last_undegraded":"2024-04-25T18:45:25.107052+0000","last_fullsized":"2024-04-25T18:45:25.107052+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:04.822629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628634+0000","last_change":"2024-04-25T18:43:12.199685+0000","last_active":"2024-04-25T18:43:56.628634+0000","last_peered":"2024-04-25T18:43:56.628634+0000","last_clean":"2024-04-25T18:43:56.628634+0000","last_became_active":"2024-04-25T18:43:12.199261+0000","last_became_peered":"2024-04-25T18:43:12.199261+0000","last_unstale":"2024-04-25T18:43:56.628634+0000","last_undegraded":"2024-04-25T18:43:56.628634+0000","last_fullsized":"2024-04-25T18:43:56.628634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:09.784616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.104269+0000","last_change":"2024-04-25T18:43:16.247745+0000","last_active":"2024-04-25T18:45:25.104269+0000","last_peered":"2024-04-25T18:45:25.104269+0000","last_clean":"2024-04-25T18:45:25.104269+0000","last_became_active":"2024-04-25T18:43:16.247311+0000","last_became_peered":"2024-04-25T18:43:16.247311+0000","last_unstale":"2024-04-25T18:45:25.104269+0000","last_undegraded":"2024-04-25T18:45:25.104269+0000","last_fullsized":"2024-04-25T18:45:25.104269+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:29.925464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.642742+0000","last_change":"2024-04-25T18:43:12.210209+0000","last_active":"2024-04-25T18:43:56.642742+0000","last_peered":"2024-04-25T18:43:56.642742+0000","last_clean":"2024-04-25T18:43:56.642742+0000","last_became_active":"2024-04-25T18:43:12.209708+0000","last_became_peered":"2024-04-25T18:43:12.209708+0000","last_unstale":"2024-04-25T18:43:56.642742+0000","last_undegraded":"2024-04-25T18:43:56.642742+0000","last_fullsized":"2024-04-25T18:43:56.642742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:07:18.331518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.112451+0000","last_change":"2024-04-25T18:43:16.263616+0000","last_active":"2024-04-25T18:45:25.112451+0000","last_peered":"2024-04-25T18:45:25.112451+0000","last_clean":"2024-04-25T18:45:25.112451+0000","last_became_active":"2024-04-25T18:43:16.263287+0000","last_became_peered":"2024-04-25T18:43:16.263287+0000","last_unstale":"2024-04-25T18:45:25.112451+0000","last_undegraded":"2024-04-25T18:45:25.112451+0000","last_fullsized":"2024-04-25T18:45:25.112451+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:46.366457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630374+0000","last_change":"2024-04-25T18:43:12.203705+0000","last_active":"2024-04-25T18:43:56.630374+0000","last_peered":"2024-04-25T18:43:56.630374+0000","last_clean":"2024-04-25T18:43:56.630374+0000","last_became_active":"2024-04-25T18:43:12.203371+0000","last_became_peered":"2024-04-25T18:43:12.203371+0000","last_unstale":"2024-04-25T18:43:56.630374+0000","last_undegraded":"2024-04-25T18:43:56.630374+0000","last_fullsized":"2024-04-25T18:43:56.630374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.119480+0000","last_change":"2024-04-25T18:43:56.647597+0000","last_active":"2024-04-25T18:45:25.119480+0000","last_peered":"2024-04-25T18:45:25.119480+0000","last_clean":"2024-04-25T18:45:25.119480+0000","last_became_active":"2024-04-25T18:43:56.647121+0000","last_became_peered":"2024-04-25T18:43:56.647121+0000","last_unstale":"2024-04-25T18:45:25.119480+0000","last_undegraded":"2024-04-25T18:45:25.119480+0000","last_fullsized":"2024-04-25T18:45:25.119480+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:07.143806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.100431+0000","last_change":"2024-04-25T18:43:16.247133+0000","last_active":"2024-04-25T18:45:25.100431+0000","last_peered":"2024-04-25T18:45:25.100431+0000","last_clean":"2024-04-25T18:45:25.100431+0000","last_became_active":"2024-04-25T18:43:16.246232+0000","last_became_peered":"2024-04-25T18:43:16.246232+0000","last_unstale":"2024-04-25T18:45:25.100431+0000","last_undegraded":"2024-04-25T18:45:25.100431+0000","last_fullsized":"2024-04-25T18:45:25.100431+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.288404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:42.493899+0000","last_change":"2024-04-25T18:43:12.209908+0000","last_active":"2024-04-25T18:45:42.493899+0000","last_peered":"2024-04-25T18:45:42.493899+0000","last_clean":"2024-04-25T18:45:42.493899+0000","last_became_active":"2024-04-25T18:43:12.209042+0000","last_became_peered":"2024-04-25T18:43:12.209042+0000","last_unstale":"2024-04-25T18:45:42.493899+0000","last_undegraded":"2024-04-25T18:45:42.493899+0000","last_fullsized":"2024-04-25T18:45:42.493899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:36.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.114435+0000","last_change":"2024-04-25T18:43:16.242364+0000","last_active":"2024-04-25T18:45:25.114435+0000","last_peered":"2024-04-25T18:45:25.114435+0000","last_clean":"2024-04-25T18:45:25.114435+0000","last_became_active":"2024-04-25T18:43:16.242019+0000","last_became_peered":"2024-04-25T18:43:16.242019+0000","last_unstale":"2024-04-25T18:45:25.114435+0000","last_undegraded":"2024-04-25T18:45:25.114435+0000","last_fullsized":"2024-04-25T18:45:25.114435+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:13.238129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628468+0000","last_change":"2024-04-25T18:43:12.206606+0000","last_active":"2024-04-25T18:43:56.628468+0000","last_peered":"2024-04-25T18:43:56.628468+0000","last_clean":"2024-04-25T18:43:56.628468+0000","last_became_active":"2024-04-25T18:43:12.206238+0000","last_became_peered":"2024-04-25T18:43:12.206238+0000","last_unstale":"2024-04-25T18:43:56.628468+0000","last_undegraded":"2024-04-25T18:43:56.628468+0000","last_fullsized":"2024-04-25T18:43:56.628468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:07.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630224+0000","last_change":"2024-04-25T18:41:40.527363+0000","last_active":"2024-04-25T18:43:56.630224+0000","last_peered":"2024-04-25T18:43:56.630224+0000","last_clean":"2024-04-25T18:43:56.630224+0000","last_became_active":"2024-04-25T18:41:40.526907+0000","last_became_peered":"2024-04-25T18:41:40.526907+0000","last_unstale":"2024-04-25T18:43:56.630224+0000","last_undegraded":"2024-04-25T18:43:56.630224+0000","last_fullsized":"2024-04-25T18:43:56.630224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_clean_scrub_stamp":"2024-04-25T18:37:45.963565+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:16.689693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.107625+0000","last_change":"2024-04-25T18:43:16.247552+0000","last_active":"2024-04-25T18:45:25.107625+0000","last_peered":"2024-04-25T18:45:25.107625+0000","last_clean":"2024-04-25T18:45:25.107625+0000","last_became_active":"2024-04-25T18:43:16.247178+0000","last_became_peered":"2024-04-25T18:43:16.247178+0000","last_unstale":"2024-04-25T18:45:25.107625+0000","last_undegraded":"2024-04-25T18:45:25.107625+0000","last_fullsized":"2024-04-25T18:45:25.107625+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:50.072986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.636660+0000","last_change":"2024-04-25T18:43:56.636197+0000","last_active":"2024-04-25T18:43:56.636660+0000","last_peered":"2024-04-25T18:43:56.636660+0000","last_clean":"2024-04-25T18:43:56.636660+0000","last_became_active":"2024-04-25T18:43:56.633431+0000","last_became_peered":"2024-04-25T18:43:56.633431+0000","last_unstale":"2024-04-25T18:43:56.636660+0000","last_undegraded":"2024-04-25T18:43:56.636660+0000","last_fullsized":"2024-04-25T18:43:56.636660+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:12.463489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628547+0000","last_change":"2024-04-25T18:43:12.202101+0000","last_active":"2024-04-25T18:43:56.628547+0000","last_peered":"2024-04-25T18:43:56.628547+0000","last_clean":"2024-04-25T18:43:56.628547+0000","last_became_active":"2024-04-25T18:43:12.201727+0000","last_became_peered":"2024-04-25T18:43:12.201727+0000","last_unstale":"2024-04-25T18:43:56.628547+0000","last_undegraded":"2024-04-25T18:43:56.628547+0000","last_fullsized":"2024-04-25T18:43:56.628547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:33.132072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.112941+0000","last_change":"2024-04-25T18:43:16.255433+0000","last_active":"2024-04-25T18:45:25.112941+0000","last_peered":"2024-04-25T18:45:25.112941+0000","last_clean":"2024-04-25T18:45:25.112941+0000","last_became_active":"2024-04-25T18:43:16.254866+0000","last_became_peered":"2024-04-25T18:43:16.254866+0000","last_unstale":"2024-04-25T18:45:25.112941+0000","last_undegraded":"2024-04-25T18:45:25.112941+0000","last_fullsized":"2024-04-25T18:45:25.112941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:07.876973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'12","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:24.239068+0000","last_change":"2024-04-25T18:43:56.656998+0000","last_active":"2024-04-25T18:45:24.239068+0000","last_peered":"2024-04-25T18:45:24.239068+0000","last_clean":"2024-04-25T18:45:24.239068+0000","last_became_active":"2024-04-25T18:43:56.647456+0000","last_became_peered":"2024-04-25T18:43:56.647456+0000","last_unstale":"2024-04-25T18:45:24.239068+0000","last_undegraded":"2024-04-25T18:45:24.239068+0000","last_fullsized":"2024-04-25T18:45:24.239068+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:00.719426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.111780+0000","last_change":"2024-04-25T18:43:16.252826+0000","last_active":"2024-04-25T18:45:25.111780+0000","last_peered":"2024-04-25T18:45:25.111780+0000","last_clean":"2024-04-25T18:45:25.111780+0000","last_became_active":"2024-04-25T18:43:16.252435+0000","last_became_peered":"2024-04-25T18:43:16.252435+0000","last_unstale":"2024-04-25T18:45:25.111780+0000","last_undegraded":"2024-04-25T18:45:25.111780+0000","last_fullsized":"2024-04-25T18:45:25.111780+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:13.401006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628653+0000","last_change":"2024-04-25T18:43:12.376121+0000","last_active":"2024-04-25T18:43:56.628653+0000","last_peered":"2024-04-25T18:43:56.628653+0000","last_clean":"2024-04-25T18:43:56.628653+0000","last_became_active":"2024-04-25T18:43:12.375757+0000","last_became_peered":"2024-04-25T18:43:12.375757+0000","last_unstale":"2024-04-25T18:43:56.628653+0000","last_undegraded":"2024-04-25T18:43:56.628653+0000","last_fullsized":"2024-04-25T18:43:56.628653+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:15.610442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.124747+0000","last_change":"2024-04-25T18:43:16.266504+0000","last_active":"2024-04-25T18:45:25.124747+0000","last_peered":"2024-04-25T18:45:25.124747+0000","last_clean":"2024-04-25T18:45:25.124747+0000","last_became_active":"2024-04-25T18:43:16.265048+0000","last_became_peered":"2024-04-25T18:43:16.265048+0000","last_unstale":"2024-04-25T18:45:25.124747+0000","last_undegraded":"2024-04-25T18:45:25.124747+0000","last_fullsized":"2024-04-25T18:45:25.124747+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:36.924872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104609+0000","last_change":"2024-04-25T18:43:12.207529+0000","last_active":"2024-04-25T18:43:57.104609+0000","last_peered":"2024-04-25T18:43:57.104609+0000","last_clean":"2024-04-25T18:43:57.104609+0000","last_became_active":"2024-04-25T18:43:12.207122+0000","last_became_peered":"2024-04-25T18:43:12.207122+0000","last_unstale":"2024-04-25T18:43:57.104609+0000","last_undegraded":"2024-04-25T18:43:57.104609+0000","last_fullsized":"2024-04-25T18:43:57.104609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:44.518497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.082553+0000","last_change":"2024-04-25T18:43:16.249616+0000","last_active":"2024-04-25T18:45:25.082553+0000","last_peered":"2024-04-25T18:45:25.082553+0000","last_clean":"2024-04-25T18:45:25.082553+0000","last_became_active":"2024-04-25T18:43:16.249252+0000","last_became_peered":"2024-04-25T18:43:16.249252+0000","last_unstale":"2024-04-25T18:45:25.082553+0000","last_undegraded":"2024-04-25T18:45:25.082553+0000","last_fullsized":"2024-04-25T18:45:25.082553+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:34.721513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.640003+0000","last_change":"2024-04-25T18:43:12.209304+0000","last_active":"2024-04-25T18:43:56.640003+0000","last_peered":"2024-04-25T18:43:56.640003+0000","last_clean":"2024-04-25T18:43:56.640003+0000","last_became_active":"2024-04-25T18:43:12.208887+0000","last_became_peered":"2024-04-25T18:43:12.208887+0000","last_unstale":"2024-04-25T18:43:56.640003+0000","last_undegraded":"2024-04-25T18:43:56.640003+0000","last_fullsized":"2024-04-25T18:43:56.640003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:22.049809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.123227+0000","last_change":"2024-04-25T18:43:16.262482+0000","last_active":"2024-04-25T18:45:25.123227+0000","last_peered":"2024-04-25T18:45:25.123227+0000","last_clean":"2024-04-25T18:45:25.123227+0000","last_became_active":"2024-04-25T18:43:16.262108+0000","last_became_peered":"2024-04-25T18:43:16.262108+0000","last_unstale":"2024-04-25T18:45:25.123227+0000","last_undegraded":"2024-04-25T18:45:25.123227+0000","last_fullsized":"2024-04-25T18:45:25.123227+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:14.777939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.625963+0000","last_change":"2024-04-25T18:43:12.205742+0000","last_active":"2024-04-25T18:43:56.625963+0000","last_peered":"2024-04-25T18:43:56.625963+0000","last_clean":"2024-04-25T18:43:56.625963+0000","last_became_active":"2024-04-25T18:43:12.205314+0000","last_became_peered":"2024-04-25T18:43:12.205314+0000","last_unstale":"2024-04-25T18:43:56.625963+0000","last_undegraded":"2024-04-25T18:43:56.625963+0000","last_fullsized":"2024-04-25T18:43:56.625963+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:49.641568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.124378+0000","last_change":"2024-04-25T18:43:16.266100+0000","last_active":"2024-04-25T18:45:25.124378+0000","last_peered":"2024-04-25T18:45:25.124378+0000","last_clean":"2024-04-25T18:45:25.124378+0000","last_became_active":"2024-04-25T18:43:16.263060+0000","last_became_peered":"2024-04-25T18:43:16.263060+0000","last_unstale":"2024-04-25T18:45:25.124378+0000","last_undegraded":"2024-04-25T18:45:25.124378+0000","last_fullsized":"2024-04-25T18:45:25.124378+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:38.269244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628308+0000","last_change":"2024-04-25T18:43:12.196054+0000","last_active":"2024-04-25T18:43:56.628308+0000","last_peered":"2024-04-25T18:43:56.628308+0000","last_clean":"2024-04-25T18:43:56.628308+0000","last_became_active":"2024-04-25T18:43:12.195503+0000","last_became_peered":"2024-04-25T18:43:12.195503+0000","last_unstale":"2024-04-25T18:43:56.628308+0000","last_undegraded":"2024-04-25T18:43:56.628308+0000","last_fullsized":"2024-04-25T18:43:56.628308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:55.187176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.117523+0000","last_change":"2024-04-25T18:43:16.242536+0000","last_active":"2024-04-25T18:45:25.117523+0000","last_peered":"2024-04-25T18:45:25.117523+0000","last_clean":"2024-04-25T18:45:25.117523+0000","last_became_active":"2024-04-25T18:43:16.241060+0000","last_became_peered":"2024-04-25T18:43:16.241060+0000","last_unstale":"2024-04-25T18:45:25.117523+0000","last_undegraded":"2024-04-25T18:45:25.117523+0000","last_fullsized":"2024-04-25T18:45:25.117523+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:38.306190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.653946+0000","last_change":"2024-04-25T18:43:56.653377+0000","last_active":"2024-04-25T18:43:56.653946+0000","last_peered":"2024-04-25T18:43:56.653946+0000","last_clean":"2024-04-25T18:43:56.653946+0000","last_became_active":"2024-04-25T18:43:56.645516+0000","last_became_peered":"2024-04-25T18:43:56.645516+0000","last_unstale":"2024-04-25T18:43:56.653946+0000","last_undegraded":"2024-04-25T18:43:56.653946+0000","last_fullsized":"2024-04-25T18:43:56.653946+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:11.189494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.109409+0000","last_change":"2024-04-25T18:43:16.247610+0000","last_active":"2024-04-25T18:45:25.109409+0000","last_peered":"2024-04-25T18:45:25.109409+0000","last_clean":"2024-04-25T18:45:25.109409+0000","last_became_active":"2024-04-25T18:43:16.247273+0000","last_became_peered":"2024-04-25T18:43:16.247273+0000","last_unstale":"2024-04-25T18:45:25.109409+0000","last_undegraded":"2024-04-25T18:45:25.109409+0000","last_fullsized":"2024-04-25T18:45:25.109409+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:26.151480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632903+0000","last_change":"2024-04-25T18:43:12.202011+0000","last_active":"2024-04-25T18:43:56.632903+0000","last_peered":"2024-04-25T18:43:56.632903+0000","last_clean":"2024-04-25T18:43:56.632903+0000","last_became_active":"2024-04-25T18:43:12.201447+0000","last_became_peered":"2024-04-25T18:43:12.201447+0000","last_unstale":"2024-04-25T18:43:56.632903+0000","last_undegraded":"2024-04-25T18:43:56.632903+0000","last_fullsized":"2024-04-25T18:43:56.632903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:48.094757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.116594+0000","last_change":"2024-04-25T18:43:16.249033+0000","last_active":"2024-04-25T18:45:25.116594+0000","last_peered":"2024-04-25T18:45:25.116594+0000","last_clean":"2024-04-25T18:45:25.116594+0000","last_became_active":"2024-04-25T18:43:16.248691+0000","last_became_peered":"2024-04-25T18:43:16.248691+0000","last_unstale":"2024-04-25T18:45:25.116594+0000","last_undegraded":"2024-04-25T18:45:25.116594+0000","last_fullsized":"2024-04-25T18:45:25.116594+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:37.335824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630038+0000","last_change":"2024-04-25T18:43:12.205134+0000","last_active":"2024-04-25T18:43:56.630038+0000","last_peered":"2024-04-25T18:43:56.630038+0000","last_clean":"2024-04-25T18:43:56.630038+0000","last_became_active":"2024-04-25T18:43:12.204568+0000","last_became_peered":"2024-04-25T18:43:12.204568+0000","last_unstale":"2024-04-25T18:43:56.630038+0000","last_undegraded":"2024-04-25T18:43:56.630038+0000","last_fullsized":"2024-04-25T18:43:56.630038+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:32.806580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.106892+0000","last_change":"2024-04-25T18:43:16.247337+0000","last_active":"2024-04-25T18:45:25.106892+0000","last_peered":"2024-04-25T18:45:25.106892+0000","last_clean":"2024-04-25T18:45:25.106892+0000","last_became_active":"2024-04-25T18:43:16.246971+0000","last_became_peered":"2024-04-25T18:43:16.246971+0000","last_unstale":"2024-04-25T18:45:25.106892+0000","last_undegraded":"2024-04-25T18:45:25.106892+0000","last_fullsized":"2024-04-25T18:45:25.106892+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.882941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.104959+0000","last_change":"2024-04-25T18:43:16.252196+0000","last_active":"2024-04-25T18:45:25.104959+0000","last_peered":"2024-04-25T18:45:25.104959+0000","last_clean":"2024-04-25T18:45:25.104959+0000","last_became_active":"2024-04-25T18:43:16.251715+0000","last_became_peered":"2024-04-25T18:43:16.251715+0000","last_unstale":"2024-04-25T18:45:25.104959+0000","last_undegraded":"2024-04-25T18:45:25.104959+0000","last_fullsized":"2024-04-25T18:45:25.104959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:17.218925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.101589+0000","last_change":"2024-04-25T18:43:16.247632+0000","last_active":"2024-04-25T18:45:25.101589+0000","last_peered":"2024-04-25T18:45:25.101589+0000","last_clean":"2024-04-25T18:45:25.101589+0000","last_became_active":"2024-04-25T18:43:16.247305+0000","last_became_peered":"2024-04-25T18:43:16.247305+0000","last_unstale":"2024-04-25T18:45:25.101589+0000","last_undegraded":"2024-04-25T18:45:25.101589+0000","last_fullsized":"2024-04-25T18:45:25.101589+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:52.576146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.118343+0000","last_change":"2024-04-25T18:43:16.266539+0000","last_active":"2024-04-25T18:45:25.118343+0000","last_peered":"2024-04-25T18:45:25.118343+0000","last_clean":"2024-04-25T18:45:25.118343+0000","last_became_active":"2024-04-25T18:43:16.266083+0000","last_became_peered":"2024-04-25T18:43:16.266083+0000","last_unstale":"2024-04-25T18:45:25.118343+0000","last_undegraded":"2024-04-25T18:45:25.118343+0000","last_fullsized":"2024-04-25T18:45:25.118343+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:49.589019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.125345+0000","last_change":"2024-04-25T18:43:16.264002+0000","last_active":"2024-04-25T18:45:25.125345+0000","last_peered":"2024-04-25T18:45:25.125345+0000","last_clean":"2024-04-25T18:45:25.125345+0000","last_became_active":"2024-04-25T18:43:16.261555+0000","last_became_peered":"2024-04-25T18:43:16.261555+0000","last_unstale":"2024-04-25T18:45:25.125345+0000","last_undegraded":"2024-04-25T18:45:25.125345+0000","last_fullsized":"2024-04-25T18:45:25.125345+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:12.637040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.115907+0000","last_change":"2024-04-25T18:43:16.270914+0000","last_active":"2024-04-25T18:45:25.115907+0000","last_peered":"2024-04-25T18:45:25.115907+0000","last_clean":"2024-04-25T18:45:25.115907+0000","last_became_active":"2024-04-25T18:43:16.270117+0000","last_became_peered":"2024-04-25T18:43:16.270117+0000","last_unstale":"2024-04-25T18:45:25.115907+0000","last_undegraded":"2024-04-25T18:45:25.115907+0000","last_fullsized":"2024-04-25T18:45:25.115907+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:19.874006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.104493+0000","last_change":"2024-04-25T18:43:16.247032+0000","last_active":"2024-04-25T18:45:25.104493+0000","last_peered":"2024-04-25T18:45:25.104493+0000","last_clean":"2024-04-25T18:45:25.104493+0000","last_became_active":"2024-04-25T18:43:16.246734+0000","last_became_peered":"2024-04-25T18:43:16.246734+0000","last_unstale":"2024-04-25T18:45:25.104493+0000","last_undegraded":"2024-04-25T18:45:25.104493+0000","last_fullsized":"2024-04-25T18:45:25.104493+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:39.159143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.099305+0000","last_change":"2024-04-25T18:43:16.245263+0000","last_active":"2024-04-25T18:45:25.099305+0000","last_peered":"2024-04-25T18:45:25.099305+0000","last_clean":"2024-04-25T18:45:25.099305+0000","last_became_active":"2024-04-25T18:43:16.244856+0000","last_became_peered":"2024-04-25T18:43:16.244856+0000","last_unstale":"2024-04-25T18:45:25.099305+0000","last_undegraded":"2024-04-25T18:45:25.099305+0000","last_fullsized":"2024-04-25T18:45:25.099305+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:36.157927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.103241+0000","last_change":"2024-04-25T18:43:16.253523+0000","last_active":"2024-04-25T18:45:25.103241+0000","last_peered":"2024-04-25T18:45:25.103241+0000","last_clean":"2024-04-25T18:45:25.103241+0000","last_became_active":"2024-04-25T18:43:16.252689+0000","last_became_peered":"2024-04-25T18:43:16.252689+0000","last_unstale":"2024-04-25T18:45:25.103241+0000","last_undegraded":"2024-04-25T18:45:25.103241+0000","last_fullsized":"2024-04-25T18:45:25.103241+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.035788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.105943+0000","last_change":"2024-04-25T18:43:16.255818+0000","last_active":"2024-04-25T18:45:25.105943+0000","last_peered":"2024-04-25T18:45:25.105943+0000","last_clean":"2024-04-25T18:45:25.105943+0000","last_became_active":"2024-04-25T18:43:16.255521+0000","last_became_peered":"2024-04-25T18:43:16.255521+0000","last_unstale":"2024-04-25T18:45:25.105943+0000","last_undegraded":"2024-04-25T18:45:25.105943+0000","last_fullsized":"2024-04-25T18:45:25.105943+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:15.457585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.108190+0000","last_change":"2024-04-25T18:43:16.259680+0000","last_active":"2024-04-25T18:45:25.108190+0000","last_peered":"2024-04-25T18:45:25.108190+0000","last_clean":"2024-04-25T18:45:25.108190+0000","last_became_active":"2024-04-25T18:43:16.250065+0000","last_became_peered":"2024-04-25T18:43:16.250065+0000","last_unstale":"2024-04-25T18:45:25.108190+0000","last_undegraded":"2024-04-25T18:45:25.108190+0000","last_fullsized":"2024-04-25T18:45:25.108190+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:44.974919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.117115+0000","last_change":"2024-04-25T18:43:16.239801+0000","last_active":"2024-04-25T18:45:25.117115+0000","last_peered":"2024-04-25T18:45:25.117115+0000","last_clean":"2024-04-25T18:45:25.117115+0000","last_became_active":"2024-04-25T18:43:16.239475+0000","last_became_peered":"2024-04-25T18:43:16.239475+0000","last_unstale":"2024-04-25T18:45:25.117115+0000","last_undegraded":"2024-04-25T18:45:25.117115+0000","last_fullsized":"2024-04-25T18:45:25.117115+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:57.407476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.124797+0000","last_change":"2024-04-25T18:43:16.270419+0000","last_active":"2024-04-25T18:45:25.124797+0000","last_peered":"2024-04-25T18:45:25.124797+0000","last_clean":"2024-04-25T18:45:25.124797+0000","last_became_active":"2024-04-25T18:43:16.269995+0000","last_became_peered":"2024-04-25T18:43:16.269995+0000","last_unstale":"2024-04-25T18:45:25.124797+0000","last_undegraded":"2024-04-25T18:45:25.124797+0000","last_fullsized":"2024-04-25T18:45:25.124797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:27.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.104737+0000","last_change":"2024-04-25T18:43:16.246948+0000","last_active":"2024-04-25T18:45:25.104737+0000","last_peered":"2024-04-25T18:45:25.104737+0000","last_clean":"2024-04-25T18:45:25.104737+0000","last_became_active":"2024-04-25T18:43:16.246616+0000","last_became_peered":"2024-04-25T18:43:16.246616+0000","last_unstale":"2024-04-25T18:45:25.104737+0000","last_undegraded":"2024-04-25T18:45:25.104737+0000","last_fullsized":"2024-04-25T18:45:25.104737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:51.817448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69115,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":761,"num_read_kb":452,"num_write":174,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":582,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252912,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":498757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1728512,"data_stored":1155211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710777,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":498993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972416,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":498913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234054,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1090984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430283,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":498382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":563148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":499653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":564448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117085,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":498693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":499401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691921,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1090668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:45:48.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:45:48.638 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:50.432 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:50 smithi112 ceph-mon[27864]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:50 smithi161 ceph-mon[32609]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:50.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:50 smithi171 ceph-mon[33266]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:45:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.976+0000 7fc129e00700 1 -- 172.21.15.112:0/2907126295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124108a30 msgr2=0x7fc124108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.976+0000 7fc129e00700 1 --2- 172.21.15.112:0/2907126295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124108a30 0x7fc124108e10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc114009990 tx=0x7fc114009ca0 comp rx=0 tx=0).stop 2024-04-25T18:45:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.976+0000 7fc129e00700 1 -- 172.21.15.112:0/2907126295 shutdown_connections 2024-04-25T18:45:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.977+0000 7fc129e00700 1 --2- 172.21.15.112:0/2907126295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc12406b0f0 0x7fc12410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.977+0000 7fc129e00700 1 --2- 172.21.15.112:0/2907126295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc124108a30 0x7fc124108e10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.977+0000 7fc129e00700 1 --2- 172.21.15.112:0/2907126295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc12406a750 0x7fc12406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.977+0000 7fc129e00700 1 -- 172.21.15.112:0/2907126295 >> 172.21.15.112:0/2907126295 conn(0x7fc1240788f0 msgr2=0x7fc124078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.977+0000 7fc129e00700 1 -- 172.21.15.112:0/2907126295 shutdown_connections 2024-04-25T18:45:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.977+0000 7fc129e00700 1 -- 172.21.15.112:0/2907126295 wait complete. 2024-04-25T18:45:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.979+0000 7fc129e00700 1 Processor -- start 2024-04-25T18:45:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.979+0000 7fc129e00700 1 -- start start 2024-04-25T18:45:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.979+0000 7fc129e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc12406a750 0x7fc124101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.979+0000 7fc129e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc12406b0f0 0x7fc124101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc129e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124108a30 0x7fc124102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc129e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc124112bd0 con 0x7fc124108a30 2024-04-25T18:45:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc129e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc124112a50 con 0x7fc12406b0f0 2024-04-25T18:45:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc129e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc124112d50 con 0x7fc12406a750 2024-04-25T18:45:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc123400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124108a30 0x7fc124102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc122a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc12406a750 0x7fc124101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc123400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124108a30 0x7fc124102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47592/0 (socket says 172.21.15.112:47592) 2024-04-25T18:45:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc123400700 1 -- 172.21.15.112:0/3233876208 learned_addr learned my addr 172.21.15.112:0/3233876208 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc122a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc12406a750 0x7fc124101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:40006/0 (socket says 172.21.15.112:40006) 2024-04-25T18:45:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc11be00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc12406b0f0 0x7fc124101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc122a00700 1 -- 172.21.15.112:0/3233876208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc12406b0f0 msgr2=0x7fc124101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc122a00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc12406b0f0 0x7fc124101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc122a00700 1 -- 172.21.15.112:0/3233876208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124108a30 msgr2=0x7fc124102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc122a00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124108a30 0x7fc124102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc122a00700 1 -- 172.21.15.112:0/3233876208 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1241a7600 con 0x7fc12406a750 2024-04-25T18:45:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.980+0000 7fc123400700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124108a30 0x7fc124102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.981+0000 7fc122a00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc12406a750 0x7fc124101a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc1140059d0 tx=0x7fc1140047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.981+0000 7fc128a00700 1 -- 172.21.15.112:0/3233876208 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114027070 con 0x7fc12406a750 2024-04-25T18:45:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.981+0000 7fc128a00700 1 -- 172.21.15.112:0/3233876208 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc114022470 con 0x7fc12406a750 2024-04-25T18:45:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.981+0000 7fc128a00700 1 -- 172.21.15.112:0/3233876208 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114017650 con 0x7fc12406a750 2024-04-25T18:45:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.981+0000 7fc129e00700 1 -- 172.21.15.112:0/3233876208 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1241a7890 con 0x7fc12406a750 2024-04-25T18:45:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.981+0000 7fc129e00700 1 -- 172.21.15.112:0/3233876208 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1241a7c90 con 0x7fc12406a750 2024-04-25T18:45:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.982+0000 7fc129e00700 1 -- 172.21.15.112:0/3233876208 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc124069730 con 0x7fc12406a750 2024-04-25T18:45:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.985+0000 7fc128a00700 1 -- 172.21.15.112:0/3233876208 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fc11402a070 con 0x7fc12406a750 2024-04-25T18:45:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.986+0000 7fc128a00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc1080a3c40 0x7fc1080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.986+0000 7fc128a00700 1 -- 172.21.15.112:0/3233876208 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fc1140d1530 con 0x7fc12406a750 2024-04-25T18:45:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.986+0000 7fc11be00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc1080a3c40 0x7fc1080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.987+0000 7fc11be00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc1080a3c40 0x7fc1080a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc124078f60 tx=0x7fc10c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:50.988+0000 7fc128a00700 1 -- 172.21.15.112:0/3233876208 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc114013070 con 0x7fc12406a750 2024-04-25T18:45:51.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.209+0000 7fc129e00700 1 -- 172.21.15.112:0/3233876208 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc12402e090 con 0x7fc1080a3c40 2024-04-25T18:45:51.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.221+0000 7fc128a00700 1 -- 172.21.15.112:0/3233876208 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258851 (secure 0 0 0) 0x7fc12402e090 con 0x7fc1080a3c40 2024-04-25T18:45:51.226 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:45:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 -- 172.21.15.112:0/3233876208 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc1080a3c40 msgr2=0x7fc1080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc1080a3c40 0x7fc1080a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc124078f60 tx=0x7fc10c009260 comp rx=0 tx=0).stop 2024-04-25T18:45:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 -- 172.21.15.112:0/3233876208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc12406a750 msgr2=0x7fc124101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc12406a750 0x7fc124101a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc1140059d0 tx=0x7fc1140047b0 comp rx=0 tx=0).stop 2024-04-25T18:45:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 -- 172.21.15.112:0/3233876208 shutdown_connections 2024-04-25T18:45:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fc1080a3c40 0x7fc1080a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124108a30 0x7fc124102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc12406a750 0x7fc124101a10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 --2- 172.21.15.112:0/3233876208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc12406b0f0 0x7fc124101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:51.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 -- 172.21.15.112:0/3233876208 >> 172.21.15.112:0/3233876208 conn(0x7fc1240788f0 msgr2=0x7fc1240ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:51.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 -- 172.21.15.112:0/3233876208 shutdown_connections 2024-04-25T18:45:51.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:51.226+0000 7fc129e00700 1 -- 172.21.15.112:0/3233876208 wait complete. 2024-04-25T18:45:51.229 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T18:45:52.278 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-04-25T18:45:51.158271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659717,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":927,"num_read_kb":586,"num_write":357,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181318,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379976,"kb_used_data":14184,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561848,"statfs":{"total":1151940427776,"available":1151551332352,"internally_reserved":0,"allocated":14524416,"data_stored":7957251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004523"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.122166+0000","last_change":"2024-04-25T18:43:16.262618+0000","last_active":"2024-04-25T18:45:25.122166+0000","last_peered":"2024-04-25T18:45:25.122166+0000","last_clean":"2024-04-25T18:45:25.122166+0000","last_became_active":"2024-04-25T18:43:16.262215+0000","last_became_peered":"2024-04-25T18:43:16.262215+0000","last_unstale":"2024-04-25T18:45:25.122166+0000","last_undegraded":"2024-04-25T18:45:25.122166+0000","last_fullsized":"2024-04-25T18:45:25.122166+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:40.319102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.123754+0000","last_change":"2024-04-25T18:43:16.254085+0000","last_active":"2024-04-25T18:45:25.123754+0000","last_peered":"2024-04-25T18:45:25.123754+0000","last_clean":"2024-04-25T18:45:25.123754+0000","last_became_active":"2024-04-25T18:43:16.248943+0000","last_became_peered":"2024-04-25T18:43:16.248943+0000","last_unstale":"2024-04-25T18:45:25.123754+0000","last_undegraded":"2024-04-25T18:45:25.123754+0000","last_fullsized":"2024-04-25T18:45:25.123754+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:53.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.123304+0000","last_change":"2024-04-25T18:43:16.252684+0000","last_active":"2024-04-25T18:45:25.123304+0000","last_peered":"2024-04-25T18:45:25.123304+0000","last_clean":"2024-04-25T18:45:25.123304+0000","last_became_active":"2024-04-25T18:43:16.252318+0000","last_became_peered":"2024-04-25T18:43:16.252318+0000","last_unstale":"2024-04-25T18:45:25.123304+0000","last_undegraded":"2024-04-25T18:45:25.123304+0000","last_fullsized":"2024-04-25T18:45:25.123304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:58.904865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.092379+0000","last_change":"2024-04-25T18:43:16.247059+0000","last_active":"2024-04-25T18:45:25.092379+0000","last_peered":"2024-04-25T18:45:25.092379+0000","last_clean":"2024-04-25T18:45:25.092379+0000","last_became_active":"2024-04-25T18:43:16.246655+0000","last_became_peered":"2024-04-25T18:43:16.246655+0000","last_unstale":"2024-04-25T18:45:25.092379+0000","last_undegraded":"2024-04-25T18:45:25.092379+0000","last_fullsized":"2024-04-25T18:45:25.092379+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:31.026579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.099041+0000","last_change":"2024-04-25T18:43:16.250033+0000","last_active":"2024-04-25T18:45:25.099041+0000","last_peered":"2024-04-25T18:45:25.099041+0000","last_clean":"2024-04-25T18:45:25.099041+0000","last_became_active":"2024-04-25T18:43:16.249017+0000","last_became_peered":"2024-04-25T18:43:16.249017+0000","last_unstale":"2024-04-25T18:45:25.099041+0000","last_undegraded":"2024-04-25T18:45:25.099041+0000","last_fullsized":"2024-04-25T18:45:25.099041+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:34.304263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.110424+0000","last_change":"2024-04-25T18:43:56.642410+0000","last_active":"2024-04-25T18:45:25.110424+0000","last_peered":"2024-04-25T18:45:25.110424+0000","last_clean":"2024-04-25T18:45:25.110424+0000","last_became_active":"2024-04-25T18:43:56.642009+0000","last_became_peered":"2024-04-25T18:43:56.642009+0000","last_unstale":"2024-04-25T18:45:25.110424+0000","last_undegraded":"2024-04-25T18:45:25.110424+0000","last_fullsized":"2024-04-25T18:45:25.110424+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:19.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.109133+0000","last_change":"2024-04-25T18:43:56.647158+0000","last_active":"2024-04-25T18:45:25.109133+0000","last_peered":"2024-04-25T18:45:25.109133+0000","last_clean":"2024-04-25T18:45:25.109133+0000","last_became_active":"2024-04-25T18:43:56.646779+0000","last_became_peered":"2024-04-25T18:43:56.646779+0000","last_unstale":"2024-04-25T18:45:25.109133+0000","last_undegraded":"2024-04-25T18:45:25.109133+0000","last_fullsized":"2024-04-25T18:45:25.109133+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:04:14.055762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.114034+0000","last_change":"2024-04-25T18:43:16.247607+0000","last_active":"2024-04-25T18:45:25.114034+0000","last_peered":"2024-04-25T18:45:25.114034+0000","last_clean":"2024-04-25T18:45:25.114034+0000","last_became_active":"2024-04-25T18:43:16.247234+0000","last_became_peered":"2024-04-25T18:43:16.247234+0000","last_unstale":"2024-04-25T18:45:25.114034+0000","last_undegraded":"2024-04-25T18:45:25.114034+0000","last_fullsized":"2024-04-25T18:45:25.114034+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:57.117900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.106758+0000","last_change":"2024-04-25T18:43:16.247909+0000","last_active":"2024-04-25T18:45:25.106758+0000","last_peered":"2024-04-25T18:45:25.106758+0000","last_clean":"2024-04-25T18:45:25.106758+0000","last_became_active":"2024-04-25T18:43:16.247533+0000","last_became_peered":"2024-04-25T18:43:16.247533+0000","last_unstale":"2024-04-25T18:45:25.106758+0000","last_undegraded":"2024-04-25T18:45:25.106758+0000","last_fullsized":"2024-04-25T18:45:25.106758+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:02.257832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.105378+0000","last_change":"2024-04-25T18:43:16.252088+0000","last_active":"2024-04-25T18:45:25.105378+0000","last_peered":"2024-04-25T18:45:25.105378+0000","last_clean":"2024-04-25T18:45:25.105378+0000","last_became_active":"2024-04-25T18:43:16.251580+0000","last_became_peered":"2024-04-25T18:43:16.251580+0000","last_unstale":"2024-04-25T18:45:25.105378+0000","last_undegraded":"2024-04-25T18:45:25.105378+0000","last_fullsized":"2024-04-25T18:45:25.105378+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:22.992087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.109045+0000","last_change":"2024-04-25T18:43:16.239696+0000","last_active":"2024-04-25T18:45:25.109045+0000","last_peered":"2024-04-25T18:45:25.109045+0000","last_clean":"2024-04-25T18:45:25.109045+0000","last_became_active":"2024-04-25T18:43:16.239354+0000","last_became_peered":"2024-04-25T18:43:16.239354+0000","last_unstale":"2024-04-25T18:45:25.109045+0000","last_undegraded":"2024-04-25T18:45:25.109045+0000","last_fullsized":"2024-04-25T18:45:25.109045+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:39.682138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.100506+0000","last_change":"2024-04-25T18:43:16.242487+0000","last_active":"2024-04-25T18:45:25.100506+0000","last_peered":"2024-04-25T18:45:25.100506+0000","last_clean":"2024-04-25T18:45:25.100506+0000","last_became_active":"2024-04-25T18:43:16.242158+0000","last_became_peered":"2024-04-25T18:43:16.242158+0000","last_unstale":"2024-04-25T18:45:25.100506+0000","last_undegraded":"2024-04-25T18:45:25.100506+0000","last_fullsized":"2024-04-25T18:45:25.100506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:36.477440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.118327+0000","last_change":"2024-04-25T18:43:16.244154+0000","last_active":"2024-04-25T18:45:25.118327+0000","last_peered":"2024-04-25T18:45:25.118327+0000","last_clean":"2024-04-25T18:45:25.118327+0000","last_became_active":"2024-04-25T18:43:16.243829+0000","last_became_peered":"2024-04-25T18:43:16.243829+0000","last_unstale":"2024-04-25T18:45:25.118327+0000","last_undegraded":"2024-04-25T18:45:25.118327+0000","last_fullsized":"2024-04-25T18:45:25.118327+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:43.262751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.124140+0000","last_change":"2024-04-25T18:43:16.261227+0000","last_active":"2024-04-25T18:45:25.124140+0000","last_peered":"2024-04-25T18:45:25.124140+0000","last_clean":"2024-04-25T18:45:25.124140+0000","last_became_active":"2024-04-25T18:43:16.250468+0000","last_became_peered":"2024-04-25T18:43:16.250468+0000","last_unstale":"2024-04-25T18:45:25.124140+0000","last_undegraded":"2024-04-25T18:45:25.124140+0000","last_fullsized":"2024-04-25T18:45:25.124140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:44.805643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.123276+0000","last_change":"2024-04-25T18:43:16.257956+0000","last_active":"2024-04-25T18:45:25.123276+0000","last_peered":"2024-04-25T18:45:25.123276+0000","last_clean":"2024-04-25T18:45:25.123276+0000","last_became_active":"2024-04-25T18:43:16.255126+0000","last_became_peered":"2024-04-25T18:43:16.255126+0000","last_unstale":"2024-04-25T18:45:25.123276+0000","last_undegraded":"2024-04-25T18:45:25.123276+0000","last_fullsized":"2024-04-25T18:45:25.123276+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:30.047165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.123810+0000","last_change":"2024-04-25T18:43:16.256208+0000","last_active":"2024-04-25T18:45:25.123810+0000","last_peered":"2024-04-25T18:45:25.123810+0000","last_clean":"2024-04-25T18:45:25.123810+0000","last_became_active":"2024-04-25T18:43:16.255614+0000","last_became_peered":"2024-04-25T18:43:16.255614+0000","last_unstale":"2024-04-25T18:45:25.123810+0000","last_undegraded":"2024-04-25T18:45:25.123810+0000","last_fullsized":"2024-04-25T18:45:25.123810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:46.700209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.126947+0000","last_change":"2024-04-25T18:43:16.262730+0000","last_active":"2024-04-25T18:45:25.126947+0000","last_peered":"2024-04-25T18:45:25.126947+0000","last_clean":"2024-04-25T18:45:25.126947+0000","last_became_active":"2024-04-25T18:43:16.262192+0000","last_became_peered":"2024-04-25T18:43:16.262192+0000","last_unstale":"2024-04-25T18:45:25.126947+0000","last_undegraded":"2024-04-25T18:45:25.126947+0000","last_fullsized":"2024-04-25T18:45:25.126947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:29.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.110843+0000","last_change":"2024-04-25T18:43:16.261161+0000","last_active":"2024-04-25T18:45:25.110843+0000","last_peered":"2024-04-25T18:45:25.110843+0000","last_clean":"2024-04-25T18:45:25.110843+0000","last_became_active":"2024-04-25T18:43:16.258217+0000","last_became_peered":"2024-04-25T18:43:16.258217+0000","last_unstale":"2024-04-25T18:45:25.110843+0000","last_undegraded":"2024-04-25T18:45:25.110843+0000","last_fullsized":"2024-04-25T18:45:25.110843+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.564609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.106213+0000","last_change":"2024-04-25T18:43:16.266660+0000","last_active":"2024-04-25T18:45:25.106213+0000","last_peered":"2024-04-25T18:45:25.106213+0000","last_clean":"2024-04-25T18:45:25.106213+0000","last_became_active":"2024-04-25T18:43:16.266295+0000","last_became_peered":"2024-04-25T18:43:16.266295+0000","last_unstale":"2024-04-25T18:45:25.106213+0000","last_undegraded":"2024-04-25T18:45:25.106213+0000","last_fullsized":"2024-04-25T18:45:25.106213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:42.406585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.643467+0000","last_change":"2024-04-25T18:43:12.200172+0000","last_active":"2024-04-25T18:43:56.643467+0000","last_peered":"2024-04-25T18:43:56.643467+0000","last_clean":"2024-04-25T18:43:56.643467+0000","last_became_active":"2024-04-25T18:43:12.199760+0000","last_became_peered":"2024-04-25T18:43:12.199760+0000","last_unstale":"2024-04-25T18:43:56.643467+0000","last_undegraded":"2024-04-25T18:43:56.643467+0000","last_fullsized":"2024-04-25T18:43:56.643467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:57.460274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.106430+0000","last_change":"2024-04-25T18:43:16.255989+0000","last_active":"2024-04-25T18:45:25.106430+0000","last_peered":"2024-04-25T18:45:25.106430+0000","last_clean":"2024-04-25T18:45:25.106430+0000","last_became_active":"2024-04-25T18:43:16.255114+0000","last_became_peered":"2024-04-25T18:43:16.255114+0000","last_unstale":"2024-04-25T18:45:25.106430+0000","last_undegraded":"2024-04-25T18:45:25.106430+0000","last_fullsized":"2024-04-25T18:45:25.106430+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:52.888876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.639724+0000","last_change":"2024-04-25T18:43:12.208194+0000","last_active":"2024-04-25T18:43:56.639724+0000","last_peered":"2024-04-25T18:43:56.639724+0000","last_clean":"2024-04-25T18:43:56.639724+0000","last_became_active":"2024-04-25T18:43:12.207801+0000","last_became_peered":"2024-04-25T18:43:12.207801+0000","last_unstale":"2024-04-25T18:43:56.639724+0000","last_undegraded":"2024-04-25T18:43:56.639724+0000","last_fullsized":"2024-04-25T18:43:56.639724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:24.742486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.128382+0000","last_change":"2024-04-25T18:43:16.268960+0000","last_active":"2024-04-25T18:45:25.128382+0000","last_peered":"2024-04-25T18:45:25.128382+0000","last_clean":"2024-04-25T18:45:25.128382+0000","last_became_active":"2024-04-25T18:43:16.263626+0000","last_became_peered":"2024-04-25T18:43:16.263626+0000","last_unstale":"2024-04-25T18:45:25.128382+0000","last_undegraded":"2024-04-25T18:45:25.128382+0000","last_fullsized":"2024-04-25T18:45:25.128382+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:57.721918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.110665+0000","last_change":"2024-04-25T18:43:57.110208+0000","last_active":"2024-04-25T18:43:57.110665+0000","last_peered":"2024-04-25T18:43:57.110665+0000","last_clean":"2024-04-25T18:43:57.110665+0000","last_became_active":"2024-04-25T18:43:57.106027+0000","last_became_peered":"2024-04-25T18:43:57.106027+0000","last_unstale":"2024-04-25T18:43:57.110665+0000","last_undegraded":"2024-04-25T18:43:57.110665+0000","last_fullsized":"2024-04-25T18:43:57.110665+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:20.819961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.118322+0000","last_change":"2024-04-25T18:43:16.250414+0000","last_active":"2024-04-25T18:45:25.118322+0000","last_peered":"2024-04-25T18:45:25.118322+0000","last_clean":"2024-04-25T18:45:25.118322+0000","last_became_active":"2024-04-25T18:43:16.249832+0000","last_became_peered":"2024-04-25T18:43:16.249832+0000","last_unstale":"2024-04-25T18:45:25.118322+0000","last_undegraded":"2024-04-25T18:45:25.118322+0000","last_fullsized":"2024-04-25T18:45:25.118322+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.577854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628858+0000","last_change":"2024-04-25T18:43:12.195099+0000","last_active":"2024-04-25T18:43:56.628858+0000","last_peered":"2024-04-25T18:43:56.628858+0000","last_clean":"2024-04-25T18:43:56.628858+0000","last_became_active":"2024-04-25T18:43:12.194726+0000","last_became_peered":"2024-04-25T18:43:12.194726+0000","last_unstale":"2024-04-25T18:43:56.628858+0000","last_undegraded":"2024-04-25T18:43:56.628858+0000","last_fullsized":"2024-04-25T18:43:56.628858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.866905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.107679+0000","last_change":"2024-04-25T18:43:16.245208+0000","last_active":"2024-04-25T18:45:25.107679+0000","last_peered":"2024-04-25T18:45:25.107679+0000","last_clean":"2024-04-25T18:45:25.107679+0000","last_became_active":"2024-04-25T18:43:16.244836+0000","last_became_peered":"2024-04-25T18:43:16.244836+0000","last_unstale":"2024-04-25T18:45:25.107679+0000","last_undegraded":"2024-04-25T18:45:25.107679+0000","last_fullsized":"2024-04-25T18:45:25.107679+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:35.917060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":165,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:46.469953+0000","last_change":"2024-04-25T18:43:12.198252+0000","last_active":"2024-04-25T18:45:46.469953+0000","last_peered":"2024-04-25T18:45:46.469953+0000","last_clean":"2024-04-25T18:45:46.469953+0000","last_became_active":"2024-04-25T18:43:12.197691+0000","last_became_peered":"2024-04-25T18:43:12.197691+0000","last_unstale":"2024-04-25T18:45:46.469953+0000","last_undegraded":"2024-04-25T18:45:46.469953+0000","last_fullsized":"2024-04-25T18:45:46.469953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.900012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.101318+0000","last_change":"2024-04-25T18:43:16.255576+0000","last_active":"2024-04-25T18:45:25.101318+0000","last_peered":"2024-04-25T18:45:25.101318+0000","last_clean":"2024-04-25T18:45:25.101318+0000","last_became_active":"2024-04-25T18:43:16.255195+0000","last_became_peered":"2024-04-25T18:43:16.255195+0000","last_unstale":"2024-04-25T18:45:25.101318+0000","last_undegraded":"2024-04-25T18:45:25.101318+0000","last_fullsized":"2024-04-25T18:45:25.101318+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:41.510872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626060+0000","last_change":"2024-04-25T18:43:12.200786+0000","last_active":"2024-04-25T18:43:56.626060+0000","last_peered":"2024-04-25T18:43:56.626060+0000","last_clean":"2024-04-25T18:43:56.626060+0000","last_became_active":"2024-04-25T18:43:12.200409+0000","last_became_peered":"2024-04-25T18:43:12.200409+0000","last_unstale":"2024-04-25T18:43:56.626060+0000","last_undegraded":"2024-04-25T18:43:56.626060+0000","last_fullsized":"2024-04-25T18:43:56.626060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:52.177787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.105803+0000","last_change":"2024-04-25T18:43:16.255563+0000","last_active":"2024-04-25T18:45:25.105803+0000","last_peered":"2024-04-25T18:45:25.105803+0000","last_clean":"2024-04-25T18:45:25.105803+0000","last_became_active":"2024-04-25T18:43:16.255174+0000","last_became_peered":"2024-04-25T18:43:16.255174+0000","last_unstale":"2024-04-25T18:45:25.105803+0000","last_undegraded":"2024-04-25T18:45:25.105803+0000","last_fullsized":"2024-04-25T18:45:25.105803+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:32:15.735051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628920+0000","last_change":"2024-04-25T18:43:12.210242+0000","last_active":"2024-04-25T18:43:56.628920+0000","last_peered":"2024-04-25T18:43:56.628920+0000","last_clean":"2024-04-25T18:43:56.628920+0000","last_became_active":"2024-04-25T18:43:12.209600+0000","last_became_peered":"2024-04-25T18:43:12.209600+0000","last_unstale":"2024-04-25T18:43:56.628920+0000","last_undegraded":"2024-04-25T18:43:56.628920+0000","last_fullsized":"2024-04-25T18:43:56.628920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.886712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.105558+0000","last_change":"2024-04-25T18:43:56.649102+0000","last_active":"2024-04-25T18:45:25.105558+0000","last_peered":"2024-04-25T18:45:25.105558+0000","last_clean":"2024-04-25T18:45:25.105558+0000","last_became_active":"2024-04-25T18:43:56.647127+0000","last_became_peered":"2024-04-25T18:43:56.647127+0000","last_unstale":"2024-04-25T18:45:25.105558+0000","last_undegraded":"2024-04-25T18:45:25.105558+0000","last_fullsized":"2024-04-25T18:45:25.105558+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:43.111539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632572+0000","last_change":"2024-04-25T18:43:12.198957+0000","last_active":"2024-04-25T18:43:56.632572+0000","last_peered":"2024-04-25T18:43:56.632572+0000","last_clean":"2024-04-25T18:43:56.632572+0000","last_became_active":"2024-04-25T18:43:12.198605+0000","last_became_peered":"2024-04-25T18:43:12.198605+0000","last_unstale":"2024-04-25T18:43:56.632572+0000","last_undegraded":"2024-04-25T18:43:56.632572+0000","last_fullsized":"2024-04-25T18:43:56.632572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:11.759430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.108032+0000","last_change":"2024-04-25T18:43:16.266829+0000","last_active":"2024-04-25T18:45:25.108032+0000","last_peered":"2024-04-25T18:45:25.108032+0000","last_clean":"2024-04-25T18:45:25.108032+0000","last_became_active":"2024-04-25T18:43:16.266256+0000","last_became_peered":"2024-04-25T18:43:16.266256+0000","last_unstale":"2024-04-25T18:45:25.108032+0000","last_undegraded":"2024-04-25T18:45:25.108032+0000","last_fullsized":"2024-04-25T18:45:25.108032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:43.343613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":157,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:45.467475+0000","last_change":"2024-04-25T18:43:12.201641+0000","last_active":"2024-04-25T18:45:45.467475+0000","last_peered":"2024-04-25T18:45:45.467475+0000","last_clean":"2024-04-25T18:45:45.467475+0000","last_became_active":"2024-04-25T18:43:12.201274+0000","last_became_peered":"2024-04-25T18:43:12.201274+0000","last_unstale":"2024-04-25T18:45:45.467475+0000","last_undegraded":"2024-04-25T18:45:45.467475+0000","last_fullsized":"2024-04-25T18:45:45.467475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:47.683850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.111637+0000","last_change":"2024-04-25T18:43:16.259551+0000","last_active":"2024-04-25T18:45:25.111637+0000","last_peered":"2024-04-25T18:45:25.111637+0000","last_clean":"2024-04-25T18:45:25.111637+0000","last_became_active":"2024-04-25T18:43:16.259165+0000","last_became_peered":"2024-04-25T18:43:16.259165+0000","last_unstale":"2024-04-25T18:45:25.111637+0000","last_undegraded":"2024-04-25T18:45:25.111637+0000","last_fullsized":"2024-04-25T18:45:25.111637+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:23.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630339+0000","last_change":"2024-04-25T18:43:12.197534+0000","last_active":"2024-04-25T18:43:56.630339+0000","last_peered":"2024-04-25T18:43:56.630339+0000","last_clean":"2024-04-25T18:43:56.630339+0000","last_became_active":"2024-04-25T18:43:12.197106+0000","last_became_peered":"2024-04-25T18:43:12.197106+0000","last_unstale":"2024-04-25T18:43:56.630339+0000","last_undegraded":"2024-04-25T18:43:56.630339+0000","last_fullsized":"2024-04-25T18:43:56.630339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:35.399044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.127461+0000","last_change":"2024-04-25T18:43:16.257668+0000","last_active":"2024-04-25T18:45:25.127461+0000","last_peered":"2024-04-25T18:45:25.127461+0000","last_clean":"2024-04-25T18:45:25.127461+0000","last_became_active":"2024-04-25T18:43:16.253609+0000","last_became_peered":"2024-04-25T18:43:16.253609+0000","last_unstale":"2024-04-25T18:45:25.127461+0000","last_undegraded":"2024-04-25T18:45:25.127461+0000","last_fullsized":"2024-04-25T18:45:25.127461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:36.048386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.633120+0000","last_change":"2024-04-25T18:43:12.207322+0000","last_active":"2024-04-25T18:43:56.633120+0000","last_peered":"2024-04-25T18:43:56.633120+0000","last_clean":"2024-04-25T18:43:56.633120+0000","last_became_active":"2024-04-25T18:43:12.206947+0000","last_became_peered":"2024-04-25T18:43:12.206947+0000","last_unstale":"2024-04-25T18:43:56.633120+0000","last_undegraded":"2024-04-25T18:43:56.633120+0000","last_fullsized":"2024-04-25T18:43:56.633120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:20.536598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.110098+0000","last_change":"2024-04-25T18:43:16.239575+0000","last_active":"2024-04-25T18:45:25.110098+0000","last_peered":"2024-04-25T18:45:25.110098+0000","last_clean":"2024-04-25T18:45:25.110098+0000","last_became_active":"2024-04-25T18:43:16.239208+0000","last_became_peered":"2024-04-25T18:43:16.239208+0000","last_unstale":"2024-04-25T18:45:25.110098+0000","last_undegraded":"2024-04-25T18:45:25.110098+0000","last_fullsized":"2024-04-25T18:45:25.110098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:58.195855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":324,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:46.542172+0000","last_change":"2024-04-25T18:43:12.197199+0000","last_active":"2024-04-25T18:45:46.542172+0000","last_peered":"2024-04-25T18:45:46.542172+0000","last_clean":"2024-04-25T18:45:46.542172+0000","last_became_active":"2024-04-25T18:43:12.196776+0000","last_became_peered":"2024-04-25T18:43:12.196776+0000","last_unstale":"2024-04-25T18:45:46.542172+0000","last_undegraded":"2024-04-25T18:45:46.542172+0000","last_fullsized":"2024-04-25T18:45:46.542172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:54.149782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64973,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":262,"num_read_kb":194,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.087909+0000","last_change":"2024-04-25T18:43:16.250993+0000","last_active":"2024-04-25T18:45:25.087909+0000","last_peered":"2024-04-25T18:45:25.087909+0000","last_clean":"2024-04-25T18:45:25.087909+0000","last_became_active":"2024-04-25T18:43:16.250216+0000","last_became_peered":"2024-04-25T18:43:16.250216+0000","last_unstale":"2024-04-25T18:45:25.087909+0000","last_undegraded":"2024-04-25T18:45:25.087909+0000","last_fullsized":"2024-04-25T18:45:25.087909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:37.996463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.626183+0000","last_change":"2024-04-25T18:43:12.204606+0000","last_active":"2024-04-25T18:43:56.626183+0000","last_peered":"2024-04-25T18:43:56.626183+0000","last_clean":"2024-04-25T18:43:56.626183+0000","last_became_active":"2024-04-25T18:43:12.204306+0000","last_became_peered":"2024-04-25T18:43:12.204306+0000","last_unstale":"2024-04-25T18:43:56.626183+0000","last_undegraded":"2024-04-25T18:43:56.626183+0000","last_fullsized":"2024-04-25T18:43:56.626183+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:29.903775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.115780+0000","last_change":"2024-04-25T18:43:16.250093+0000","last_active":"2024-04-25T18:45:25.115780+0000","last_peered":"2024-04-25T18:45:25.115780+0000","last_clean":"2024-04-25T18:45:25.115780+0000","last_became_active":"2024-04-25T18:43:16.249567+0000","last_became_peered":"2024-04-25T18:43:16.249567+0000","last_unstale":"2024-04-25T18:45:25.115780+0000","last_undegraded":"2024-04-25T18:45:25.115780+0000","last_fullsized":"2024-04-25T18:45:25.115780+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:35.498641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.652890+0000","last_change":"2024-04-25T18:43:56.652488+0000","last_active":"2024-04-25T18:43:56.652890+0000","last_peered":"2024-04-25T18:43:56.652890+0000","last_clean":"2024-04-25T18:43:56.652890+0000","last_became_active":"2024-04-25T18:43:56.647745+0000","last_became_peered":"2024-04-25T18:43:56.647745+0000","last_unstale":"2024-04-25T18:43:56.652890+0000","last_undegraded":"2024-04-25T18:43:56.652890+0000","last_fullsized":"2024-04-25T18:43:56.652890+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:33.474470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.117963+0000","last_change":"2024-04-25T18:43:56.642673+0000","last_active":"2024-04-25T18:45:25.117963+0000","last_peered":"2024-04-25T18:45:25.117963+0000","last_clean":"2024-04-25T18:45:25.117963+0000","last_became_active":"2024-04-25T18:43:56.642313+0000","last_became_peered":"2024-04-25T18:43:56.642313+0000","last_unstale":"2024-04-25T18:45:25.117963+0000","last_undegraded":"2024-04-25T18:45:25.117963+0000","last_fullsized":"2024-04-25T18:45:25.117963+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:45.805738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":161,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:48.504897+0000","last_change":"2024-04-25T18:43:12.197341+0000","last_active":"2024-04-25T18:45:48.504897+0000","last_peered":"2024-04-25T18:45:48.504897+0000","last_clean":"2024-04-25T18:45:48.504897+0000","last_became_active":"2024-04-25T18:43:12.196952+0000","last_became_peered":"2024-04-25T18:43:12.196952+0000","last_unstale":"2024-04-25T18:45:48.504897+0000","last_undegraded":"2024-04-25T18:45:48.504897+0000","last_fullsized":"2024-04-25T18:45:48.504897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:52.902579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.106340+0000","last_change":"2024-04-25T18:43:16.245436+0000","last_active":"2024-04-25T18:45:25.106340+0000","last_peered":"2024-04-25T18:45:25.106340+0000","last_clean":"2024-04-25T18:45:25.106340+0000","last_became_active":"2024-04-25T18:43:16.244943+0000","last_became_peered":"2024-04-25T18:43:16.244943+0000","last_unstale":"2024-04-25T18:45:25.106340+0000","last_undegraded":"2024-04-25T18:45:25.106340+0000","last_fullsized":"2024-04-25T18:45:25.106340+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:03.402999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628148+0000","last_change":"2024-04-25T18:43:12.197642+0000","last_active":"2024-04-25T18:43:56.628148+0000","last_peered":"2024-04-25T18:43:56.628148+0000","last_clean":"2024-04-25T18:43:56.628148+0000","last_became_active":"2024-04-25T18:43:12.196395+0000","last_became_peered":"2024-04-25T18:43:12.196395+0000","last_unstale":"2024-04-25T18:43:56.628148+0000","last_undegraded":"2024-04-25T18:43:56.628148+0000","last_fullsized":"2024-04-25T18:43:56.628148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:11.783422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.107052+0000","last_change":"2024-04-25T18:43:16.245344+0000","last_active":"2024-04-25T18:45:25.107052+0000","last_peered":"2024-04-25T18:45:25.107052+0000","last_clean":"2024-04-25T18:45:25.107052+0000","last_became_active":"2024-04-25T18:43:16.244984+0000","last_became_peered":"2024-04-25T18:43:16.244984+0000","last_unstale":"2024-04-25T18:45:25.107052+0000","last_undegraded":"2024-04-25T18:45:25.107052+0000","last_fullsized":"2024-04-25T18:45:25.107052+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:04.822629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628634+0000","last_change":"2024-04-25T18:43:12.199685+0000","last_active":"2024-04-25T18:43:56.628634+0000","last_peered":"2024-04-25T18:43:56.628634+0000","last_clean":"2024-04-25T18:43:56.628634+0000","last_became_active":"2024-04-25T18:43:12.199261+0000","last_became_peered":"2024-04-25T18:43:12.199261+0000","last_unstale":"2024-04-25T18:43:56.628634+0000","last_undegraded":"2024-04-25T18:43:56.628634+0000","last_fullsized":"2024-04-25T18:43:56.628634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:09.784616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.104269+0000","last_change":"2024-04-25T18:43:16.247745+0000","last_active":"2024-04-25T18:45:25.104269+0000","last_peered":"2024-04-25T18:45:25.104269+0000","last_clean":"2024-04-25T18:45:25.104269+0000","last_became_active":"2024-04-25T18:43:16.247311+0000","last_became_peered":"2024-04-25T18:43:16.247311+0000","last_unstale":"2024-04-25T18:45:25.104269+0000","last_undegraded":"2024-04-25T18:45:25.104269+0000","last_fullsized":"2024-04-25T18:45:25.104269+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:29.925464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.642742+0000","last_change":"2024-04-25T18:43:12.210209+0000","last_active":"2024-04-25T18:43:56.642742+0000","last_peered":"2024-04-25T18:43:56.642742+0000","last_clean":"2024-04-25T18:43:56.642742+0000","last_became_active":"2024-04-25T18:43:12.209708+0000","last_became_peered":"2024-04-25T18:43:12.209708+0000","last_unstale":"2024-04-25T18:43:56.642742+0000","last_undegraded":"2024-04-25T18:43:56.642742+0000","last_fullsized":"2024-04-25T18:43:56.642742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:07:18.331518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.112451+0000","last_change":"2024-04-25T18:43:16.263616+0000","last_active":"2024-04-25T18:45:25.112451+0000","last_peered":"2024-04-25T18:45:25.112451+0000","last_clean":"2024-04-25T18:45:25.112451+0000","last_became_active":"2024-04-25T18:43:16.263287+0000","last_became_peered":"2024-04-25T18:43:16.263287+0000","last_unstale":"2024-04-25T18:45:25.112451+0000","last_undegraded":"2024-04-25T18:45:25.112451+0000","last_fullsized":"2024-04-25T18:45:25.112451+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:46.366457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630374+0000","last_change":"2024-04-25T18:43:12.203705+0000","last_active":"2024-04-25T18:43:56.630374+0000","last_peered":"2024-04-25T18:43:56.630374+0000","last_clean":"2024-04-25T18:43:56.630374+0000","last_became_active":"2024-04-25T18:43:12.203371+0000","last_became_peered":"2024-04-25T18:43:12.203371+0000","last_unstale":"2024-04-25T18:43:56.630374+0000","last_undegraded":"2024-04-25T18:43:56.630374+0000","last_fullsized":"2024-04-25T18:43:56.630374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.119480+0000","last_change":"2024-04-25T18:43:56.647597+0000","last_active":"2024-04-25T18:45:25.119480+0000","last_peered":"2024-04-25T18:45:25.119480+0000","last_clean":"2024-04-25T18:45:25.119480+0000","last_became_active":"2024-04-25T18:43:56.647121+0000","last_became_peered":"2024-04-25T18:43:56.647121+0000","last_unstale":"2024-04-25T18:45:25.119480+0000","last_undegraded":"2024-04-25T18:45:25.119480+0000","last_fullsized":"2024-04-25T18:45:25.119480+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:07.143806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.100431+0000","last_change":"2024-04-25T18:43:16.247133+0000","last_active":"2024-04-25T18:45:25.100431+0000","last_peered":"2024-04-25T18:45:25.100431+0000","last_clean":"2024-04-25T18:45:25.100431+0000","last_became_active":"2024-04-25T18:43:16.246232+0000","last_became_peered":"2024-04-25T18:43:16.246232+0000","last_unstale":"2024-04-25T18:45:25.100431+0000","last_undegraded":"2024-04-25T18:45:25.100431+0000","last_fullsized":"2024-04-25T18:45:25.100431+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.288404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:47.502897+0000","last_change":"2024-04-25T18:43:12.209908+0000","last_active":"2024-04-25T18:45:47.502897+0000","last_peered":"2024-04-25T18:45:47.502897+0000","last_clean":"2024-04-25T18:45:47.502897+0000","last_became_active":"2024-04-25T18:43:12.209042+0000","last_became_peered":"2024-04-25T18:43:12.209042+0000","last_unstale":"2024-04-25T18:45:47.502897+0000","last_undegraded":"2024-04-25T18:45:47.502897+0000","last_fullsized":"2024-04-25T18:45:47.502897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:36.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.114435+0000","last_change":"2024-04-25T18:43:16.242364+0000","last_active":"2024-04-25T18:45:25.114435+0000","last_peered":"2024-04-25T18:45:25.114435+0000","last_clean":"2024-04-25T18:45:25.114435+0000","last_became_active":"2024-04-25T18:43:16.242019+0000","last_became_peered":"2024-04-25T18:43:16.242019+0000","last_unstale":"2024-04-25T18:45:25.114435+0000","last_undegraded":"2024-04-25T18:45:25.114435+0000","last_fullsized":"2024-04-25T18:45:25.114435+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:13.238129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628468+0000","last_change":"2024-04-25T18:43:12.206606+0000","last_active":"2024-04-25T18:43:56.628468+0000","last_peered":"2024-04-25T18:43:56.628468+0000","last_clean":"2024-04-25T18:43:56.628468+0000","last_became_active":"2024-04-25T18:43:12.206238+0000","last_became_peered":"2024-04-25T18:43:12.206238+0000","last_unstale":"2024-04-25T18:43:56.628468+0000","last_undegraded":"2024-04-25T18:43:56.628468+0000","last_fullsized":"2024-04-25T18:43:56.628468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:07.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630224+0000","last_change":"2024-04-25T18:41:40.527363+0000","last_active":"2024-04-25T18:43:56.630224+0000","last_peered":"2024-04-25T18:43:56.630224+0000","last_clean":"2024-04-25T18:43:56.630224+0000","last_became_active":"2024-04-25T18:41:40.526907+0000","last_became_peered":"2024-04-25T18:41:40.526907+0000","last_unstale":"2024-04-25T18:43:56.630224+0000","last_undegraded":"2024-04-25T18:43:56.630224+0000","last_fullsized":"2024-04-25T18:43:56.630224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_clean_scrub_stamp":"2024-04-25T18:37:45.963565+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:16.689693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.107625+0000","last_change":"2024-04-25T18:43:16.247552+0000","last_active":"2024-04-25T18:45:25.107625+0000","last_peered":"2024-04-25T18:45:25.107625+0000","last_clean":"2024-04-25T18:45:25.107625+0000","last_became_active":"2024-04-25T18:43:16.247178+0000","last_became_peered":"2024-04-25T18:43:16.247178+0000","last_unstale":"2024-04-25T18:45:25.107625+0000","last_undegraded":"2024-04-25T18:45:25.107625+0000","last_fullsized":"2024-04-25T18:45:25.107625+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:50.072986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.636660+0000","last_change":"2024-04-25T18:43:56.636197+0000","last_active":"2024-04-25T18:43:56.636660+0000","last_peered":"2024-04-25T18:43:56.636660+0000","last_clean":"2024-04-25T18:43:56.636660+0000","last_became_active":"2024-04-25T18:43:56.633431+0000","last_became_peered":"2024-04-25T18:43:56.633431+0000","last_unstale":"2024-04-25T18:43:56.636660+0000","last_undegraded":"2024-04-25T18:43:56.636660+0000","last_fullsized":"2024-04-25T18:43:56.636660+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:12.463489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628547+0000","last_change":"2024-04-25T18:43:12.202101+0000","last_active":"2024-04-25T18:43:56.628547+0000","last_peered":"2024-04-25T18:43:56.628547+0000","last_clean":"2024-04-25T18:43:56.628547+0000","last_became_active":"2024-04-25T18:43:12.201727+0000","last_became_peered":"2024-04-25T18:43:12.201727+0000","last_unstale":"2024-04-25T18:43:56.628547+0000","last_undegraded":"2024-04-25T18:43:56.628547+0000","last_fullsized":"2024-04-25T18:43:56.628547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:33.132072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.112941+0000","last_change":"2024-04-25T18:43:16.255433+0000","last_active":"2024-04-25T18:45:25.112941+0000","last_peered":"2024-04-25T18:45:25.112941+0000","last_clean":"2024-04-25T18:45:25.112941+0000","last_became_active":"2024-04-25T18:43:16.254866+0000","last_became_peered":"2024-04-25T18:43:16.254866+0000","last_unstale":"2024-04-25T18:45:25.112941+0000","last_undegraded":"2024-04-25T18:45:25.112941+0000","last_fullsized":"2024-04-25T18:45:25.112941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:07.876973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'12","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:24.239068+0000","last_change":"2024-04-25T18:43:56.656998+0000","last_active":"2024-04-25T18:45:24.239068+0000","last_peered":"2024-04-25T18:45:24.239068+0000","last_clean":"2024-04-25T18:45:24.239068+0000","last_became_active":"2024-04-25T18:43:56.647456+0000","last_became_peered":"2024-04-25T18:43:56.647456+0000","last_unstale":"2024-04-25T18:45:24.239068+0000","last_undegraded":"2024-04-25T18:45:24.239068+0000","last_fullsized":"2024-04-25T18:45:24.239068+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:00.719426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.111780+0000","last_change":"2024-04-25T18:43:16.252826+0000","last_active":"2024-04-25T18:45:25.111780+0000","last_peered":"2024-04-25T18:45:25.111780+0000","last_clean":"2024-04-25T18:45:25.111780+0000","last_became_active":"2024-04-25T18:43:16.252435+0000","last_became_peered":"2024-04-25T18:43:16.252435+0000","last_unstale":"2024-04-25T18:45:25.111780+0000","last_undegraded":"2024-04-25T18:45:25.111780+0000","last_fullsized":"2024-04-25T18:45:25.111780+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:13.401006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628653+0000","last_change":"2024-04-25T18:43:12.376121+0000","last_active":"2024-04-25T18:43:56.628653+0000","last_peered":"2024-04-25T18:43:56.628653+0000","last_clean":"2024-04-25T18:43:56.628653+0000","last_became_active":"2024-04-25T18:43:12.375757+0000","last_became_peered":"2024-04-25T18:43:12.375757+0000","last_unstale":"2024-04-25T18:43:56.628653+0000","last_undegraded":"2024-04-25T18:43:56.628653+0000","last_fullsized":"2024-04-25T18:43:56.628653+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:15.610442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.124747+0000","last_change":"2024-04-25T18:43:16.266504+0000","last_active":"2024-04-25T18:45:25.124747+0000","last_peered":"2024-04-25T18:45:25.124747+0000","last_clean":"2024-04-25T18:45:25.124747+0000","last_became_active":"2024-04-25T18:43:16.265048+0000","last_became_peered":"2024-04-25T18:43:16.265048+0000","last_unstale":"2024-04-25T18:45:25.124747+0000","last_undegraded":"2024-04-25T18:45:25.124747+0000","last_fullsized":"2024-04-25T18:45:25.124747+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:36.924872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:57.104609+0000","last_change":"2024-04-25T18:43:12.207529+0000","last_active":"2024-04-25T18:43:57.104609+0000","last_peered":"2024-04-25T18:43:57.104609+0000","last_clean":"2024-04-25T18:43:57.104609+0000","last_became_active":"2024-04-25T18:43:12.207122+0000","last_became_peered":"2024-04-25T18:43:12.207122+0000","last_unstale":"2024-04-25T18:43:57.104609+0000","last_undegraded":"2024-04-25T18:43:57.104609+0000","last_fullsized":"2024-04-25T18:43:57.104609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:44.518497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.082553+0000","last_change":"2024-04-25T18:43:16.249616+0000","last_active":"2024-04-25T18:45:25.082553+0000","last_peered":"2024-04-25T18:45:25.082553+0000","last_clean":"2024-04-25T18:45:25.082553+0000","last_became_active":"2024-04-25T18:43:16.249252+0000","last_became_peered":"2024-04-25T18:43:16.249252+0000","last_unstale":"2024-04-25T18:45:25.082553+0000","last_undegraded":"2024-04-25T18:45:25.082553+0000","last_fullsized":"2024-04-25T18:45:25.082553+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:34.721513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.640003+0000","last_change":"2024-04-25T18:43:12.209304+0000","last_active":"2024-04-25T18:43:56.640003+0000","last_peered":"2024-04-25T18:43:56.640003+0000","last_clean":"2024-04-25T18:43:56.640003+0000","last_became_active":"2024-04-25T18:43:12.208887+0000","last_became_peered":"2024-04-25T18:43:12.208887+0000","last_unstale":"2024-04-25T18:43:56.640003+0000","last_undegraded":"2024-04-25T18:43:56.640003+0000","last_fullsized":"2024-04-25T18:43:56.640003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:22.049809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.123227+0000","last_change":"2024-04-25T18:43:16.262482+0000","last_active":"2024-04-25T18:45:25.123227+0000","last_peered":"2024-04-25T18:45:25.123227+0000","last_clean":"2024-04-25T18:45:25.123227+0000","last_became_active":"2024-04-25T18:43:16.262108+0000","last_became_peered":"2024-04-25T18:43:16.262108+0000","last_unstale":"2024-04-25T18:45:25.123227+0000","last_undegraded":"2024-04-25T18:45:25.123227+0000","last_fullsized":"2024-04-25T18:45:25.123227+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:14.777939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.625963+0000","last_change":"2024-04-25T18:43:12.205742+0000","last_active":"2024-04-25T18:43:56.625963+0000","last_peered":"2024-04-25T18:43:56.625963+0000","last_clean":"2024-04-25T18:43:56.625963+0000","last_became_active":"2024-04-25T18:43:12.205314+0000","last_became_peered":"2024-04-25T18:43:12.205314+0000","last_unstale":"2024-04-25T18:43:56.625963+0000","last_undegraded":"2024-04-25T18:43:56.625963+0000","last_fullsized":"2024-04-25T18:43:56.625963+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:49.641568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.124378+0000","last_change":"2024-04-25T18:43:16.266100+0000","last_active":"2024-04-25T18:45:25.124378+0000","last_peered":"2024-04-25T18:45:25.124378+0000","last_clean":"2024-04-25T18:45:25.124378+0000","last_became_active":"2024-04-25T18:43:16.263060+0000","last_became_peered":"2024-04-25T18:43:16.263060+0000","last_unstale":"2024-04-25T18:45:25.124378+0000","last_undegraded":"2024-04-25T18:45:25.124378+0000","last_fullsized":"2024-04-25T18:45:25.124378+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:38.269244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.628308+0000","last_change":"2024-04-25T18:43:12.196054+0000","last_active":"2024-04-25T18:43:56.628308+0000","last_peered":"2024-04-25T18:43:56.628308+0000","last_clean":"2024-04-25T18:43:56.628308+0000","last_became_active":"2024-04-25T18:43:12.195503+0000","last_became_peered":"2024-04-25T18:43:12.195503+0000","last_unstale":"2024-04-25T18:43:56.628308+0000","last_undegraded":"2024-04-25T18:43:56.628308+0000","last_fullsized":"2024-04-25T18:43:56.628308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:55.187176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.117523+0000","last_change":"2024-04-25T18:43:16.242536+0000","last_active":"2024-04-25T18:45:25.117523+0000","last_peered":"2024-04-25T18:45:25.117523+0000","last_clean":"2024-04-25T18:45:25.117523+0000","last_became_active":"2024-04-25T18:43:16.241060+0000","last_became_peered":"2024-04-25T18:43:16.241060+0000","last_unstale":"2024-04-25T18:45:25.117523+0000","last_undegraded":"2024-04-25T18:45:25.117523+0000","last_fullsized":"2024-04-25T18:45:25.117523+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:38.306190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.653946+0000","last_change":"2024-04-25T18:43:56.653377+0000","last_active":"2024-04-25T18:43:56.653946+0000","last_peered":"2024-04-25T18:43:56.653946+0000","last_clean":"2024-04-25T18:43:56.653946+0000","last_became_active":"2024-04-25T18:43:56.645516+0000","last_became_peered":"2024-04-25T18:43:56.645516+0000","last_unstale":"2024-04-25T18:43:56.653946+0000","last_undegraded":"2024-04-25T18:43:56.653946+0000","last_fullsized":"2024-04-25T18:43:56.653946+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:11.189494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.109409+0000","last_change":"2024-04-25T18:43:16.247610+0000","last_active":"2024-04-25T18:45:25.109409+0000","last_peered":"2024-04-25T18:45:25.109409+0000","last_clean":"2024-04-25T18:45:25.109409+0000","last_became_active":"2024-04-25T18:43:16.247273+0000","last_became_peered":"2024-04-25T18:43:16.247273+0000","last_unstale":"2024-04-25T18:45:25.109409+0000","last_undegraded":"2024-04-25T18:45:25.109409+0000","last_fullsized":"2024-04-25T18:45:25.109409+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:26.151480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.632903+0000","last_change":"2024-04-25T18:43:12.202011+0000","last_active":"2024-04-25T18:43:56.632903+0000","last_peered":"2024-04-25T18:43:56.632903+0000","last_clean":"2024-04-25T18:43:56.632903+0000","last_became_active":"2024-04-25T18:43:12.201447+0000","last_became_peered":"2024-04-25T18:43:12.201447+0000","last_unstale":"2024-04-25T18:43:56.632903+0000","last_undegraded":"2024-04-25T18:43:56.632903+0000","last_fullsized":"2024-04-25T18:43:56.632903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:48.094757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.116594+0000","last_change":"2024-04-25T18:43:16.249033+0000","last_active":"2024-04-25T18:45:25.116594+0000","last_peered":"2024-04-25T18:45:25.116594+0000","last_clean":"2024-04-25T18:45:25.116594+0000","last_became_active":"2024-04-25T18:43:16.248691+0000","last_became_peered":"2024-04-25T18:43:16.248691+0000","last_unstale":"2024-04-25T18:45:25.116594+0000","last_undegraded":"2024-04-25T18:45:25.116594+0000","last_fullsized":"2024-04-25T18:45:25.116594+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:37.335824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:43:56.630038+0000","last_change":"2024-04-25T18:43:12.205134+0000","last_active":"2024-04-25T18:43:56.630038+0000","last_peered":"2024-04-25T18:43:56.630038+0000","last_clean":"2024-04-25T18:43:56.630038+0000","last_became_active":"2024-04-25T18:43:12.204568+0000","last_became_peered":"2024-04-25T18:43:12.204568+0000","last_unstale":"2024-04-25T18:43:56.630038+0000","last_undegraded":"2024-04-25T18:43:56.630038+0000","last_fullsized":"2024-04-25T18:43:56.630038+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:32.806580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.106892+0000","last_change":"2024-04-25T18:43:16.247337+0000","last_active":"2024-04-25T18:45:25.106892+0000","last_peered":"2024-04-25T18:45:25.106892+0000","last_clean":"2024-04-25T18:45:25.106892+0000","last_became_active":"2024-04-25T18:43:16.246971+0000","last_became_peered":"2024-04-25T18:43:16.246971+0000","last_unstale":"2024-04-25T18:45:25.106892+0000","last_undegraded":"2024-04-25T18:45:25.106892+0000","last_fullsized":"2024-04-25T18:45:25.106892+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.882941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.104959+0000","last_change":"2024-04-25T18:43:16.252196+0000","last_active":"2024-04-25T18:45:25.104959+0000","last_peered":"2024-04-25T18:45:25.104959+0000","last_clean":"2024-04-25T18:45:25.104959+0000","last_became_active":"2024-04-25T18:43:16.251715+0000","last_became_peered":"2024-04-25T18:43:16.251715+0000","last_unstale":"2024-04-25T18:45:25.104959+0000","last_undegraded":"2024-04-25T18:45:25.104959+0000","last_fullsized":"2024-04-25T18:45:25.104959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:17.218925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.101589+0000","last_change":"2024-04-25T18:43:16.247632+0000","last_active":"2024-04-25T18:45:25.101589+0000","last_peered":"2024-04-25T18:45:25.101589+0000","last_clean":"2024-04-25T18:45:25.101589+0000","last_became_active":"2024-04-25T18:43:16.247305+0000","last_became_peered":"2024-04-25T18:43:16.247305+0000","last_unstale":"2024-04-25T18:45:25.101589+0000","last_undegraded":"2024-04-25T18:45:25.101589+0000","last_fullsized":"2024-04-25T18:45:25.101589+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:52.576146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.118343+0000","last_change":"2024-04-25T18:43:16.266539+0000","last_active":"2024-04-25T18:45:25.118343+0000","last_peered":"2024-04-25T18:45:25.118343+0000","last_clean":"2024-04-25T18:45:25.118343+0000","last_became_active":"2024-04-25T18:43:16.266083+0000","last_became_peered":"2024-04-25T18:43:16.266083+0000","last_unstale":"2024-04-25T18:45:25.118343+0000","last_undegraded":"2024-04-25T18:45:25.118343+0000","last_fullsized":"2024-04-25T18:45:25.118343+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:49.589019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.125345+0000","last_change":"2024-04-25T18:43:16.264002+0000","last_active":"2024-04-25T18:45:25.125345+0000","last_peered":"2024-04-25T18:45:25.125345+0000","last_clean":"2024-04-25T18:45:25.125345+0000","last_became_active":"2024-04-25T18:43:16.261555+0000","last_became_peered":"2024-04-25T18:43:16.261555+0000","last_unstale":"2024-04-25T18:45:25.125345+0000","last_undegraded":"2024-04-25T18:45:25.125345+0000","last_fullsized":"2024-04-25T18:45:25.125345+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:12.637040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.115907+0000","last_change":"2024-04-25T18:43:16.270914+0000","last_active":"2024-04-25T18:45:25.115907+0000","last_peered":"2024-04-25T18:45:25.115907+0000","last_clean":"2024-04-25T18:45:25.115907+0000","last_became_active":"2024-04-25T18:43:16.270117+0000","last_became_peered":"2024-04-25T18:43:16.270117+0000","last_unstale":"2024-04-25T18:45:25.115907+0000","last_undegraded":"2024-04-25T18:45:25.115907+0000","last_fullsized":"2024-04-25T18:45:25.115907+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:19.874006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.104493+0000","last_change":"2024-04-25T18:43:16.247032+0000","last_active":"2024-04-25T18:45:25.104493+0000","last_peered":"2024-04-25T18:45:25.104493+0000","last_clean":"2024-04-25T18:45:25.104493+0000","last_became_active":"2024-04-25T18:43:16.246734+0000","last_became_peered":"2024-04-25T18:43:16.246734+0000","last_unstale":"2024-04-25T18:45:25.104493+0000","last_undegraded":"2024-04-25T18:45:25.104493+0000","last_fullsized":"2024-04-25T18:45:25.104493+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:39.159143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.099305+0000","last_change":"2024-04-25T18:43:16.245263+0000","last_active":"2024-04-25T18:45:25.099305+0000","last_peered":"2024-04-25T18:45:25.099305+0000","last_clean":"2024-04-25T18:45:25.099305+0000","last_became_active":"2024-04-25T18:43:16.244856+0000","last_became_peered":"2024-04-25T18:43:16.244856+0000","last_unstale":"2024-04-25T18:45:25.099305+0000","last_undegraded":"2024-04-25T18:45:25.099305+0000","last_fullsized":"2024-04-25T18:45:25.099305+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:36.157927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.103241+0000","last_change":"2024-04-25T18:43:16.253523+0000","last_active":"2024-04-25T18:45:25.103241+0000","last_peered":"2024-04-25T18:45:25.103241+0000","last_clean":"2024-04-25T18:45:25.103241+0000","last_became_active":"2024-04-25T18:43:16.252689+0000","last_became_peered":"2024-04-25T18:43:16.252689+0000","last_unstale":"2024-04-25T18:45:25.103241+0000","last_undegraded":"2024-04-25T18:45:25.103241+0000","last_fullsized":"2024-04-25T18:45:25.103241+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.035788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.105943+0000","last_change":"2024-04-25T18:43:16.255818+0000","last_active":"2024-04-25T18:45:25.105943+0000","last_peered":"2024-04-25T18:45:25.105943+0000","last_clean":"2024-04-25T18:45:25.105943+0000","last_became_active":"2024-04-25T18:43:16.255521+0000","last_became_peered":"2024-04-25T18:43:16.255521+0000","last_unstale":"2024-04-25T18:45:25.105943+0000","last_undegraded":"2024-04-25T18:45:25.105943+0000","last_fullsized":"2024-04-25T18:45:25.105943+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:15.457585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.108190+0000","last_change":"2024-04-25T18:43:16.259680+0000","last_active":"2024-04-25T18:45:25.108190+0000","last_peered":"2024-04-25T18:45:25.108190+0000","last_clean":"2024-04-25T18:45:25.108190+0000","last_became_active":"2024-04-25T18:43:16.250065+0000","last_became_peered":"2024-04-25T18:43:16.250065+0000","last_unstale":"2024-04-25T18:45:25.108190+0000","last_undegraded":"2024-04-25T18:45:25.108190+0000","last_fullsized":"2024-04-25T18:45:25.108190+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:44.974919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.117115+0000","last_change":"2024-04-25T18:43:16.239801+0000","last_active":"2024-04-25T18:45:25.117115+0000","last_peered":"2024-04-25T18:45:25.117115+0000","last_clean":"2024-04-25T18:45:25.117115+0000","last_became_active":"2024-04-25T18:43:16.239475+0000","last_became_peered":"2024-04-25T18:43:16.239475+0000","last_unstale":"2024-04-25T18:45:25.117115+0000","last_undegraded":"2024-04-25T18:45:25.117115+0000","last_fullsized":"2024-04-25T18:45:25.117115+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:57.407476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.124797+0000","last_change":"2024-04-25T18:43:16.270419+0000","last_active":"2024-04-25T18:45:25.124797+0000","last_peered":"2024-04-25T18:45:25.124797+0000","last_clean":"2024-04-25T18:45:25.124797+0000","last_became_active":"2024-04-25T18:43:16.269995+0000","last_became_peered":"2024-04-25T18:43:16.269995+0000","last_unstale":"2024-04-25T18:45:25.124797+0000","last_undegraded":"2024-04-25T18:45:25.124797+0000","last_fullsized":"2024-04-25T18:45:25.124797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:27.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:45:25.104737+0000","last_change":"2024-04-25T18:43:16.246948+0000","last_active":"2024-04-25T18:45:25.104737+0000","last_peered":"2024-04-25T18:45:25.104737+0000","last_clean":"2024-04-25T18:45:25.104737+0000","last_became_active":"2024-04-25T18:43:16.246616+0000","last_became_peered":"2024-04-25T18:43:16.246616+0000","last_unstale":"2024-04-25T18:45:25.104737+0000","last_undegraded":"2024-04-25T18:45:25.104737+0000","last_fullsized":"2024-04-25T18:45:25.104737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:51.817448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69115,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":791,"num_read_kb":467,"num_write":174,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":582,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":498757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514550,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1728512,"data_stored":1155211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710778,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":498993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972417,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":498913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234056,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1090984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430284,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":498382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":563148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":499653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":564448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117086,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":498693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":499401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691922,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1090668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:45:52.282 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T18:45:52.283 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T18:45:52.283 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T18:45:52.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T18:45:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:52 smithi161 ceph-mon[32609]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:52 smithi161 ceph-mon[32609]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:45:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:52 smithi112 ceph-mon[27864]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:52 smithi112 ceph-mon[27864]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:45:52.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:52 smithi171 ceph-mon[33266]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:52.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:52 smithi171 ceph-mon[33266]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:45:52.768 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:54.650 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:54 smithi112 ceph-mon[27864]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:54.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:54 smithi171 ceph-mon[33266]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:54.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:54 smithi161 ceph-mon[32609]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.147+0000 7f27d7400700 1 -- 172.21.15.112:0/2335461549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d0108a30 msgr2=0x7f27d0108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.147+0000 7f27d7400700 1 --2- 172.21.15.112:0/2335461549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d0108a30 0x7f27d0108e10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f27c000b0a0 tx=0x7f27c000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:45:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.147+0000 7f27d7400700 1 -- 172.21.15.112:0/2335461549 shutdown_connections 2024-04-25T18:45:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.147+0000 7f27d7400700 1 --2- 172.21.15.112:0/2335461549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d006b0f0 0x7f27d010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.147+0000 7f27d7400700 1 --2- 172.21.15.112:0/2335461549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d0108a30 0x7f27d0108e10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.147+0000 7f27d7400700 1 --2- 172.21.15.112:0/2335461549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006a750 0x7f27d006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.147+0000 7f27d7400700 1 -- 172.21.15.112:0/2335461549 >> 172.21.15.112:0/2335461549 conn(0x7f27d00788f0 msgr2=0x7f27d0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.147+0000 7f27d7400700 1 -- 172.21.15.112:0/2335461549 shutdown_connections 2024-04-25T18:45:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.147+0000 7f27d7400700 1 -- 172.21.15.112:0/2335461549 wait complete. 2024-04-25T18:45:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.148+0000 7f27d7400700 1 Processor -- start 2024-04-25T18:45:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.148+0000 7f27d7400700 1 -- start start 2024-04-25T18:45:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.149+0000 7f27d7400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006a750 0x7f27d019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.149+0000 7f27d7400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d006b0f0 0x7f27d019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.149+0000 7f27d7400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d0108a30 0x7f27d019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.149+0000 7f27d7400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0112c20 con 0x7f27d0108a30 2024-04-25T18:45:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.149+0000 7f27d7400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0112aa0 con 0x7f27d006a750 2024-04-25T18:45:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.149+0000 7f27d7400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0112da0 con 0x7f27d006b0f0 2024-04-25T18:45:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.149+0000 7f27c7e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d006b0f0 0x7f27d019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.149+0000 7f27c7e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d006b0f0 0x7f27d019b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:40034/0 (socket says 172.21.15.112:40034) 2024-04-25T18:45:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.149+0000 7f27d4a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006a750 0x7f27d019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:55.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.149+0000 7f27c7e00700 1 -- 172.21.15.112:0/3591933656 learned_addr learned my addr 172.21.15.112:0/3591933656 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:55.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.149+0000 7f27d5400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d0108a30 0x7f27d019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27c7e00700 1 -- 172.21.15.112:0/3591933656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006a750 msgr2=0x7f27d019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27c7e00700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006a750 0x7f27d019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27c7e00700 1 -- 172.21.15.112:0/3591933656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d0108a30 msgr2=0x7f27d019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27c7e00700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d0108a30 0x7f27d019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27c7e00700 1 -- 172.21.15.112:0/3591933656 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d01a0360 con 0x7f27d006b0f0 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27d4a00700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006a750 0x7f27d019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27c7e00700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d006b0f0 0x7f27d019b890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f27b800d980 tx=0x7f27b800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27ce000700 1 -- 172.21.15.112:0/3591933656 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8016070 con 0x7f27d006b0f0 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27d7400700 1 -- 172.21.15.112:0/3591933656 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d01a0650 con 0x7f27d006b0f0 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27ce000700 1 -- 172.21.15.112:0/3591933656 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b8004830 con 0x7f27d006b0f0 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27ce000700 1 -- 172.21.15.112:0/3591933656 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8005540 con 0x7f27d006b0f0 2024-04-25T18:45:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.150+0000 7f27d7400700 1 -- 172.21.15.112:0/3591933656 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d01a7ed0 con 0x7f27d006b0f0 2024-04-25T18:45:55.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.151+0000 7f27d7400700 1 -- 172.21.15.112:0/3591933656 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ac005330 con 0x7f27d006b0f0 2024-04-25T18:45:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.154+0000 7f27ce000700 1 -- 172.21.15.112:0/3591933656 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f27b8004220 con 0x7f27d006b0f0 2024-04-25T18:45:55.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.155+0000 7f27ce000700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27b40a3c40 0x7f27b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:55.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.155+0000 7f27ce000700 1 -- 172.21.15.112:0/3591933656 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f27b80d05f0 con 0x7f27d006b0f0 2024-04-25T18:45:55.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.155+0000 7f27d4a00700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27b40a3c40 0x7f27b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:55.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.156+0000 7f27d4a00700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27b40a3c40 0x7f27b40a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f27c00062a0 tx=0x7f27c0006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:55.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.157+0000 7f27ce000700 1 -- 172.21.15.112:0/3591933656 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27b801f020 con 0x7f27d006b0f0 2024-04-25T18:45:55.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.432+0000 7f27d7400700 1 -- 172.21.15.112:0/3591933656 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f27ac005620 con 0x7f27d006b0f0 2024-04-25T18:45:55.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.432+0000 7f27ce000700 1 -- 172.21.15.112:0/3591933656 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f27b809cf50 con 0x7f27d006b0f0 2024-04-25T18:45:55.434 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:45:55.435 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T18:45:55.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.436+0000 7f27d7400700 1 -- 172.21.15.112:0/3591933656 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27b40a3c40 msgr2=0x7f27b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.436+0000 7f27d7400700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27b40a3c40 0x7f27b40a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f27c00062a0 tx=0x7f27c0006210 comp rx=0 tx=0).stop 2024-04-25T18:45:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.436+0000 7f27d7400700 1 -- 172.21.15.112:0/3591933656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d006b0f0 msgr2=0x7f27d019b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.436+0000 7f27d7400700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d006b0f0 0x7f27d019b890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f27b800d980 tx=0x7f27b800dc90 comp rx=0 tx=0).stop 2024-04-25T18:45:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.436+0000 7f27d7400700 1 -- 172.21.15.112:0/3591933656 shutdown_connections 2024-04-25T18:45:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.436+0000 7f27d7400700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f27b40a3c40 0x7f27b40a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.436+0000 7f27d7400700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27d0108a30 0x7f27d019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.436+0000 7f27d7400700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006a750 0x7f27d019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.436+0000 7f27d7400700 1 --2- 172.21.15.112:0/3591933656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d006b0f0 0x7f27d019b890 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.437+0000 7f27d7400700 1 -- 172.21.15.112:0/3591933656 >> 172.21.15.112:0/3591933656 conn(0x7f27d00788f0 msgr2=0x7f27d010de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.437+0000 7f27d7400700 1 -- 172.21.15.112:0/3591933656 shutdown_connections 2024-04-25T18:45:55.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:55.437+0000 7f27d7400700 1 -- 172.21.15.112:0/3591933656 wait complete. 2024-04-25T18:45:56.445 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T18:45:56.445 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:45:56.521 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:56 smithi112 ceph-mon[27864]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:56.521 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:56 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3591933656' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T18:45:56.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:56 smithi171 ceph-mon[33266]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:56.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:56 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3591933656' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T18:45:56.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:56 smithi161 ceph-mon[32609]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:45:56.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:56 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3591933656' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T18:45:56.895 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:45:58.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:58 smithi112 ceph-mon[27864]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:45:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:58 smithi171 ceph-mon[33266]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:45:58.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:58 smithi161 ceph-mon[32609]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:45:59.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.291+0000 7f06c7a00700 1 -- 172.21.15.112:0/474993002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06c0076f70 msgr2=0x7f06c0077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:59.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.291+0000 7f06c7a00700 1 --2- 172.21.15.112:0/474993002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06c0076f70 0x7f06c0077350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06b40098f0 tx=0x7f06b4009c00 comp rx=0 tx=0).stop 2024-04-25T18:45:59.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.291+0000 7f06c7a00700 1 -- 172.21.15.112:0/474993002 shutdown_connections 2024-04-25T18:45:59.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.291+0000 7f06c7a00700 1 --2- 172.21.15.112:0/474993002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c0113920 0x7f06c0115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:59.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.291+0000 7f06c7a00700 1 --2- 172.21.15.112:0/474993002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06c0076f70 0x7f06c0077350 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:59.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.291+0000 7f06c7a00700 1 --2- 172.21.15.112:0/474993002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06c0077890 0x7f06c01133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.291+0000 7f06c7a00700 1 -- 172.21.15.112:0/474993002 >> 172.21.15.112:0/474993002 conn(0x7f06c0100770 msgr2=0x7f06c0102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.292+0000 7f06c7a00700 1 -- 172.21.15.112:0/474993002 shutdown_connections 2024-04-25T18:45:59.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.292+0000 7f06c7a00700 1 -- 172.21.15.112:0/474993002 wait complete. 2024-04-25T18:45:59.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.293+0000 7f06c7a00700 1 Processor -- start 2024-04-25T18:45:59.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.293+0000 7f06c7a00700 1 -- start start 2024-04-25T18:45:59.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.293+0000 7f06c7a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06c0076f70 0x7f06c01a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:59.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.293+0000 7f06c7a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c0077890 0x7f06c01a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:59.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.293+0000 7f06c7a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06c0113920 0x7f06c019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:59.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.293+0000 7f06c7a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06c01192c0 con 0x7f06c0077890 2024-04-25T18:45:59.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.293+0000 7f06c7a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f06c0119140 con 0x7f06c0113920 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.293+0000 7f06c7a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f06c0119440 con 0x7f06c0076f70 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.294+0000 7f06bfe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c0077890 0x7f06c01a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.294+0000 7f06c5000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06c0076f70 0x7f06c01a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.294+0000 7f06c5000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06c0076f70 0x7f06c01a55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:40054/0 (socket says 172.21.15.112:40054) 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.294+0000 7f06c5000700 1 -- 172.21.15.112:0/2003356869 learned_addr learned my addr 172.21.15.112:0/2003356869 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.294+0000 7f06c5a00700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06c0113920 0x7f06c019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.294+0000 7f06c5000700 1 -- 172.21.15.112:0/2003356869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06c0113920 msgr2=0x7f06c019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.294+0000 7f06c5000700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06c0113920 0x7f06c019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.294+0000 7f06c5000700 1 -- 172.21.15.112:0/2003356869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c0077890 msgr2=0x7f06c01a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.294+0000 7f06c5000700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c0077890 0x7f06c01a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.294+0000 7f06c5000700 1 -- 172.21.15.112:0/2003356869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c019ffd0 con 0x7f06c0076f70 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.294+0000 7f06bfe00700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c0077890 0x7f06c01a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:45:59.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.295+0000 7f06c5000700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06c0076f70 0x7f06c01a55d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06b400f990 tx=0x7f06b400fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:59.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.295+0000 7f06bd600700 1 -- 172.21.15.112:0/2003356869 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4005400 con 0x7f06c0076f70 2024-04-25T18:45:59.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.295+0000 7f06bd600700 1 -- 172.21.15.112:0/2003356869 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b40055a0 con 0x7f06c0076f70 2024-04-25T18:45:59.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.295+0000 7f06c7a00700 1 -- 172.21.15.112:0/2003356869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c01a0260 con 0x7f06c0076f70 2024-04-25T18:45:59.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.295+0000 7f06bd600700 1 -- 172.21.15.112:0/2003356869 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4004dc0 con 0x7f06c0076f70 2024-04-25T18:45:59.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.295+0000 7f06c7a00700 1 -- 172.21.15.112:0/2003356869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c01ac550 con 0x7f06c0076f70 2024-04-25T18:45:59.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.296+0000 7f06c7a00700 1 -- 172.21.15.112:0/2003356869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c01a05c0 con 0x7f06c0076f70 2024-04-25T18:45:59.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.298+0000 7f06bd600700 1 -- 172.21.15.112:0/2003356869 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f06b400b560 con 0x7f06c0076f70 2024-04-25T18:45:59.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.299+0000 7f06bd600700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f06a80a3cc0 0x7f06a80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:45:59.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.300+0000 7f06bd600700 1 -- 172.21.15.112:0/2003356869 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f06b40d1300 con 0x7f06c0076f70 2024-04-25T18:45:59.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.300+0000 7f06bfe00700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f06a80a3cc0 0x7f06a80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:45:59.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.300+0000 7f06bfe00700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f06a80a3cc0 0x7f06a80a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f06c01a0e60 tx=0x7f06ac009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:45:59.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.305+0000 7f06bd600700 1 -- 172.21.15.112:0/2003356869 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06b4013030 con 0x7f06c0076f70 2024-04-25T18:45:59.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:45:59 smithi112 ceph-mon[27864]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:59.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.579+0000 7f06c7a00700 1 -- 172.21.15.112:0/2003356869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06c01a0bd0 con 0x7f06c0076f70 2024-04-25T18:45:59.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.580+0000 7f06bd600700 1 -- 172.21.15.112:0/2003356869 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7f06b409dcf0 con 0x7f06c0076f70 2024-04-25T18:45:59.583 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:45:59.583 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:43:45.212327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24511,24512,24493]},"id":1}]} 2024-04-25T18:45:59.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 -- 172.21.15.112:0/2003356869 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f06a80a3cc0 msgr2=0x7f06a80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:59.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f06a80a3cc0 0x7f06a80a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f06c01a0e60 tx=0x7f06ac009310 comp rx=0 tx=0).stop 2024-04-25T18:45:59.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 -- 172.21.15.112:0/2003356869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06c0076f70 msgr2=0x7f06c01a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:45:59.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06c0076f70 0x7f06c01a55d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06b400f990 tx=0x7f06b400fa70 comp rx=0 tx=0).stop 2024-04-25T18:45:59.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 -- 172.21.15.112:0/2003356869 shutdown_connections 2024-04-25T18:45:59.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f06a80a3cc0 0x7f06a80a6180 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:59.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06c0113920 0x7f06c019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:59.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06c0076f70 0x7f06c01a55d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:59.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 --2- 172.21.15.112:0/2003356869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06c0077890 0x7f06c01a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:45:59.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 -- 172.21.15.112:0/2003356869 >> 172.21.15.112:0/2003356869 conn(0x7f06c0100770 msgr2=0x7f06c0102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:45:59.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 -- 172.21.15.112:0/2003356869 shutdown_connections 2024-04-25T18:45:59.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:45:59.583+0000 7f06c7a00700 1 -- 172.21.15.112:0/2003356869 wait complete. 2024-04-25T18:45:59.586 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-04-25T18:45:59.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:45:59 smithi171 ceph-mon[33266]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:45:59.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:45:59 smithi161 ceph-mon[32609]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:00 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2003356869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:00.615 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:46:00.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:00 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2003356869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:00.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:00 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2003356869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:01.042 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:01.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:01 smithi112 ceph-mon[27864]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:01.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:01 smithi171 ceph-mon[33266]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:01.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:01 smithi161 ceph-mon[32609]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:03.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.445+0000 7fee9fc00700 1 -- 172.21.15.112:0/936502583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee98109c80 msgr2=0x7fee9810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:03.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.445+0000 7fee9fc00700 1 --2- 172.21.15.112:0/936502583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee98109c80 0x7fee9810dcd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee840099c0 tx=0x7fee84009cd0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.445+0000 7fee9fc00700 1 -- 172.21.15.112:0/936502583 shutdown_connections 2024-04-25T18:46:03.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.445+0000 7fee9fc00700 1 --2- 172.21.15.112:0/936502583 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee9810e3b0 0x7fee98115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.445+0000 7fee9fc00700 1 --2- 172.21.15.112:0/936502583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee981092d0 0x7fee981096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.445+0000 7fee9fc00700 1 --2- 172.21.15.112:0/936502583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee98109c80 0x7fee9810dcd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.445+0000 7fee9fc00700 1 -- 172.21.15.112:0/936502583 >> 172.21.15.112:0/936502583 conn(0x7fee98076210 msgr2=0x7fee98107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:03.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.445+0000 7fee9fc00700 1 -- 172.21.15.112:0/936502583 shutdown_connections 2024-04-25T18:46:03.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.445+0000 7fee9fc00700 1 -- 172.21.15.112:0/936502583 wait complete. 2024-04-25T18:46:03.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.446+0000 7fee9fc00700 1 Processor -- start 2024-04-25T18:46:03.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.447+0000 7fee9fc00700 1 -- start start 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.447+0000 7fee9fc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee981092d0 0x7fee9819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.447+0000 7fee9fc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee98109c80 0x7fee9819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.447+0000 7fee9fc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee9810e3b0 0x7fee981a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.447+0000 7fee9fc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee98119240 con 0x7fee98109c80 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.447+0000 7fee9fc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fee981190c0 con 0x7fee981092d0 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.447+0000 7fee9fc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fee981193c0 con 0x7fee9810e3b0 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.447+0000 7fee97e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee98109c80 0x7fee9819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.447+0000 7fee9d200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee981092d0 0x7fee9819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.447+0000 7fee9dc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee9810e3b0 0x7fee981a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.448+0000 7fee9d200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee981092d0 0x7fee9819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:60188/0 (socket says 172.21.15.112:60188) 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.448+0000 7fee9d200700 1 -- 172.21.15.112:0/924628969 learned_addr learned my addr 172.21.15.112:0/924628969 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.448+0000 7fee97e00700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee98109c80 0x7fee9819fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48416/0 (socket says 172.21.15.112:48416) 2024-04-25T18:46:03.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.448+0000 7fee9d200700 1 -- 172.21.15.112:0/924628969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee9810e3b0 msgr2=0x7fee981a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:03.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.448+0000 7fee9d200700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee9810e3b0 0x7fee981a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.448+0000 7fee9d200700 1 -- 172.21.15.112:0/924628969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee98109c80 msgr2=0x7fee9819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:03.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.448+0000 7fee9d200700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee98109c80 0x7fee9819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.448+0000 7fee9d200700 1 -- 172.21.15.112:0/924628969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee981a46e0 con 0x7fee981092d0 2024-04-25T18:46:03.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.448+0000 7fee9dc00700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee9810e3b0 0x7fee981a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:03.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.448+0000 7fee9d200700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee981092d0 0x7fee9819f6d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00e9a0 tx=0x7fee8c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:03.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.449+0000 7fee95600700 1 -- 172.21.15.112:0/924628969 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00ca10 con 0x7fee981092d0 2024-04-25T18:46:03.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.449+0000 7fee95600700 1 -- 172.21.15.112:0/924628969 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8c00cbb0 con 0x7fee981092d0 2024-04-25T18:46:03.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.449+0000 7fee95600700 1 -- 172.21.15.112:0/924628969 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c018680 con 0x7fee981092d0 2024-04-25T18:46:03.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.449+0000 7fee9fc00700 1 -- 172.21.15.112:0/924628969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee981a49d0 con 0x7fee981092d0 2024-04-25T18:46:03.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.449+0000 7fee9fc00700 1 -- 172.21.15.112:0/924628969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee981ac250 con 0x7fee981092d0 2024-04-25T18:46:03.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.450+0000 7fee9fc00700 1 -- 172.21.15.112:0/924628969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee98069730 con 0x7fee981092d0 2024-04-25T18:46:03.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.452+0000 7fee95600700 1 -- 172.21.15.112:0/924628969 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fee8c010430 con 0x7fee981092d0 2024-04-25T18:46:03.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.453+0000 7fee95600700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fee800a3cc0 0x7fee800a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:03.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.453+0000 7fee95600700 1 -- 172.21.15.112:0/924628969 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fee8c0d0620 con 0x7fee981092d0 2024-04-25T18:46:03.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.453+0000 7fee97e00700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fee800a3cc0 0x7fee800a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:03.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.454+0000 7fee97e00700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fee800a3cc0 0x7fee800a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fee84000c00 tx=0x7fee84005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:03.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.456+0000 7fee95600700 1 -- 172.21.15.112:0/924628969 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee8c09cf80 con 0x7fee981092d0 2024-04-25T18:46:03.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.733+0000 7fee9fc00700 1 -- 172.21.15.112:0/924628969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee9802df80 con 0x7fee981092d0 2024-04-25T18:46:03.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.734+0000 7fee95600700 1 -- 172.21.15.112:0/924628969 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7fee8c09c880 con 0x7fee981092d0 2024-04-25T18:46:03.736 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:46:03.736 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:43:45.212327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24511,24512,24493]},"id":1}]} 2024-04-25T18:46:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 -- 172.21.15.112:0/924628969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fee800a3cc0 msgr2=0x7fee800a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fee800a3cc0 0x7fee800a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fee84000c00 tx=0x7fee84005fb0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 -- 172.21.15.112:0/924628969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee981092d0 msgr2=0x7fee9819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee981092d0 0x7fee9819f6d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00e9a0 tx=0x7fee8c00ecb0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 -- 172.21.15.112:0/924628969 shutdown_connections 2024-04-25T18:46:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fee800a3cc0 0x7fee800a6180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee9810e3b0 0x7fee981a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee981092d0 0x7fee9819f6d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 --2- 172.21.15.112:0/924628969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee98109c80 0x7fee9819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 -- 172.21.15.112:0/924628969 >> 172.21.15.112:0/924628969 conn(0x7fee98076210 msgr2=0x7fee98076830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 -- 172.21.15.112:0/924628969 shutdown_connections 2024-04-25T18:46:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:03.737+0000 7fee9fc00700 1 -- 172.21.15.112:0/924628969 wait complete. 2024-04-25T18:46:03.740 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-04-25T18:46:04.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:04 smithi161 ceph-mon[32609]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:04.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:04 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/924628969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:04.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:04 smithi112 ceph-mon[27864]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:04.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:04 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/924628969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:04.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:04 smithi171 ceph-mon[33266]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:04.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:04 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/924628969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:04.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:46:05.138 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:06.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:06 smithi171 ceph-mon[33266]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:06.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:06 smithi161 ceph-mon[32609]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:06.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:06 smithi112 ceph-mon[27864]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:07.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.589+0000 7f3ece000700 1 -- 172.21.15.112:0/1678827400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8109fa0 msgr2=0x7f3ec8111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:07.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.589+0000 7f3ece000700 1 --2- 172.21.15.112:0/1678827400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8109fa0 0x7f3ec8111b70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000b0d0 tx=0x7f3ec000b3e0 comp rx=0 tx=0).stop 2024-04-25T18:46:07.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.589+0000 7f3ece000700 1 -- 172.21.15.112:0/1678827400 shutdown_connections 2024-04-25T18:46:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.589+0000 7f3ece000700 1 --2- 172.21.15.112:0/1678827400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8109fa0 0x7f3ec8111b70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.589+0000 7f3ece000700 1 --2- 172.21.15.112:0/1678827400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ec8104ec0 0x7f3ec81052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.589+0000 7f3ece000700 1 --2- 172.21.15.112:0/1678827400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8105870 0x7f3ec81098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.590+0000 7f3ece000700 1 -- 172.21.15.112:0/1678827400 >> 172.21.15.112:0/1678827400 conn(0x7f3ec8100750 msgr2=0x7f3ec8102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.590+0000 7f3ece000700 1 -- 172.21.15.112:0/1678827400 shutdown_connections 2024-04-25T18:46:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.590+0000 7f3ece000700 1 -- 172.21.15.112:0/1678827400 wait complete. 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.591+0000 7f3ece000700 1 Processor -- start 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.591+0000 7f3ece000700 1 -- start start 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.592+0000 7f3ece000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8104ec0 0x7f3ec819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.592+0000 7f3ece000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ec8105870 0x7f3ec819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.592+0000 7f3ece000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8109fa0 0x7f3ec819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.592+0000 7f3ece000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8114e90 con 0x7f3ec8109fa0 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.592+0000 7f3ece000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8114d10 con 0x7f3ec8104ec0 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.592+0000 7f3ece000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8115010 con 0x7f3ec8105870 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.592+0000 7f3ec7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8109fa0 0x7f3ec819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.592+0000 7f3ec6a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ec8105870 0x7f3ec819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.592+0000 7f3ec6a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ec8105870 0x7f3ec819b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:39930/0 (socket says 172.21.15.112:39930) 2024-04-25T18:46:07.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.592+0000 7f3ec6a00700 1 -- 172.21.15.112:0/2095169417 learned_addr learned my addr 172.21.15.112:0/2095169417 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:07.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.592+0000 7f3ec7400700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8104ec0 0x7f3ec819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:07.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.593+0000 7f3ec6a00700 1 -- 172.21.15.112:0/2095169417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8104ec0 msgr2=0x7f3ec819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:07.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.593+0000 7f3ec6a00700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8104ec0 0x7f3ec819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:07.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.593+0000 7f3ec6a00700 1 -- 172.21.15.112:0/2095169417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8109fa0 msgr2=0x7f3ec819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:07.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.593+0000 7f3ec6a00700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8109fa0 0x7f3ec819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:07.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.593+0000 7f3ec6a00700 1 -- 172.21.15.112:0/2095169417 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec81a02e0 con 0x7f3ec8105870 2024-04-25T18:46:07.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.593+0000 7f3ec7e00700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8109fa0 0x7f3ec819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:07.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.593+0000 7f3ec6a00700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ec8105870 0x7f3ec819b860 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400e970 tx=0x7f3eb400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:07.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.593+0000 7f3ebfe00700 1 -- 172.21.15.112:0/2095169417 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4004a30 con 0x7f3ec8105870 2024-04-25T18:46:07.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.593+0000 7f3ebfe00700 1 -- 172.21.15.112:0/2095169417 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb400cc90 con 0x7f3ec8105870 2024-04-25T18:46:07.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.593+0000 7f3ebfe00700 1 -- 172.21.15.112:0/2095169417 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4005610 con 0x7f3ec8105870 2024-04-25T18:46:07.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.593+0000 7f3ece000700 1 -- 172.21.15.112:0/2095169417 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec81a05d0 con 0x7f3ec8105870 2024-04-25T18:46:07.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.594+0000 7f3ece000700 1 -- 172.21.15.112:0/2095169417 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec81a7e50 con 0x7f3ec8105870 2024-04-25T18:46:07.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.595+0000 7f3ece000700 1 -- 172.21.15.112:0/2095169417 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec8069730 con 0x7f3ec8105870 2024-04-25T18:46:07.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.597+0000 7f3ebfe00700 1 -- 172.21.15.112:0/2095169417 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f3eb400c7e0 con 0x7f3ec8105870 2024-04-25T18:46:07.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.598+0000 7f3ebfe00700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3eac0a3cc0 0x7f3eac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:07.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.598+0000 7f3ebfe00700 1 -- 172.21.15.112:0/2095169417 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f3eb40d04f0 con 0x7f3ec8105870 2024-04-25T18:46:07.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.598+0000 7f3ec7400700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3eac0a3cc0 0x7f3eac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:07.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.599+0000 7f3ec7400700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3eac0a3cc0 0x7f3eac0a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009c60 tx=0x7f3eb8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:07.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.602+0000 7f3ebfe00700 1 -- 172.21.15.112:0/2095169417 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3eb409ce50 con 0x7f3ec8105870 2024-04-25T18:46:07.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.829+0000 7f3ece000700 1 -- 172.21.15.112:0/2095169417 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ec802df80 con 0x7f3ec8105870 2024-04-25T18:46:07.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.831+0000 7f3ebfe00700 1 -- 172.21.15.112:0/2095169417 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+22709 (secure 0 0 0) 0x7f3eb409c750 con 0x7f3ec8105870 2024-04-25T18:46:07.834 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:46:07.835 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","created":"2024-04-25T18:33:37.379089+0000","modified":"2024-04-25T18:43:56.620122+0000","last_up_change":"2024-04-25T18:42:00.961646+0000","last_in_change":"2024-04-25T18:41:40.647162+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:37:45.106199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:43:11.017975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:43:15.116917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efb4bc01-e29b-4402-8a7a-3387d1731ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6803","nonce":232254535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6805","nonce":232254535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6809","nonce":232254535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6807","nonce":232254535}]},"public_addr":"172.21.15.112:6803/232254535","cluster_addr":"172.21.15.112:6805/232254535","heartbeat_back_addr":"172.21.15.112:6809/232254535","heartbeat_front_addr":"172.21.15.112:6807/232254535","state":["exists","up"]},{"osd":1,"uuid":"65a83fa2-96a6-4743-8c17-21531f5e885e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6801","nonce":664224744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6803","nonce":664224744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6807","nonce":664224744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6805","nonce":664224744}]},"public_addr":"172.21.15.161:6801/664224744","cluster_addr":"172.21.15.161:6803/664224744","heartbeat_back_addr":"172.21.15.161:6807/664224744","heartbeat_front_addr":"172.21.15.161:6805/664224744","state":["exists","up"]},{"osd":2,"uuid":"7562ec4c-c126-48ec-a51c-56fa461a2e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6801","nonce":345172073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6803","nonce":345172073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6807","nonce":345172073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6805","nonce":345172073}]},"public_addr":"172.21.15.171:6801/345172073","cluster_addr":"172.21.15.171:6803/345172073","heartbeat_back_addr":"172.21.15.171:6807/345172073","heartbeat_front_addr":"172.21.15.171:6805/345172073","state":["exists","up"]},{"osd":3,"uuid":"d7aae8cd-fec7-46d9-8773-25eaaca3cc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6811","nonce":1472407156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6813","nonce":1472407156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6817","nonce":1472407156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6815","nonce":1472407156}]},"public_addr":"172.21.15.112:6811/1472407156","cluster_addr":"172.21.15.112:6813/1472407156","heartbeat_back_addr":"172.21.15.112:6817/1472407156","heartbeat_front_addr":"172.21.15.112:6815/1472407156","state":["exists","up"]},{"osd":4,"uuid":"b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6809","nonce":373204306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6811","nonce":373204306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6815","nonce":373204306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6813","nonce":373204306}]},"public_addr":"172.21.15.161:6809/373204306","cluster_addr":"172.21.15.161:6811/373204306","heartbeat_back_addr":"172.21.15.161:6815/373204306","heartbeat_front_addr":"172.21.15.161:6813/373204306","state":["exists","up"]},{"osd":5,"uuid":"56124d12-bc1f-4cba-8430-938248851776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6809","nonce":2643512313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6811","nonce":2643512313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6815","nonce":2643512313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6813","nonce":2643512313}]},"public_addr":"172.21.15.171:6809/2643512313","cluster_addr":"172.21.15.171:6811/2643512313","heartbeat_back_addr":"172.21.15.171:6815/2643512313","heartbeat_front_addr":"172.21.15.171:6813/2643512313","state":["exists","up"]},{"osd":6,"uuid":"9563a9d3-352b-4686-975c-282ec167248b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6819","nonce":1933654933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6821","nonce":1933654933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6825","nonce":1933654933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6823","nonce":1933654933}]},"public_addr":"172.21.15.112:6819/1933654933","cluster_addr":"172.21.15.112:6821/1933654933","heartbeat_back_addr":"172.21.15.112:6825/1933654933","heartbeat_front_addr":"172.21.15.112:6823/1933654933","state":["exists","up"]},{"osd":7,"uuid":"f2d06e86-6aac-40f4-956e-18fe0b0e3543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6817","nonce":3060201138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6819","nonce":3060201138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6823","nonce":3060201138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6821","nonce":3060201138}]},"public_addr":"172.21.15.161:6817/3060201138","cluster_addr":"172.21.15.161:6819/3060201138","heartbeat_back_addr":"172.21.15.161:6823/3060201138","heartbeat_front_addr":"172.21.15.161:6821/3060201138","state":["exists","up"]},{"osd":8,"uuid":"f006e45f-2bb9-45d0-9020-be7c33c6a825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6817","nonce":835756571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6819","nonce":835756571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6823","nonce":835756571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6821","nonce":835756571}]},"public_addr":"172.21.15.171:6817/835756571","cluster_addr":"172.21.15.171:6819/835756571","heartbeat_back_addr":"172.21.15.171:6823/835756571","heartbeat_front_addr":"172.21.15.171:6821/835756571","state":["exists","up"]},{"osd":9,"uuid":"c5a96491-4151-4633-b3bc-2c214fe51c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6827","nonce":349758528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6829","nonce":349758528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6833","nonce":349758528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6831","nonce":349758528}]},"public_addr":"172.21.15.112:6827/349758528","cluster_addr":"172.21.15.112:6829/349758528","heartbeat_back_addr":"172.21.15.112:6833/349758528","heartbeat_front_addr":"172.21.15.112:6831/349758528","state":["exists","up"]},{"osd":10,"uuid":"244eda3e-25a0-490e-879f-c1e2726c1be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6825","nonce":3013292765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6827","nonce":3013292765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6831","nonce":3013292765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6829","nonce":3013292765}]},"public_addr":"172.21.15.161:6825/3013292765","cluster_addr":"172.21.15.161:6827/3013292765","heartbeat_back_addr":"172.21.15.161:6831/3013292765","heartbeat_front_addr":"172.21.15.161:6829/3013292765","state":["exists","up"]},{"osd":11,"uuid":"356e5c54-bbc3-45ee-bda2-85164ce3cb92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6825","nonce":1010336458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6827","nonce":1010336458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6831","nonce":1010336458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6829","nonce":1010336458}]},"public_addr":"172.21.15.171:6825/1010336458","cluster_addr":"172.21.15.171:6827/1010336458","heartbeat_back_addr":"172.21.15.171:6831/1010336458","heartbeat_front_addr":"172.21.15.171:6829/1010336458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:36:51.024650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:17.026638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:41.239726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:15.148968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:41.029118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:06.225858+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:41.639444+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:07.753400+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:32.292085+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:07.159926+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:33.749782+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:58.594447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/181862246":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/3393454453":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3195850527":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6801/3468460974":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3392048544":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/1329430933":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/735578423":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/3468460974":"2024-04-26T18:34:20.605775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:46:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.834+0000 7f3ece000700 1 -- 172.21.15.112:0/2095169417 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3eac0a3cc0 msgr2=0x7f3eac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.834+0000 7f3ece000700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3eac0a3cc0 0x7f3eac0a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009c60 tx=0x7f3eb8009260 comp rx=0 tx=0).stop 2024-04-25T18:46:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.834+0000 7f3ece000700 1 -- 172.21.15.112:0/2095169417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ec8105870 msgr2=0x7f3ec819b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.834+0000 7f3ece000700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ec8105870 0x7f3ec819b860 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400e970 tx=0x7f3eb400ec80 comp rx=0 tx=0).stop 2024-04-25T18:46:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.834+0000 7f3ece000700 1 -- 172.21.15.112:0/2095169417 shutdown_connections 2024-04-25T18:46:07.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.834+0000 7f3ece000700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f3eac0a3cc0 0x7f3eac0a6180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:07.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.834+0000 7f3ece000700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec8109fa0 0x7f3ec819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:07.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.834+0000 7f3ece000700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ec8104ec0 0x7f3ec819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:07.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.834+0000 7f3ece000700 1 --2- 172.21.15.112:0/2095169417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ec8105870 0x7f3ec819b860 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:07.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.834+0000 7f3ece000700 1 -- 172.21.15.112:0/2095169417 >> 172.21.15.112:0/2095169417 conn(0x7f3ec8100750 msgr2=0x7f3ec81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:07.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.835+0000 7f3ece000700 1 -- 172.21.15.112:0/2095169417 shutdown_connections 2024-04-25T18:46:07.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:07.835+0000 7f3ece000700 1 -- 172.21.15.112:0/2095169417 wait complete. 2024-04-25T18:46:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:08 smithi171 ceph-mon[33266]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:08 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2095169417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:46:08.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:08 smithi161 ceph-mon[32609]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:08.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:08 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2095169417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:46:08.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:08 smithi112 ceph-mon[27864]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:08.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:08 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2095169417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:46:08.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:46:09.264 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:10.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:10 smithi171 ceph-mon[33266]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:10.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:10 smithi161 ceph-mon[32609]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:10.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:10 smithi112 ceph-mon[27864]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:11.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.765+0000 7fa3ec600700 1 -- 172.21.15.112:0/1818356265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e406b980 msgr2=0x7fa3e410f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:11.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.765+0000 7fa3ec600700 1 --2- 172.21.15.112:0/1818356265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e406b980 0x7fa3e410f8f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00b0a0 tx=0x7fa3dc00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:46:11.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.765+0000 7fa3ec600700 1 -- 172.21.15.112:0/1818356265 shutdown_connections 2024-04-25T18:46:11.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.765+0000 7fa3ec600700 1 --2- 172.21.15.112:0/1818356265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e406b980 0x7fa3e410f8f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:11.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.766+0000 7fa3ec600700 1 --2- 172.21.15.112:0/1818356265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e4108a20 0x7fa3e4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:11.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.766+0000 7fa3ec600700 1 --2- 172.21.15.112:0/1818356265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa3e406afe0 0x7fa3e406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:11.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.766+0000 7fa3ec600700 1 -- 172.21.15.112:0/1818356265 >> 172.21.15.112:0/1818356265 conn(0x7fa3e40788b0 msgr2=0x7fa3e4078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:11.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.766+0000 7fa3ec600700 1 -- 172.21.15.112:0/1818356265 shutdown_connections 2024-04-25T18:46:11.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.766+0000 7fa3ec600700 1 -- 172.21.15.112:0/1818356265 wait complete. 2024-04-25T18:46:11.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.767+0000 7fa3ec600700 1 Processor -- start 2024-04-25T18:46:11.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.767+0000 7fa3ec600700 1 -- start start 2024-04-25T18:46:11.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.767+0000 7fa3ec600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa3e406afe0 0x7fa3e4101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:11.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.767+0000 7fa3ec600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e406b980 0x7fa3e4101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:11.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.767+0000 7fa3ec600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e4108a20 0x7fa3e4102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:11.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3ec600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e4112c00 con 0x7fa3e4108a20 2024-04-25T18:46:11.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3ec600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e4112a80 con 0x7fa3e406b980 2024-04-25T18:46:11.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3ec600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e4112d80 con 0x7fa3e406afe0 2024-04-25T18:46:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3e9c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa3e406afe0 0x7fa3e4101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3ea600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e4108a20 0x7fa3e4102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3e9c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa3e406afe0 0x7fa3e4101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:34472/0 (socket says 172.21.15.112:34472) 2024-04-25T18:46:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3e9c00700 1 -- 172.21.15.112:0/2606757944 learned_addr learned my addr 172.21.15.112:0/2606757944 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3e9200700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e406b980 0x7fa3e4101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3e9c00700 1 -- 172.21.15.112:0/2606757944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e406b980 msgr2=0x7fa3e4101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3e9c00700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e406b980 0x7fa3e4101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3e9c00700 1 -- 172.21.15.112:0/2606757944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e4108a20 msgr2=0x7fa3e4102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3e9c00700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e4108a20 0x7fa3e4102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3e9c00700 1 -- 172.21.15.112:0/2606757944 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e41a7600 con 0x7fa3e406afe0 2024-04-25T18:46:11.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.768+0000 7fa3ea600700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e4108a20 0x7fa3e4102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:11.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.769+0000 7fa3e9c00700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa3e406afe0 0x7fa3e4101a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800eb30 tx=0x7fa3d800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:11.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.769+0000 7fa3e2000700 1 -- 172.21.15.112:0/2606757944 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d800cbd0 con 0x7fa3e406afe0 2024-04-25T18:46:11.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.769+0000 7fa3e2000700 1 -- 172.21.15.112:0/2606757944 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d80044d0 con 0x7fa3e406afe0 2024-04-25T18:46:11.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.769+0000 7fa3ec600700 1 -- 172.21.15.112:0/2606757944 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e41a78f0 con 0x7fa3e406afe0 2024-04-25T18:46:11.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.769+0000 7fa3e2000700 1 -- 172.21.15.112:0/2606757944 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8003e30 con 0x7fa3e406afe0 2024-04-25T18:46:11.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.769+0000 7fa3ec600700 1 -- 172.21.15.112:0/2606757944 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e41a7e30 con 0x7fa3e406afe0 2024-04-25T18:46:11.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.770+0000 7fa3ec600700 1 -- 172.21.15.112:0/2606757944 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e4069730 con 0x7fa3e406afe0 2024-04-25T18:46:11.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.772+0000 7fa3e2000700 1 -- 172.21.15.112:0/2606757944 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fa3d8004000 con 0x7fa3e406afe0 2024-04-25T18:46:11.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.773+0000 7fa3e2000700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa3cc0a3c40 0x7fa3cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:11.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.774+0000 7fa3e2000700 1 -- 172.21.15.112:0/2606757944 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa3d80d0960 con 0x7fa3e406afe0 2024-04-25T18:46:11.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.774+0000 7fa3e9200700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa3cc0a3c40 0x7fa3cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:11.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.774+0000 7fa3e9200700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa3cc0a3c40 0x7fa3cc0a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4078e20 tx=0x7fa3d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:11.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:11.776+0000 7fa3e2000700 1 -- 172.21.15.112:0/2606757944 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3d8053ff0 con 0x7fa3e406afe0 2024-04-25T18:46:12.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.050+0000 7fa3ec600700 1 -- 172.21.15.112:0/2606757944 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3e402df80 con 0x7fa3e406afe0 2024-04-25T18:46:12.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.051+0000 7fa3e2000700 1 -- 172.21.15.112:0/2606757944 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7fa3d80042f0 con 0x7fa3e406afe0 2024-04-25T18:46:12.054 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:46:12.054 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:43:45.212327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24511,24512,24493]},"id":1}]} 2024-04-25T18:46:12.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.054+0000 7fa3ec600700 1 -- 172.21.15.112:0/2606757944 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa3cc0a3c40 msgr2=0x7fa3cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:12.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.054+0000 7fa3ec600700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa3cc0a3c40 0x7fa3cc0a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4078e20 tx=0x7fa3d0009260 comp rx=0 tx=0).stop 2024-04-25T18:46:12.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.054+0000 7fa3ec600700 1 -- 172.21.15.112:0/2606757944 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa3e406afe0 msgr2=0x7fa3e4101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:12.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.054+0000 7fa3ec600700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa3e406afe0 0x7fa3e4101a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800eb30 tx=0x7fa3d800ee40 comp rx=0 tx=0).stop 2024-04-25T18:46:12.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.054+0000 7fa3ec600700 1 -- 172.21.15.112:0/2606757944 shutdown_connections 2024-04-25T18:46:12.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.054+0000 7fa3ec600700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa3cc0a3c40 0x7fa3cc0a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:12.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.054+0000 7fa3ec600700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e4108a20 0x7fa3e4102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:12.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.054+0000 7fa3ec600700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa3e406afe0 0x7fa3e4101a00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:12.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.054+0000 7fa3ec600700 1 --2- 172.21.15.112:0/2606757944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3e406b980 0x7fa3e4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:12.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.055+0000 7fa3ec600700 1 -- 172.21.15.112:0/2606757944 >> 172.21.15.112:0/2606757944 conn(0x7fa3e40788b0 msgr2=0x7fa3e40ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:12.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.055+0000 7fa3ec600700 1 -- 172.21.15.112:0/2606757944 shutdown_connections 2024-04-25T18:46:12.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:12.055+0000 7fa3ec600700 1 -- 172.21.15.112:0/2606757944 wait complete. 2024-04-25T18:46:12.058 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-04-25T18:46:12.426 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:12 smithi112 ceph-mon[27864]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:12.426 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:12 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2606757944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:12.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:12 smithi171 ceph-mon[33266]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:12.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:12 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2606757944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:12.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:12 smithi161 ceph-mon[32609]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:12.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:12 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2606757944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:13.031 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T18:43:19.347467+0000', 'modified': '2024-04-25T18:43:45.212327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24511, 'mds_2': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/4007863322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 4007863322}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 4007863322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/3522712423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 3522712423}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 3522712423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'g', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6839/3241331981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 3241331981}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 3241331981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.171:6839/1642228003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 1642228003}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 1642228003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.112:6841/1028859850', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6840', 'nonce': 1028859850}, {'type': 'v1', 'addr': '172.21.15.112:6841', 'nonce': 1028859850}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.161:6839/2315334000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2315334000}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2315334000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24511, 24512, 24493]} 2024-04-25T18:46:13.031 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T18:46:13.032 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T18:46:13.477 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:14.500 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:14 smithi112 ceph-mon[27864]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:14.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:14 smithi171 ceph-mon[33266]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:14.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:14 smithi161 ceph-mon[32609]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:15.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.685+0000 7f214cc00700 1 -- 172.21.15.112:0/4266385206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f214810f440 msgr2=0x7f2148111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:15.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.685+0000 7f214cc00700 1 --2- 172.21.15.112:0/4266385206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f214810f440 0x7f2148111830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2140009f90 tx=0x7f214000b330 comp rx=0 tx=0).stop 2024-04-25T18:46:15.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.686+0000 7f214cc00700 1 -- 172.21.15.112:0/4266385206 shutdown_connections 2024-04-25T18:46:15.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.686+0000 7f214cc00700 1 --2- 172.21.15.112:0/4266385206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f214810f440 0x7f2148111830 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.686+0000 7f214cc00700 1 --2- 172.21.15.112:0/4266385206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2148074c70 0x7f2148075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.686+0000 7f214cc00700 1 --2- 172.21.15.112:0/4266385206 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2148075590 0x7f214810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.686+0000 7f214cc00700 1 -- 172.21.15.112:0/4266385206 >> 172.21.15.112:0/4266385206 conn(0x7f2148079ba0 msgr2=0x7f2148079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:15.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.686+0000 7f214cc00700 1 -- 172.21.15.112:0/4266385206 shutdown_connections 2024-04-25T18:46:15.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.686+0000 7f214cc00700 1 -- 172.21.15.112:0/4266385206 wait complete. 2024-04-25T18:46:15.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.687+0000 7f214cc00700 1 Processor -- start 2024-04-25T18:46:15.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.688+0000 7f214cc00700 1 -- start start 2024-04-25T18:46:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.688+0000 7f214cc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2148074c70 0x7f214819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.688+0000 7f214cc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2148075590 0x7f214819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.688+0000 7f214cc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f214810f440 0x7f214819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.688+0000 7f214cc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2148114e50 con 0x7f2148075590 2024-04-25T18:46:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.688+0000 7f214cc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2148114cd0 con 0x7f2148074c70 2024-04-25T18:46:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.688+0000 7f214cc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2148114fd0 con 0x7f214810f440 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.689+0000 7f2145e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2148074c70 0x7f214819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.689+0000 7f2145e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2148074c70 0x7f214819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:38608/0 (socket says 172.21.15.112:38608) 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.689+0000 7f2145e00700 1 -- 172.21.15.112:0/885821305 learned_addr learned my addr 172.21.15.112:0/885821305 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.689+0000 7f2146800700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f214810f440 0x7f214819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.689+0000 7f2145400700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2148075590 0x7f214819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.689+0000 7f2145e00700 1 -- 172.21.15.112:0/885821305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f214810f440 msgr2=0x7f214819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.689+0000 7f2145e00700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f214810f440 0x7f214819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.689+0000 7f2145e00700 1 -- 172.21.15.112:0/885821305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2148075590 msgr2=0x7f214819b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.689+0000 7f2145e00700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2148075590 0x7f214819b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.689+0000 7f2145e00700 1 -- 172.21.15.112:0/885821305 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21481a02e0 con 0x7f2148074c70 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.689+0000 7f2146800700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f214810f440 0x7f214819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.690+0000 7f2145e00700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2148074c70 0x7f214819b320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f213800ec10 tx=0x7f213800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.690+0000 7f213ea00700 1 -- 172.21.15.112:0/885821305 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213800cc90 con 0x7f2148074c70 2024-04-25T18:46:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.690+0000 7f213ea00700 1 -- 172.21.15.112:0/885821305 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f213800ce30 con 0x7f2148074c70 2024-04-25T18:46:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.690+0000 7f214cc00700 1 -- 172.21.15.112:0/885821305 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21481a05d0 con 0x7f2148074c70 2024-04-25T18:46:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.690+0000 7f213ea00700 1 -- 172.21.15.112:0/885821305 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138010430 con 0x7f2148074c70 2024-04-25T18:46:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.690+0000 7f214cc00700 1 -- 172.21.15.112:0/885821305 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21481a7e50 con 0x7f2148074c70 2024-04-25T18:46:15.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.693+0000 7f213ea00700 1 -- 172.21.15.112:0/885821305 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f2138004000 con 0x7f2148074c70 2024-04-25T18:46:15.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.694+0000 7f215750ab80 1 Processor -- start 2024-04-25T18:46:15.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f215750ab80 1 -- start start 2024-04-25T18:46:15.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f213ea00700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f212c0a3c40 0x7f212c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f213ea00700 1 -- 172.21.15.112:0/885821305 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f21380d0700 con 0x7f2148074c70 2024-04-25T18:46:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f215750ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f215750ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d60a0 0x55c5814cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f215750ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 0x55c5814d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f2146800700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f212c0a3c40 0x7f212c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f215750ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c581421e40 con 0x55c5814cfef0 2024-04-25T18:46:15.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f215750ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c5813c86e0 con 0x55c5814d5780 2024-04-25T18:46:15.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f215750ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c58141de00 con 0x55c5814d60a0 2024-04-25T18:46:15.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f2145400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d60a0 0x55c5814cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f2145400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d60a0 0x55c5814cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:34506/0 (socket says 172.21.15.112:34506) 2024-04-25T18:46:15.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f2145400700 1 -- 172.21.15.112:0/672063713 learned_addr learned my addr 172.21.15.112:0/672063713 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:15.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f2145e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.695+0000 7f2146800700 1 --2- 172.21.15.112:0/672063713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 0x55c5814d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.696+0000 7f2145e00700 1 -- 172.21.15.112:0/672063713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d60a0 msgr2=0x55c5814cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:15.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.696+0000 7f2145e00700 1 --2- 172.21.15.112:0/672063713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d60a0 0x55c5814cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.696+0000 7f2145e00700 1 -- 172.21.15.112:0/672063713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 msgr2=0x55c5814d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.696+0000 7f2145e00700 1 --2- 172.21.15.112:0/672063713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 0x55c5814d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.696+0000 7f2145e00700 1 -- 172.21.15.112:0/672063713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5813be630 con 0x55c5814d5780 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.696+0000 7f2145400700 1 --2- 172.21.15.112:0/672063713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d60a0 0x55c5814cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.696+0000 7f2145e00700 1 --2- 172.21.15.112:0/672063713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814d5b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f213809c8f0 tx=0x7f213809c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708018409171bfc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.696+0000 7f2146800700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f212c0a3c40 0x7f212c0a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2140004fd0 tx=0x7f2140004ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.696+0000 7f2133400700 1 -- 172.21.15.112:0/672063713 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213809c960 con 0x55c5814d5780 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.697+0000 7f2133400700 1 -- 172.21.15.112:0/672063713 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21380a0d10 con 0x55c5814d5780 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.697+0000 7f2133400700 1 -- 172.21.15.112:0/672063713 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21380042f0 con 0x55c5814d5780 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.697+0000 7f215750ab80 1 -- 172.21.15.112:0/672063713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 msgr2=0x55c5814d5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.697+0000 7f215750ab80 1 --2- 172.21.15.112:0/672063713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814d5b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f213809c8f0 tx=0x7f213809c4a0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.698+0000 7f215750ab80 1 -- 172.21.15.112:0/672063713 shutdown_connections 2024-04-25T18:46:15.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.698+0000 7f215750ab80 1 --2- 172.21.15.112:0/672063713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 0x55c5814d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.698+0000 7f215750ab80 1 --2- 172.21.15.112:0/672063713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814d5b60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.698+0000 7f215750ab80 1 --2- 172.21.15.112:0/672063713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d60a0 0x55c5814cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.698+0000 7f215750ab80 1 -- 172.21.15.112:0/672063713 >> 172.21.15.112:0/672063713 conn(0x55c5813fdf00 msgr2=0x55c5814ca8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.698+0000 7f215750ab80 1 -- 172.21.15.112:0/672063713 shutdown_connections 2024-04-25T18:46:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.698+0000 7f215750ab80 1 -- 172.21.15.112:0/672063713 wait complete. 2024-04-25T18:46:15.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.698+0000 7f215750ab80 1 -- start start 2024-04-25T18:46:15.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f215750ab80 1 Processor -- start 2024-04-25T18:46:15.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f215750ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814cfef0 0x55c5814dbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f215750ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814d8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f215750ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814d99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f215750ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5814dc920 con 0x55c5814d9530 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f215750ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c5813db810 con 0x55c5814d5780 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f215750ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c5814dcaa0 con 0x55c5814cfef0 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f215750ab80 10 client.0 _ll_register_callbacks cb 0x55c5813ba6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f2146800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814d99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f2145e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814cfef0 0x55c5814dbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f2145e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814cfef0 0x55c5814dbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:34522/0 (socket says 172.21.15.112:34522) 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f2145e00700 1 -- 172.21.15.112:0/1902122261 learned_addr learned my addr 172.21.15.112:0/1902122261 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f2146800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814d99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54600/0 (socket says 172.21.15.112:54600) 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.699+0000 7f2145400700 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814d8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.700+0000 7f2146800700 1 -- 172.21.15.112:0/1902122261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814cfef0 msgr2=0x55c5814dbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.700+0000 7f2146800700 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814cfef0 0x55c5814dbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.700+0000 7f2146800700 1 -- 172.21.15.112:0/1902122261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 msgr2=0x55c5814d8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.700+0000 7f2146800700 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814d8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.700+0000 7f2146800700 1 -- 172.21.15.112:0/1902122261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c58136f880 con 0x55c5814d9530 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.700+0000 7f2145400700 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814d8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.700+0000 7f2146800700 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814d99b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2140009790 tx=0x7f214000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69b2d02d0e6e82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.700+0000 7f2127e00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.701+0000 7f2127e00700 1 -- 172.21.15.112:0/1902122261 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140021670 con 0x55c5814d9530 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.701+0000 7f2127e00700 1 -- 172.21.15.112:0/1902122261 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f2140021810 con 0x55c5814d9530 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.701+0000 7f215750ab80 1 -- 172.21.15.112:0/1902122261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21380a0d10 con 0x55c5814d9530 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.701+0000 7f2127e00700 1 -- 172.21.15.112:0/1902122261 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2140021a10 con 0x55c5814d9530 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.701+0000 7f215750ab80 10 client.14751 fetch_fsmap learned FSMap version 25 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.701+0000 7f215750ab80 1 -- 172.21.15.112:0/1902122261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c5813b8aa0 con 0x55c5814d9530 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.701+0000 7f2127e00700 1 -- 172.21.15.112:0/1902122261 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21380a0d10 con 0x55c5814d9530 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.701+0000 7f2127e00700 1 -- 172.21.15.112:0/1902122261 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2140011070 con 0x55c5814d9530 2024-04-25T18:46:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.702+0000 7f215750ab80 10 client.14751 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:46:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.702+0000 7f215750ab80 10 client.14751 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:46:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.702+0000 7f215750ab80 20 client.14751 populate_metadata read hostname 'smithi112' 2024-04-25T18:46:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.702+0000 7f215750ab80 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x55c5814e9b10 0x55c5814eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.702+0000 7f215750ab80 4 client.14751 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:46:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.702+0000 7f215750ab80 1 -- 172.21.15.112:0/1902122261 --> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c58142f630 con 0x55c5814e9b10 2024-04-25T18:46:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.702+0000 7f2145e00700 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x55c5814e9b10 0x55c5814eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.703+0000 7f2145e00700 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x55c5814e9b10 0x55c5814eaf80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e11affd962bdb8c3 server_cookie=2d284821aa8eafcd in_seq=0 out_seq=0 2024-04-25T18:46:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.703+0000 7f2127e00700 10 client.14751 ms_handle_connect on v2:172.21.15.171:6838/1642228003 2024-04-25T18:46:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.703+0000 7f2127e00700 1 -- 172.21.15.112:0/1902122261 <== mds.2 v2:172.21.15.171:6838/1642228003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c58142f630 con 0x55c5814e9b10 2024-04-25T18:46:15.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f2127e00700 10 client.14751 handle_command_reply: tid=0 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 1 client.14751 shutdown 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 1 -- 172.21.15.112:0/1902122261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 msgr2=0x55c5814d99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814d99b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2140009790 tx=0x7f214000b330 comp rx=0 tx=0).stop 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 1 -- 172.21.15.112:0/1902122261 shutdown_connections 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x55c5814e9b10 0x55c5814eaf80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814d99b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f2127e00700 0 client.14751 ms_handle_reset on v2:172.21.15.171:6838/1642228003 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814cfef0 0x55c5814dbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 1 --2- 172.21.15.112:0/1902122261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814d8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 1 -- 172.21.15.112:0/1902122261 >> 172.21.15.112:0/1902122261 conn(0x55c5813fdf00 msgr2=0x55c5814c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 1 -- 172.21.15.112:0/1902122261 shutdown_connections 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 1 -- 172.21.15.112:0/1902122261 wait complete. 2024-04-25T18:46:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.704+0000 7f215750ab80 20 client.14751 trim_cache size 0 max 16384 2024-04-25T18:46:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.727+0000 7f215750ab80 1 Processor -- start 2024-04-25T18:46:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.727+0000 7f215750ab80 1 -- start start 2024-04-25T18:46:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.727+0000 7f215750ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 0x55c5814dbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.727+0000 7f215750ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d5780 0x55c5814ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.727+0000 7f215750ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d9530 0x55c5814f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.727+0000 7f215750ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5814e92c0 con 0x55c5814cfef0 2024-04-25T18:46:15.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.727+0000 7f215750ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c5814e9990 con 0x55c5814d9530 2024-04-25T18:46:15.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.727+0000 7f215750ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c5814ea060 con 0x55c5814d5780 2024-04-25T18:46:15.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2145e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 0x55c5814dbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2146800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d9530 0x55c5814f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2145400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d5780 0x55c5814ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2146800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d9530 0x55c5814f43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:38634/0 (socket says 172.21.15.112:38634) 2024-04-25T18:46:15.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2145e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 0x55c5814dbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54604/0 (socket says 172.21.15.112:54604) 2024-04-25T18:46:15.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2146800700 1 -- 172.21.15.112:0/689765013 learned_addr learned my addr 172.21.15.112:0/689765013 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2146800700 1 -- 172.21.15.112:0/689765013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d5780 msgr2=0x55c5814ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2146800700 1 --2- 172.21.15.112:0/689765013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d5780 0x55c5814ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2146800700 1 -- 172.21.15.112:0/689765013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 msgr2=0x55c5814dbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2146800700 1 --2- 172.21.15.112:0/689765013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 0x55c5814dbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2146800700 1 -- 172.21.15.112:0/689765013 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5814ebee0 con 0x55c5814d9530 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2145e00700 1 --2- 172.21.15.112:0/689765013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 0x55c5814dbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2146800700 1 --2- 172.21.15.112:0/689765013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d9530 0x55c5814f43e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2140004fa0 tx=0x7f2140006030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c971c9a67517dcc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.728+0000 7f2132a00700 1 -- 172.21.15.112:0/689765013 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140021670 con 0x55c5814d9530 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.729+0000 7f2132a00700 1 -- 172.21.15.112:0/689765013 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2140031070 con 0x55c5814d9530 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.729+0000 7f2132a00700 1 -- 172.21.15.112:0/689765013 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214001e430 con 0x55c5814d9530 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.729+0000 7f215750ab80 1 -- 172.21.15.112:0/689765013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d9530 msgr2=0x55c5814f43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.729+0000 7f215750ab80 1 --2- 172.21.15.112:0/689765013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d9530 0x55c5814f43e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2140004fa0 tx=0x7f2140006030 comp rx=0 tx=0).stop 2024-04-25T18:46:15.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.729+0000 7f215750ab80 1 -- 172.21.15.112:0/689765013 shutdown_connections 2024-04-25T18:46:15.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.729+0000 7f215750ab80 1 --2- 172.21.15.112:0/689765013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d9530 0x55c5814f43e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.729+0000 7f215750ab80 1 --2- 172.21.15.112:0/689765013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814cfef0 0x55c5814dbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.729+0000 7f215750ab80 1 --2- 172.21.15.112:0/689765013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814d5780 0x55c5814ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.729+0000 7f215750ab80 1 -- 172.21.15.112:0/689765013 >> 172.21.15.112:0/689765013 conn(0x55c5813fdf00 msgr2=0x55c5814d78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:15.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.729+0000 7f215750ab80 1 -- 172.21.15.112:0/689765013 shutdown_connections 2024-04-25T18:46:15.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.730+0000 7f215750ab80 1 -- 172.21.15.112:0/689765013 wait complete. 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.730+0000 7f215750ab80 1 -- start start 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.730+0000 7f215750ab80 1 Processor -- start 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f215750ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814cfef0 0x55c5814f4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f215750ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f215750ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f215750ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5814eae00 con 0x55c5814d9530 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f215750ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c5814ea730 con 0x55c5814d5780 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f215750ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c5814eb4d0 con 0x55c5814cfef0 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f215750ab80 10 client.0 _ll_register_callbacks cb 0x55c5813ba6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f2146800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f2146800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814ef8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54616/0 (socket says 172.21.15.112:54616) 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f2146800700 1 -- 172.21.15.112:0/2953897346 learned_addr learned my addr 172.21.15.112:0/2953897346 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f2145e00700 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814cfef0 0x55c5814f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f2145400700 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f2146800700 1 -- 172.21.15.112:0/2953897346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814cfef0 msgr2=0x55c5814f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f2146800700 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814cfef0 0x55c5814f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f2146800700 1 -- 172.21.15.112:0/2953897346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 msgr2=0x55c5814ef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f2146800700 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814ef3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.731+0000 7f2146800700 1 -- 172.21.15.112:0/2953897346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5814effe0 con 0x55c5814d9530 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.732+0000 7f2146800700 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814ef8f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2140010040 tx=0x7f2140029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b278872b441bec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.732+0000 7f2127e00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.732+0000 7f2127e00700 1 -- 172.21.15.112:0/2953897346 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140045030 con 0x55c5814d9530 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.732+0000 7f215750ab80 1 -- 172.21.15.112:0/2953897346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2140031070 con 0x55c5814d9530 2024-04-25T18:46:15.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.732+0000 7f2127e00700 1 -- 172.21.15.112:0/2953897346 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f2140027070 con 0x55c5814d9530 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.732+0000 7f2127e00700 1 -- 172.21.15.112:0/2953897346 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f214002c650 con 0x55c5814d9530 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.732+0000 7f215750ab80 10 client.14757 fetch_fsmap learned FSMap version 25 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.732+0000 7f215750ab80 1 -- 172.21.15.112:0/2953897346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c581551b20 con 0x55c5814d9530 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.733+0000 7f2127e00700 1 -- 172.21.15.112:0/2953897346 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2140031070 con 0x55c5814d9530 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.733+0000 7f2127e00700 1 -- 172.21.15.112:0/2953897346 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f214003f070 con 0x55c5814d9530 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.733+0000 7f215750ab80 10 client.14757 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.733+0000 7f215750ab80 10 client.14757 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.733+0000 7f215750ab80 20 client.14757 populate_metadata read hostname 'smithi112' 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.733+0000 7f215750ab80 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x55c581556340 0x55c581555050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.733+0000 7f215750ab80 4 client.14757 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.733+0000 7f215750ab80 1 -- 172.21.15.112:0/2953897346 --> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] -- command(tid 0: {"prefix": "status"}) -- 0x55c5814296d0 con 0x55c581556340 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.733+0000 7f2145e00700 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x55c581556340 0x55c581555050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:15.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f2145e00700 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x55c581556340 0x55c581555050 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e17c54cbcd585ec0 server_cookie=ba64d5e4fa9ccf94 in_seq=0 out_seq=0 2024-04-25T18:46:15.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f2127e00700 10 client.14757 ms_handle_connect on v2:172.21.15.171:6838/1642228003 2024-04-25T18:46:15.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f2127e00700 1 -- 172.21.15.112:0/2953897346 <== mds.2 v2:172.21.15.171:6838/1642228003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c5814296d0 con 0x55c581556340 2024-04-25T18:46:15.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f2127e00700 10 client.14757 handle_command_reply: tid=0 2024-04-25T18:46:15.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f215750ab80 1 client.14757 shutdown 2024-04-25T18:46:15.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f215750ab80 1 -- 172.21.15.112:0/2953897346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 msgr2=0x55c5814ef8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:15.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f215750ab80 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814ef8f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2140010040 tx=0x7f2140029000 comp rx=0 tx=0).stop 2024-04-25T18:46:15.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f215750ab80 1 -- 172.21.15.112:0/2953897346 shutdown_connections 2024-04-25T18:46:15.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f215750ab80 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x55c581556340 0x55c581555050 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f215750ab80 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5814d9530 0x55c5814ef8f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f215750ab80 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c5814cfef0 0x55c5814f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f215750ab80 1 --2- 172.21.15.112:0/2953897346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5814d5780 0x55c5814ef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f2127e00700 0 client.14757 ms_handle_reset on v2:172.21.15.171:6838/1642228003 2024-04-25T18:46:15.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f215750ab80 1 -- 172.21.15.112:0/2953897346 >> 172.21.15.112:0/2953897346 conn(0x55c5813fdf00 msgr2=0x55c5814d6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:15.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.734+0000 7f215750ab80 1 -- 172.21.15.112:0/2953897346 shutdown_connections 2024-04-25T18:46:15.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.735+0000 7f215750ab80 1 -- 172.21.15.112:0/2953897346 wait complete. 2024-04-25T18:46:15.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.735+0000 7f215750ab80 20 client.14757 trim_cache size 0 max 16384 2024-04-25T18:46:15.740 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24493, 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 157.514063999, 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 25, 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 94, 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 211.84928336600001 2024-04-25T18:46:15.741 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:46:15.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.735+0000 7f214cc00700 1 -- 172.21.15.112:0/885821305 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f212c0a3c40 msgr2=0x7f212c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:15.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.735+0000 7f214cc00700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f212c0a3c40 0x7f212c0a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2140004fd0 tx=0x7f2140004ec0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.735+0000 7f214cc00700 1 -- 172.21.15.112:0/885821305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2148074c70 msgr2=0x7f214819b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:15.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.735+0000 7f214cc00700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2148074c70 0x7f214819b320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f213800ec10 tx=0x7f213800ef20 comp rx=0 tx=0).stop 2024-04-25T18:46:15.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.736+0000 7f214cc00700 1 -- 172.21.15.112:0/885821305 shutdown_connections 2024-04-25T18:46:15.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.736+0000 7f214cc00700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f212c0a3c40 0x7f212c0a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.736+0000 7f214cc00700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f214810f440 0x7f214819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.736+0000 7f214cc00700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2148074c70 0x7f214819b320 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.736+0000 7f214cc00700 1 --2- 172.21.15.112:0/885821305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2148075590 0x7f214819b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:15.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.736+0000 7f214cc00700 1 -- 172.21.15.112:0/885821305 >> 172.21.15.112:0/885821305 conn(0x7f2148079ba0 msgr2=0x7f21480ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:15.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.736+0000 7f214cc00700 1 -- 172.21.15.112:0/885821305 shutdown_connections 2024-04-25T18:46:15.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:15.736+0000 7f214cc00700 1 -- 172.21.15.112:0/885821305 wait complete. 2024-04-25T18:46:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:16 smithi112 ceph-mon[27864]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:16.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.a status 2024-04-25T18:46:16.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:16 smithi171 ceph-mon[33266]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:16.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:16 smithi161 ceph-mon[32609]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:17.075 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:17.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:17 smithi112 ceph-mon[27864]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:17.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:17 smithi171 ceph-mon[33266]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:17.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:17 smithi161 ceph-mon[32609]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:19.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.459+0000 7f032e200700 1 -- 172.21.15.112:0/4219669266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0328074c70 msgr2=0x7f0328075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:19.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.459+0000 7f032e200700 1 --2- 172.21.15.112:0/4219669266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0328074c70 0x7f0328075050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0318009990 tx=0x7f0318009ca0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.459+0000 7f032e200700 1 -- 172.21.15.112:0/4219669266 shutdown_connections 2024-04-25T18:46:19.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.459+0000 7f032e200700 1 --2- 172.21.15.112:0/4219669266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0328113c10 0x7f0328116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.459+0000 7f032e200700 1 --2- 172.21.15.112:0/4219669266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0328074c70 0x7f0328075050 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.459+0000 7f032e200700 1 --2- 172.21.15.112:0/4219669266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0328075590 0x7f03281136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.459+0000 7f032e200700 1 -- 172.21.15.112:0/4219669266 >> 172.21.15.112:0/4219669266 conn(0x7f0328079ba0 msgr2=0x7f0328079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:19.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.460+0000 7f032e200700 1 -- 172.21.15.112:0/4219669266 shutdown_connections 2024-04-25T18:46:19.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.460+0000 7f032e200700 1 -- 172.21.15.112:0/4219669266 wait complete. 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.461+0000 7f032e200700 1 Processor -- start 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.461+0000 7f032e200700 1 -- start start 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.461+0000 7f032e200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0328074c70 0x7f032819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.461+0000 7f032e200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0328075590 0x7f032819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.461+0000 7f032e200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0328113c10 0x7f03281a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.461+0000 7f032e200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03281191f0 con 0x7f0328075590 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.461+0000 7f032e200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0328119070 con 0x7f0328074c70 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.461+0000 7f032e200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0328119370 con 0x7f0328113c10 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0327400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0328074c70 0x7f032819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0326a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0328075590 0x7f032819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0327400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0328074c70 0x7f032819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:38646/0 (socket says 172.21.15.112:38646) 2024-04-25T18:46:19.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0327400700 1 -- 172.21.15.112:0/531645632 learned_addr learned my addr 172.21.15.112:0/531645632 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:19.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0327e00700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0328113c10 0x7f03281a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0326a00700 1 -- 172.21.15.112:0/531645632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0328113c10 msgr2=0x7f03281a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:19.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0326a00700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0328113c10 0x7f03281a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0326a00700 1 -- 172.21.15.112:0/531645632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0328074c70 msgr2=0x7f032819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:19.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0326a00700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0328074c70 0x7f032819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0326a00700 1 -- 172.21.15.112:0/531645632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03281a4730 con 0x7f0328075590 2024-04-25T18:46:19.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0327400700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0328074c70 0x7f032819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:19.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.462+0000 7f0326a00700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0328075590 0x7f032819fc60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f031400e9f0 tx=0x7f031400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:19.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.463+0000 7f031fe00700 1 -- 172.21.15.112:0/531645632 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031400c9c0 con 0x7f0328075590 2024-04-25T18:46:19.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.463+0000 7f031fe00700 1 -- 172.21.15.112:0/531645632 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031400cb60 con 0x7f0328075590 2024-04-25T18:46:19.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.463+0000 7f032e200700 1 -- 172.21.15.112:0/531645632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03281a4a20 con 0x7f0328075590 2024-04-25T18:46:19.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.463+0000 7f031fe00700 1 -- 172.21.15.112:0/531645632 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314018680 con 0x7f0328075590 2024-04-25T18:46:19.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.463+0000 7f032e200700 1 -- 172.21.15.112:0/531645632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03281ac2a0 con 0x7f0328075590 2024-04-25T18:46:19.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.466+0000 7f031fe00700 1 -- 172.21.15.112:0/531645632 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f0314010430 con 0x7f0328075590 2024-04-25T18:46:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.468+0000 7f0338abeb80 1 Processor -- start 2024-04-25T18:46:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.468+0000 7f0338abeb80 1 -- start start 2024-04-25T18:46:19.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.468+0000 7f031fe00700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f030c0a3cc0 0x7f030c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c13f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f40a0 0x5634c13ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 0x5634c13ee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f031fe00700 1 -- 172.21.15.112:0/531645632 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f03140d0560 con 0x7f0328075590 2024-04-25T18:46:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0338abeb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5634c133fe40 con 0x5634c13f40a0 2024-04-25T18:46:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0338abeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5634c12e66e0 con 0x5634c13edef0 2024-04-25T18:46:19.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0338abeb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5634c133be00 con 0x5634c13f3780 2024-04-25T18:46:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0327e00700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f030c0a3cc0 0x7f030c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0326a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f40a0 0x5634c13ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0326a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f40a0 0x5634c13ed9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54640/0 (socket says 172.21.15.112:54640) 2024-04-25T18:46:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0326a00700 1 -- 172.21.15.112:0/892270272 learned_addr learned my addr 172.21.15.112:0/892270272 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0327e00700 1 --2- 172.21.15.112:0/892270272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 0x5634c13ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.469+0000 7f0327400700 1 --2- 172.21.15.112:0/892270272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c13f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.470+0000 7f0326a00700 1 -- 172.21.15.112:0/892270272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 msgr2=0x5634c13f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.470+0000 7f0326a00700 1 --2- 172.21.15.112:0/892270272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c13f3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.470+0000 7f0326a00700 1 -- 172.21.15.112:0/892270272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 msgr2=0x5634c13ee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.470+0000 7f0326a00700 1 --2- 172.21.15.112:0/892270272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 0x5634c13ee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.470+0000 7f0326a00700 1 -- 172.21.15.112:0/892270272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c12dc630 con 0x5634c13f40a0 2024-04-25T18:46:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.470+0000 7f0326a00700 1 --2- 172.21.15.112:0/892270272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f40a0 0x5634c13ed9b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f03140a4f00 tx=0x7f031409cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b459d29798bd2c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.470+0000 7f0327e00700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f030c0a3cc0 0x7f030c0a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0320009ec0 tx=0x7f03200134a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.470+0000 7f031cc00700 1 -- 172.21.15.112:0/892270272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03140a0b70 con 0x5634c13f40a0 2024-04-25T18:46:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.470+0000 7f031cc00700 1 -- 172.21.15.112:0/892270272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03140a5430 con 0x5634c13f40a0 2024-04-25T18:46:19.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.470+0000 7f031cc00700 1 -- 172.21.15.112:0/892270272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03140d0350 con 0x5634c13f40a0 2024-04-25T18:46:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.471+0000 7f0338abeb80 1 -- 172.21.15.112:0/892270272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f40a0 msgr2=0x5634c13ed9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.471+0000 7f0338abeb80 1 --2- 172.21.15.112:0/892270272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f40a0 0x5634c13ed9b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f03140a4f00 tx=0x7f031409cec0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.471+0000 7f0338abeb80 1 -- 172.21.15.112:0/892270272 shutdown_connections 2024-04-25T18:46:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.471+0000 7f0338abeb80 1 --2- 172.21.15.112:0/892270272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 0x5634c13ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.471+0000 7f0338abeb80 1 --2- 172.21.15.112:0/892270272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c13f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.471+0000 7f0338abeb80 1 --2- 172.21.15.112:0/892270272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f40a0 0x5634c13ed9b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.471+0000 7f0338abeb80 1 -- 172.21.15.112:0/892270272 >> 172.21.15.112:0/892270272 conn(0x5634c131bf00 msgr2=0x5634c13e88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.471+0000 7f0338abeb80 1 -- 172.21.15.112:0/892270272 shutdown_connections 2024-04-25T18:46:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.471+0000 7f0338abeb80 1 -- 172.21.15.112:0/892270272 wait complete. 2024-04-25T18:46:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.472+0000 7f0338abeb80 1 -- start start 2024-04-25T18:46:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.472+0000 7f0338abeb80 1 Processor -- start 2024-04-25T18:46:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.472+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c13f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13f7480 0x5634c13f7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0338abeb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5634c13fa8c0 con 0x5634c13edef0 2024-04-25T18:46:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0338abeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5634c12f9810 con 0x5634c13f7480 2024-04-25T18:46:19.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0338abeb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5634c13faa40 con 0x5634c13f3780 2024-04-25T18:46:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0338abeb80 10 client.0 _ll_register_callbacks cb 0x5634c12d86d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:46:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0327400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0327400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54650/0 (socket says 172.21.15.112:54650) 2024-04-25T18:46:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0327400700 1 -- 172.21.15.112:0/3969407264 learned_addr learned my addr 172.21.15.112:0/3969407264 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0326a00700 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c13f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0327e00700 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13f7480 0x5634c13f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0327400700 1 -- 172.21.15.112:0/3969407264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 msgr2=0x5634c13f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:19.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0327400700 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c13f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0327400700 1 -- 172.21.15.112:0/3969407264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13f7480 msgr2=0x5634c13f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0327400700 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13f7480 0x5634c13f7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.473+0000 7f0327400700 1 -- 172.21.15.112:0/3969407264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c128d880 con 0x5634c13edef0 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.474+0000 7f0327400700 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f9c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f03180052a0 tx=0x7f03180052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1330ef37968ee1e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.474+0000 7f0311600700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.474+0000 7f0311600700 1 -- 172.21.15.112:0/3969407264 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318029060 con 0x5634c13edef0 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.474+0000 7f0311600700 1 -- 172.21.15.112:0/3969407264 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f0318003a90 con 0x5634c13edef0 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.474+0000 7f0338abeb80 1 -- 172.21.15.112:0/3969407264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03140a5430 con 0x5634c13edef0 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.474+0000 7f0311600700 1 -- 172.21.15.112:0/3969407264 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03180056d0 con 0x5634c13edef0 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.474+0000 7f0338abeb80 10 client.14769 fetch_fsmap learned FSMap version 25 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.474+0000 7f0338abeb80 1 -- 172.21.15.112:0/3969407264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634c12d6aa0 con 0x5634c13edef0 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.475+0000 7f0311600700 1 -- 172.21.15.112:0/3969407264 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03140a5430 con 0x5634c13edef0 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.475+0000 7f0311600700 1 -- 172.21.15.112:0/3969407264 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f031802f400 con 0x5634c13edef0 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.475+0000 7f0338abeb80 10 client.14769 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.475+0000 7f0338abeb80 10 client.14769 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.475+0000 7f0338abeb80 20 client.14769 populate_metadata read hostname 'smithi112' 2024-04-25T18:46:19.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.475+0000 7f0338abeb80 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5634c14079a0 0x5634c1408e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.475+0000 7f0338abeb80 4 client.14769 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:46:19.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.475+0000 7f0338abeb80 1 -- 172.21.15.112:0/3969407264 --> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634c134d630 con 0x5634c14079a0 2024-04-25T18:46:19.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.475+0000 7f0326a00700 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5634c14079a0 0x5634c1408e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.476+0000 7f0326a00700 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5634c14079a0 0x5634c1408e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=360efe0fcc585b60 server_cookie=917da016573e2ac9 in_seq=0 out_seq=0 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.476+0000 7f0311600700 10 client.14769 ms_handle_connect on v2:172.21.15.112:6840/1028859850 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0311600700 1 -- 172.21.15.112:0/3969407264 <== mds.1 v2:172.21.15.112:6840/1028859850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634c134d630 con 0x5634c14079a0 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0311600700 10 client.14769 handle_command_reply: tid=0 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0338abeb80 1 client.14769 shutdown 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0338abeb80 1 -- 172.21.15.112:0/3969407264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 msgr2=0x5634c13f9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0338abeb80 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f9c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f03180052a0 tx=0x7f03180052d0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0338abeb80 1 -- 172.21.15.112:0/3969407264 shutdown_connections 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0338abeb80 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5634c14079a0 0x5634c1408e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0338abeb80 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13f7480 0x5634c13f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0338abeb80 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f9c40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0338abeb80 1 --2- 172.21.15.112:0/3969407264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c13f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0338abeb80 1 -- 172.21.15.112:0/3969407264 >> 172.21.15.112:0/3969407264 conn(0x5634c131bf00 msgr2=0x5634c13e7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:19.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.477+0000 7f0311600700 0 client.14769 ms_handle_reset on v2:172.21.15.112:6840/1028859850 2024-04-25T18:46:19.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.478+0000 7f0338abeb80 1 -- 172.21.15.112:0/3969407264 shutdown_connections 2024-04-25T18:46:19.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.478+0000 7f0338abeb80 1 -- 172.21.15.112:0/3969407264 wait complete. 2024-04-25T18:46:19.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.478+0000 7f0338abeb80 20 client.14769 trim_cache size 0 max 16384 2024-04-25T18:46:19.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.500+0000 7f0338abeb80 1 Processor -- start 2024-04-25T18:46:19.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.500+0000 7f0338abeb80 1 -- start start 2024-04-25T18:46:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.500+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.500+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c1403e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.500+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13f7480 0x5634c13f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.500+0000 7f0338abeb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5634c1407150 con 0x5634c13edef0 2024-04-25T18:46:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.500+0000 7f0338abeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5634c1407820 con 0x5634c13f7480 2024-04-25T18:46:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.500+0000 7f0338abeb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5634c1407ef0 con 0x5634c13f3780 2024-04-25T18:46:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.501+0000 7f0327400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.501+0000 7f0327400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54656/0 (socket says 172.21.15.112:54656) 2024-04-25T18:46:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.501+0000 7f0327400700 1 -- 172.21.15.112:0/2606118534 learned_addr learned my addr 172.21.15.112:0/2606118534 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.501+0000 7f0326a00700 1 --2- 172.21.15.112:0/2606118534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c1403e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.501+0000 7f0327e00700 1 --2- 172.21.15.112:0/2606118534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13f7480 0x5634c13f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.501+0000 7f0327400700 1 -- 172.21.15.112:0/2606118534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 msgr2=0x5634c1403e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:19.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.501+0000 7f0327400700 1 --2- 172.21.15.112:0/2606118534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c1403e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.501+0000 7f0327400700 1 -- 172.21.15.112:0/2606118534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13f7480 msgr2=0x5634c13f92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:19.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.501+0000 7f0327400700 1 --2- 172.21.15.112:0/2606118534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13f7480 0x5634c13f92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.501+0000 7f0327400700 1 -- 172.21.15.112:0/2606118534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c13fa220 con 0x5634c13edef0 2024-04-25T18:46:19.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.501+0000 7f0327400700 1 --2- 172.21.15.112:0/2606118534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f4ba0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0318009350 tx=0x7f0318009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9744f258022d2f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.502+0000 7f0313e00700 1 -- 172.21.15.112:0/2606118534 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031802f600 con 0x5634c13edef0 2024-04-25T18:46:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.502+0000 7f0313e00700 1 -- 172.21.15.112:0/2606118534 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0318003d40 con 0x5634c13edef0 2024-04-25T18:46:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.502+0000 7f0313e00700 1 -- 172.21.15.112:0/2606118534 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318024770 con 0x5634c13edef0 2024-04-25T18:46:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.502+0000 7f0338abeb80 1 -- 172.21.15.112:0/2606118534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 msgr2=0x5634c13f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.502+0000 7f0338abeb80 1 --2- 172.21.15.112:0/2606118534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f4ba0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0318009350 tx=0x7f0318009ca0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.502+0000 7f0338abeb80 1 -- 172.21.15.112:0/2606118534 shutdown_connections 2024-04-25T18:46:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.502+0000 7f0338abeb80 1 --2- 172.21.15.112:0/2606118534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13f7480 0x5634c13f92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.502+0000 7f0338abeb80 1 --2- 172.21.15.112:0/2606118534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13edef0 0x5634c13f4ba0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.502+0000 7f0338abeb80 1 --2- 172.21.15.112:0/2606118534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c1403e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.502+0000 7f0338abeb80 1 -- 172.21.15.112:0/2606118534 >> 172.21.15.112:0/2606118534 conn(0x5634c131bf00 msgr2=0x5634c13f4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:19.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.503+0000 7f0338abeb80 1 -- 172.21.15.112:0/2606118534 shutdown_connections 2024-04-25T18:46:19.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.503+0000 7f0338abeb80 1 -- 172.21.15.112:0/2606118534 wait complete. 2024-04-25T18:46:19.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.503+0000 7f0338abeb80 1 -- start start 2024-04-25T18:46:19.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.503+0000 7f0338abeb80 1 Processor -- start 2024-04-25T18:46:19.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 0x5634c1412290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c14127d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0338abeb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f7480 0x5634c140d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0338abeb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5634c1408c90 con 0x5634c13f7480 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0338abeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5634c14085c0 con 0x5634c13edef0 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0338abeb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5634c1409360 con 0x5634c13f3780 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0338abeb80 10 client.0 _ll_register_callbacks cb 0x5634c12d86d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0327400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 0x5634c1412290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0327400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 0x5634c1412290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:38686/0 (socket says 172.21.15.112:38686) 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0327400700 1 -- 172.21.15.112:0/488419008 learned_addr learned my addr 172.21.15.112:0/488419008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0326a00700 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c14127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0327e00700 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f7480 0x5634c140d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0327400700 1 -- 172.21.15.112:0/488419008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 msgr2=0x5634c14127d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0327400700 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c14127d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0327400700 1 -- 172.21.15.112:0/488419008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f7480 msgr2=0x5634c140d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0327400700 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f7480 0x5634c140d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0327400700 1 -- 172.21.15.112:0/488419008 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c140da80 con 0x5634c13edef0 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.504+0000 7f0327e00700 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f7480 0x5634c140d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.505+0000 7f0327400700 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 0x5634c1412290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0318005850 tx=0x7f0318003f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30686962ec1cd492 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:19.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.505+0000 7f0311600700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.505+0000 7f0311600700 1 -- 172.21.15.112:0/488419008 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318003890 con 0x5634c13edef0 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.505+0000 7f0338abeb80 1 -- 172.21.15.112:0/488419008 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0318003d40 con 0x5634c13edef0 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.505+0000 7f0311600700 1 -- 172.21.15.112:0/488419008 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f0318003a30 con 0x5634c13edef0 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.505+0000 7f0311600700 1 -- 172.21.15.112:0/488419008 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0318047070 con 0x5634c13edef0 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.505+0000 7f0338abeb80 10 client.24884 fetch_fsmap learned FSMap version 25 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.505+0000 7f0338abeb80 1 -- 172.21.15.112:0/488419008 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634c146f590 con 0x5634c13edef0 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.505+0000 7f0311600700 1 -- 172.21.15.112:0/488419008 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0318003d40 con 0x5634c13edef0 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.506+0000 7f0311600700 1 -- 172.21.15.112:0/488419008 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f03180434e0 con 0x5634c13edef0 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.506+0000 7f0338abeb80 10 client.24884 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.506+0000 7f0338abeb80 10 client.24884 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.506+0000 7f0338abeb80 20 client.24884 populate_metadata read hostname 'smithi112' 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.506+0000 7f0338abeb80 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5634c1473f50 0x5634c1472c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.506+0000 7f0338abeb80 4 client.24884 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.506+0000 7f0338abeb80 1 -- 172.21.15.112:0/488419008 --> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] -- command(tid 0: {"prefix": "status"}) -- 0x5634c13476d0 con 0x5634c1473f50 2024-04-25T18:46:19.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.506+0000 7f0326a00700 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5634c1473f50 0x5634c1472c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:19.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0326a00700 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5634c1473f50 0x5634c1472c60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b6d649188163667 server_cookie=89bb22ea92f8701b in_seq=0 out_seq=0 2024-04-25T18:46:19.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0311600700 10 client.24884 ms_handle_connect on v2:172.21.15.112:6840/1028859850 2024-04-25T18:46:19.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0311600700 1 -- 172.21.15.112:0/488419008 <== mds.1 v2:172.21.15.112:6840/1028859850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5634c13476d0 con 0x5634c1473f50 2024-04-25T18:46:19.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0311600700 10 client.24884 handle_command_reply: tid=0 2024-04-25T18:46:19.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0338abeb80 1 client.24884 shutdown 2024-04-25T18:46:19.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0338abeb80 1 -- 172.21.15.112:0/488419008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 msgr2=0x5634c1412290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0338abeb80 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 0x5634c1412290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0318005850 tx=0x7f0318003f60 comp rx=0 tx=0).stop 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0338abeb80 1 -- 172.21.15.112:0/488419008 shutdown_connections 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0338abeb80 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634c13f7480 0x5634c140d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0338abeb80 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5634c1473f50 0x5634c1472c60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0338abeb80 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c13edef0 0x5634c1412290 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0338abeb80 1 --2- 172.21.15.112:0/488419008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634c13f3780 0x5634c14127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0311600700 0 client.24884 ms_handle_reset on v2:172.21.15.112:6840/1028859850 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0338abeb80 1 -- 172.21.15.112:0/488419008 >> 172.21.15.112:0/488419008 conn(0x5634c131bf00 msgr2=0x5634c13f5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0338abeb80 1 -- 172.21.15.112:0/488419008 shutdown_connections 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.507+0000 7f0338abeb80 1 -- 172.21.15.112:0/488419008 wait complete. 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.508+0000 7f0338abeb80 20 client.24884 trim_cache size 0 max 16384 2024-04-25T18:46:19.511 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24511, 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 162.29760235000001, 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 25, 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 94, 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 205.881842154 2024-04-25T18:46:19.512 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:46:19.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.508+0000 7f032e200700 1 -- 172.21.15.112:0/531645632 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f030c0a3cc0 msgr2=0x7f030c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:19.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.508+0000 7f032e200700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f030c0a3cc0 0x7f030c0a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0320009ec0 tx=0x7f03200134a0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.508+0000 7f032e200700 1 -- 172.21.15.112:0/531645632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0328075590 msgr2=0x7f032819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:19.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.508+0000 7f032e200700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0328075590 0x7f032819fc60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f031400e9f0 tx=0x7f031400ed00 comp rx=0 tx=0).stop 2024-04-25T18:46:19.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.508+0000 7f032e200700 1 -- 172.21.15.112:0/531645632 shutdown_connections 2024-04-25T18:46:19.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.508+0000 7f032e200700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f030c0a3cc0 0x7f030c0a6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.508+0000 7f032e200700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0328113c10 0x7f03281a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.508+0000 7f032e200700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0328074c70 0x7f032819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.508+0000 7f032e200700 1 --2- 172.21.15.112:0/531645632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0328075590 0x7f032819fc60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:19.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.508+0000 7f032e200700 1 -- 172.21.15.112:0/531645632 >> 172.21.15.112:0/531645632 conn(0x7f0328079ba0 msgr2=0x7f0328112ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:19.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.509+0000 7f032e200700 1 -- 172.21.15.112:0/531645632 shutdown_connections 2024-04-25T18:46:19.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:19.509+0000 7f032e200700 1 -- 172.21.15.112:0/531645632 wait complete. 2024-04-25T18:46:20.392 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T18:46:20.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:20 smithi171 ceph-mon[33266]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:20 smithi161 ceph-mon[32609]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:20.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:20 smithi112 ceph-mon[27864]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:22.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:22 smithi112 ceph-mon[27864]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:22 smithi171 ceph-mon[33266]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:22 smithi161 ceph-mon[32609]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:23.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.062+0000 7fcc3e600700 1 -- 172.21.15.112:0/2454510195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38108a30 msgr2=0x7fcc38108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:23.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.062+0000 7fcc3e600700 1 --2- 172.21.15.112:0/2454510195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38108a30 0x7fcc38108e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc280098f0 tx=0x7fcc28009c00 comp rx=0 tx=0).stop 2024-04-25T18:46:23.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.062+0000 7fcc3e600700 1 -- 172.21.15.112:0/2454510195 shutdown_connections 2024-04-25T18:46:23.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.062+0000 7fcc3e600700 1 --2- 172.21.15.112:0/2454510195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3806b0f0 0x7fcc3810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.062+0000 7fcc3e600700 1 --2- 172.21.15.112:0/2454510195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38108a30 0x7fcc38108e10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.062+0000 7fcc3e600700 1 --2- 172.21.15.112:0/2454510195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc3806a750 0x7fcc3806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.062+0000 7fcc3e600700 1 -- 172.21.15.112:0/2454510195 >> 172.21.15.112:0/2454510195 conn(0x7fcc380788f0 msgr2=0x7fcc38078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:23.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.062+0000 7fcc3e600700 1 -- 172.21.15.112:0/2454510195 shutdown_connections 2024-04-25T18:46:23.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.062+0000 7fcc3e600700 1 -- 172.21.15.112:0/2454510195 wait complete. 2024-04-25T18:46:23.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.063+0000 7fcc3e600700 1 Processor -- start 2024-04-25T18:46:23.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc3e600700 1 -- start start 2024-04-25T18:46:23.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc3e600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc3806a750 0x7fcc3819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc3e600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3806b0f0 0x7fcc3819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc3e600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc38108a30 0x7fcc3819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc3e600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38112c00 con 0x7fcc3806a750 2024-04-25T18:46:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc3e600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38112a80 con 0x7fcc3806b0f0 2024-04-25T18:46:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc3e600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38112d80 con 0x7fcc38108a30 2024-04-25T18:46:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc37400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc3806a750 0x7fcc3819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc2fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3806b0f0 0x7fcc3819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc37400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc3806a750 0x7fcc3819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35904/0 (socket says 172.21.15.112:35904) 2024-04-25T18:46:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc37400700 1 -- 172.21.15.112:0/1715079677 learned_addr learned my addr 172.21.15.112:0/1715079677 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.064+0000 7fcc37e00700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc38108a30 0x7fcc3819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.065+0000 7fcc37400700 1 -- 172.21.15.112:0/1715079677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc38108a30 msgr2=0x7fcc3819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.065+0000 7fcc37400700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc38108a30 0x7fcc3819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.065+0000 7fcc37400700 1 -- 172.21.15.112:0/1715079677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3806b0f0 msgr2=0x7fcc3819b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.065+0000 7fcc37400700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3806b0f0 0x7fcc3819b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.065+0000 7fcc37400700 1 -- 172.21.15.112:0/1715079677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc381a0460 con 0x7fcc3806a750 2024-04-25T18:46:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.065+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3806b0f0 0x7fcc3819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.065+0000 7fcc37e00700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc38108a30 0x7fcc3819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:23.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.065+0000 7fcc37400700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc3806a750 0x7fcc3819b450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009230 tx=0x7fcc2800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:23.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.065+0000 7fcc34c00700 1 -- 172.21.15.112:0/1715079677 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28005140 con 0x7fcc3806a750 2024-04-25T18:46:23.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.066+0000 7fcc34c00700 1 -- 172.21.15.112:0/1715079677 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc2800b560 con 0x7fcc3806a750 2024-04-25T18:46:23.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.066+0000 7fcc34c00700 1 -- 172.21.15.112:0/1715079677 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28003970 con 0x7fcc3806a750 2024-04-25T18:46:23.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.066+0000 7fcc3e600700 1 -- 172.21.15.112:0/1715079677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc381a06f0 con 0x7fcc3806a750 2024-04-25T18:46:23.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.066+0000 7fcc3e600700 1 -- 172.21.15.112:0/1715079677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc381a7fd0 con 0x7fcc3806a750 2024-04-25T18:46:23.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.069+0000 7fcc34c00700 1 -- 172.21.15.112:0/1715079677 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fcc2800ba50 con 0x7fcc3806a750 2024-04-25T18:46:23.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.070+0000 7fcc34c00700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fcc1c0a3c40 0x7fcc1c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.070+0000 7fcc34c00700 1 -- 172.21.15.112:0/1715079677 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fcc280d13b0 con 0x7fcc3806a750 2024-04-25T18:46:23.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.071+0000 7fcc48fa4b80 1 Processor -- start 2024-04-25T18:46:23.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.071+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fcc1c0a3c40 0x7fcc1c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.071+0000 7fcc48fa4b80 1 -- start start 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.071+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c5b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.071+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5bc10 0x5581a3c66570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.071+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c68ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.071+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581a3baee40 con 0x5581a3c5b2f0 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.071+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5581a3b556e0 con 0x5581a3c5bc10 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.071+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5581a3baae00 con 0x5581a3c66ab0 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc37400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c5b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc37e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c68ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc37e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c68ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:43838/0 (socket says 172.21.15.112:43838) 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc37400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c5b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35908/0 (socket says 172.21.15.112:35908) 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc37e00700 1 -- 172.21.15.112:0/1723919167 learned_addr learned my addr 172.21.15.112:0/1723919167 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fcc1c0a3c40 0x7fcc1c0a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009750 tx=0x7fcc20009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:23.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/1723919167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5bc10 0x5581a3c66570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc37e00700 1 -- 172.21.15.112:0/1723919167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5bc10 msgr2=0x5581a3c66570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:23.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc37e00700 1 --2- 172.21.15.112:0/1723919167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5bc10 0x5581a3c66570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc37e00700 1 -- 172.21.15.112:0/1723919167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 msgr2=0x5581a3c5b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:23.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc37e00700 1 --2- 172.21.15.112:0/1723919167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c5b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc37e00700 1 -- 172.21.15.112:0/1723919167 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581a3b4b630 con 0x5581a3c66ab0 2024-04-25T18:46:23.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.072+0000 7fcc37e00700 1 --2- 172.21.15.112:0/1723919167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c68ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009510 tx=0x7fcc30009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb3a62cb9f721246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:23.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.073+0000 7fcc2cc00700 1 -- 172.21.15.112:0/1723919167 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3001b020 con 0x5581a3c66ab0 2024-04-25T18:46:23.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.073+0000 7fcc2cc00700 1 -- 172.21.15.112:0/1723919167 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3000eba0 con 0x5581a3c66ab0 2024-04-25T18:46:23.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.073+0000 7fcc2cc00700 1 -- 172.21.15.112:0/1723919167 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30017d50 con 0x5581a3c66ab0 2024-04-25T18:46:23.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.073+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/1723919167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 msgr2=0x5581a3c68ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:23.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.073+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/1723919167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c68ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009510 tx=0x7fcc30009920 comp rx=0 tx=0).stop 2024-04-25T18:46:23.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.074+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/1723919167 shutdown_connections 2024-04-25T18:46:23.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.074+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/1723919167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c68ea0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.074+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/1723919167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c5b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.074+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/1723919167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5bc10 0x5581a3c66570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.074+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/1723919167 >> 172.21.15.112:0/1723919167 conn(0x5581a3b8a550 msgr2=0x5581a3c57790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:23.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.074+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/1723919167 shutdown_connections 2024-04-25T18:46:23.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.074+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/1723919167 wait complete. 2024-04-25T18:46:23.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.074+0000 7fcc48fa4b80 1 -- start start 2024-04-25T18:46:23.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.074+0000 7fcc48fa4b80 1 Processor -- start 2024-04-25T18:46:23.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c662d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5bc10 0x5581a3c635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c63b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581a3c695a0 con 0x5581a3c5b2f0 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5581a3b68810 con 0x5581a3c5bc10 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5581a3c69720 con 0x5581a3c66ab0 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc48fa4b80 10 client.0 _ll_register_callbacks cb 0x5581a3b476d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc37400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc37e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c63b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc37e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c63b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:43844/0 (socket says 172.21.15.112:43844) 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc37e00700 1 -- 172.21.15.112:0/183816787 learned_addr learned my addr 172.21.15.112:0/183816787 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.075+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5bc10 0x5581a3c635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc37e00700 1 -- 172.21.15.112:0/183816787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5bc10 msgr2=0x5581a3c635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc37e00700 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5bc10 0x5581a3c635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc37e00700 1 -- 172.21.15.112:0/183816787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 msgr2=0x5581a3c662d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc37e00700 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c662d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc37e00700 1 -- 172.21.15.112:0/183816787 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581a3afc880 con 0x5581a3c66ab0 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5bc10 0x5581a3c635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc37400700 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c662d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc37e00700 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c63b10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcc30017b80 tx=0x7fcc30017bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c925679548d48ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc25600700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc25600700 1 -- 172.21.15.112:0/183816787 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3001b040 con 0x5581a3c66ab0 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/183816787 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc3000eba0 con 0x5581a3c66ab0 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.076+0000 7fcc25600700 1 -- 172.21.15.112:0/183816787 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fcc3000e700 con 0x5581a3c66ab0 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.077+0000 7fcc25600700 1 -- 172.21.15.112:0/183816787 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc30030070 con 0x5581a3c66ab0 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.077+0000 7fcc48fa4b80 10 client.24904 fetch_fsmap learned FSMap version 25 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.077+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/183816787 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5581a3b45aa0 con 0x5581a3c66ab0 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.077+0000 7fcc25600700 1 -- 172.21.15.112:0/183816787 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3000eba0 con 0x5581a3c66ab0 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.077+0000 7fcc25600700 1 -- 172.21.15.112:0/183816787 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fcc3002c4c0 con 0x5581a3c66ab0 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.077+0000 7fcc48fa4b80 10 client.24904 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:46:23.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.077+0000 7fcc48fa4b80 10 client.24904 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T18:46:23.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.077+0000 7fcc48fa4b80 20 client.24904 populate_metadata read hostname 'smithi112' 2024-04-25T18:46:23.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.077+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5581a3c75f80 0x5581a3c773f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.077+0000 7fcc48fa4b80 4 client.24904 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:46:23.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.077+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/183816787 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581a3bbc630 con 0x5581a3c75f80 2024-04-25T18:46:23.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.078+0000 7fcc37400700 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5581a3c75f80 0x5581a3c773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.078+0000 7fcc37400700 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5581a3c75f80 0x5581a3c773f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c385e4fb88e43c3f server_cookie=2ed88b4259dbfc13 in_seq=0 out_seq=0 2024-04-25T18:46:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.078+0000 7fcc25600700 10 client.24904 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:46:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc25600700 1 -- 172.21.15.112:0/183816787 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581a3bbc630 con 0x5581a3c75f80 2024-04-25T18:46:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc25600700 10 client.24904 handle_command_reply: tid=0 2024-04-25T18:46:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc48fa4b80 1 client.24904 shutdown 2024-04-25T18:46:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/183816787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 msgr2=0x5581a3c63b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c63b10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcc30017b80 tx=0x7fcc30017bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/183816787 shutdown_connections 2024-04-25T18:46:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5581a3c75f80 0x5581a3c773f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c63b10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc25600700 0 client.24904 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:46:23.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c662d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/183816787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5bc10 0x5581a3c635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.079+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/183816787 >> 172.21.15.112:0/183816787 conn(0x5581a3b8a550 msgr2=0x5581a3c56950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:23.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.080+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/183816787 shutdown_connections 2024-04-25T18:46:23.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.080+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/183816787 wait complete. 2024-04-25T18:46:23.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.080+0000 7fcc48fa4b80 20 client.24904 trim_cache size 0 max 16384 2024-04-25T18:46:23.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.102+0000 7fcc48fa4b80 1 Processor -- start 2024-04-25T18:46:23.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.102+0000 7fcc48fa4b80 1 -- start start 2024-04-25T18:46:23.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.102+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5b2f0 0x5581a3c5eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.102+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5bc10 0x5581a3c80850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.102+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c80d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.102+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581a3c75730 con 0x5581a3c5bc10 2024-04-25T18:46:23.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.102+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5581a3c75e00 con 0x5581a3c5b2f0 2024-04-25T18:46:23.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.102+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5581a3c764d0 con 0x5581a3c66ab0 2024-04-25T18:46:23.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.103+0000 7fcc2fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5bc10 0x5581a3c80850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.103+0000 7fcc2fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5bc10 0x5581a3c80850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35924/0 (socket says 172.21.15.112:35924) 2024-04-25T18:46:23.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.103+0000 7fcc2fe00700 1 -- 172.21.15.112:0/3839623850 learned_addr learned my addr 172.21.15.112:0/3839623850 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:23.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.103+0000 7fcc37e00700 1 --2- 172.21.15.112:0/3839623850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c80d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.103+0000 7fcc37400700 1 --2- 172.21.15.112:0/3839623850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5b2f0 0x5581a3c5eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.103+0000 7fcc2fe00700 1 -- 172.21.15.112:0/3839623850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 msgr2=0x5581a3c80d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:23.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.103+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/3839623850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c80d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.103+0000 7fcc2fe00700 1 -- 172.21.15.112:0/3839623850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5b2f0 msgr2=0x5581a3c5eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:23.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.103+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/3839623850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5b2f0 0x5581a3c5eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.103+0000 7fcc2fe00700 1 -- 172.21.15.112:0/3839623850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581a3c4f020 con 0x5581a3c5bc10 2024-04-25T18:46:23.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.103+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/3839623850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5bc10 0x5581a3c80850 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcc200041f0 tx=0x7fcc20014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716cd585fabb0753 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:23.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.104+0000 7fcc27e00700 1 -- 172.21.15.112:0/3839623850 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20003d10 con 0x5581a3c5bc10 2024-04-25T18:46:23.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.104+0000 7fcc27e00700 1 -- 172.21.15.112:0/3839623850 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc200298f0 con 0x5581a3c5bc10 2024-04-25T18:46:23.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.104+0000 7fcc27e00700 1 -- 172.21.15.112:0/3839623850 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2001f6b0 con 0x5581a3c5bc10 2024-04-25T18:46:23.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.104+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/3839623850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5bc10 msgr2=0x5581a3c80850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:23.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.104+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/3839623850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5bc10 0x5581a3c80850 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcc200041f0 tx=0x7fcc20014000 comp rx=0 tx=0).stop 2024-04-25T18:46:23.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.105+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/3839623850 shutdown_connections 2024-04-25T18:46:23.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.105+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/3839623850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c66ab0 0x5581a3c80d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.105+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/3839623850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c5b2f0 0x5581a3c5eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.105+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/3839623850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5bc10 0x5581a3c80850 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.105+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/3839623850 >> 172.21.15.112:0/3839623850 conn(0x5581a3b8a550 msgr2=0x5581a3c5e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:23.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.105+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/3839623850 shutdown_connections 2024-04-25T18:46:23.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.105+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/3839623850 wait complete. 2024-04-25T18:46:23.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.105+0000 7fcc48fa4b80 1 -- start start 2024-04-25T18:46:23.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.105+0000 7fcc48fa4b80 1 Processor -- start 2024-04-25T18:46:23.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c80610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c5bc10 0x5581a3c80b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc48fa4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c66ab0 0x5581a3c7a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581a3c77270 con 0x5581a3c5b2f0 2024-04-25T18:46:23.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5581a3c76ba0 con 0x5581a3c66ab0 2024-04-25T18:46:23.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc48fa4b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5581a3c77940 con 0x5581a3c5bc10 2024-04-25T18:46:23.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc48fa4b80 10 client.0 _ll_register_callbacks cb 0x7fcc28009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:46:23.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc37400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c80610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc37400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c80610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35932/0 (socket says 172.21.15.112:35932) 2024-04-25T18:46:23.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc37400700 1 -- 172.21.15.112:0/2190794030 learned_addr learned my addr 172.21.15.112:0/2190794030 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:23.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c5bc10 0x5581a3c80b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.106+0000 7fcc37e00700 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c66ab0 0x5581a3c7a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.107+0000 7fcc37400700 1 -- 172.21.15.112:0/2190794030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c5bc10 msgr2=0x5581a3c80b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.107+0000 7fcc37400700 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c5bc10 0x5581a3c80b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.107+0000 7fcc37400700 1 -- 172.21.15.112:0/2190794030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c66ab0 msgr2=0x5581a3c7a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.107+0000 7fcc37400700 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c66ab0 0x5581a3c7a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.107+0000 7fcc37400700 1 -- 172.21.15.112:0/2190794030 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581a3c7b1d0 con 0x5581a3c5b2f0 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.107+0000 7fcc37400700 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c80610 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc28005bf0 tx=0x7fcc28019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3922ffe1ed0f5b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.107+0000 7fcc25600700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.107+0000 7fcc25600700 1 -- 172.21.15.112:0/2190794030 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc280880b0 con 0x5581a3c5b2f0 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.107+0000 7fcc25600700 1 -- 172.21.15.112:0/2190794030 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fcc2800b710 con 0x5581a3c5b2f0 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.107+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/2190794030 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc200298f0 con 0x5581a3c5b2f0 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.108+0000 7fcc25600700 1 -- 172.21.15.112:0/2190794030 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc2809d6a0 con 0x5581a3c5b2f0 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.108+0000 7fcc48fa4b80 10 client.14793 fetch_fsmap learned FSMap version 25 2024-04-25T18:46:23.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.108+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/2190794030 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5581a3ce63c0 con 0x5581a3c5b2f0 2024-04-25T18:46:23.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.108+0000 7fcc25600700 1 -- 172.21.15.112:0/2190794030 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc200298f0 con 0x5581a3c5b2f0 2024-04-25T18:46:23.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.108+0000 7fcc25600700 1 -- 172.21.15.112:0/2190794030 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fcc280d0040 con 0x5581a3c5b2f0 2024-04-25T18:46:23.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.108+0000 7fcc48fa4b80 10 client.14793 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:46:23.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.108+0000 7fcc48fa4b80 10 client.14793 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T18:46:23.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.108+0000 7fcc48fa4b80 20 client.14793 populate_metadata read hostname 'smithi112' 2024-04-25T18:46:23.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.108+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5581a3ceaf80 0x5581a3ce9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:23.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.108+0000 7fcc48fa4b80 4 client.14793 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:46:23.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.108+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/2190794030 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "status"}) -- 0x5581a3bb66d0 con 0x5581a3ceaf80 2024-04-25T18:46:23.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.109+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5581a3ceaf80 0x5581a3ce9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:23.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.109+0000 7fcc2fe00700 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5581a3ceaf80 0x5581a3ce9c90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e5f15220f50cbd server_cookie=73c587808b71aba4 in_seq=0 out_seq=0 2024-04-25T18:46:23.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.109+0000 7fcc25600700 10 client.14793 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.109+0000 7fcc25600700 1 -- 172.21.15.112:0/2190794030 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5581a3bb66d0 con 0x5581a3ceaf80 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.109+0000 7fcc25600700 10 client.14793 handle_command_reply: tid=0 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.109+0000 7fcc48fa4b80 1 client.14793 shutdown 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/2190794030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 msgr2=0x5581a3c80610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c80610 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc28005bf0 tx=0x7fcc28019040 comp rx=0 tx=0).stop 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/2190794030 shutdown_connections 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5581a3ceaf80 0x5581a3ce9c90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581a3c66ab0 0x5581a3c7a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581a3c5b2f0 0x5581a3c80610 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc48fa4b80 1 --2- 172.21.15.112:0/2190794030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5581a3c5bc10 0x5581a3c80b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/2190794030 >> 172.21.15.112:0/2190794030 conn(0x5581a3b8a550 msgr2=0x5581a3c65690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc25600700 0 client.14793 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/2190794030 shutdown_connections 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc48fa4b80 1 -- 172.21.15.112:0/2190794030 wait complete. 2024-04-25T18:46:23.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.110+0000 7fcc48fa4b80 20 client.14793 trim_cache size 0 max 16384 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24512, 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 182.824970665, 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 25, 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 94, 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 206.708494196 2024-04-25T18:46:23.114 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:46:23.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 -- 172.21.15.112:0/1715079677 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fcc1c0a3c40 msgr2=0x7fcc1c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:23.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fcc1c0a3c40 0x7fcc1c0a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009750 tx=0x7fcc20009310 comp rx=0 tx=0).stop 2024-04-25T18:46:23.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 -- 172.21.15.112:0/1715079677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc3806a750 msgr2=0x7fcc3819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:23.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc3806a750 0x7fcc3819b450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009230 tx=0x7fcc2800fa70 comp rx=0 tx=0).stop 2024-04-25T18:46:23.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 -- 172.21.15.112:0/1715079677 shutdown_connections 2024-04-25T18:46:23.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fcc1c0a3c40 0x7fcc1c0a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc38108a30 0x7fcc3819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc3806a750 0x7fcc3819b450 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 --2- 172.21.15.112:0/1715079677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc3806b0f0 0x7fcc3819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:23.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 -- 172.21.15.112:0/1715079677 >> 172.21.15.112:0/1715079677 conn(0x7fcc380788f0 msgr2=0x7fcc3810df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:23.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 -- 172.21.15.112:0/1715079677 shutdown_connections 2024-04-25T18:46:23.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:23.111+0000 7fcc3e600700 1 -- 172.21.15.112:0/1715079677 wait complete. 2024-04-25T18:46:24.013 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T18:46:24.013 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T18:46:24.023 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T18:46:24.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T18:46:24.473 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:24.622 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:24 smithi112 ceph-mon[27864]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:24.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:24 smithi171 ceph-mon[33266]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:24.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:24 smithi161 ceph-mon[32609]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:26.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:26 smithi112 ceph-mon[27864]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:26.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:26 smithi171 ceph-mon[33266]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:26.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:26 smithi161 ceph-mon[32609]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.782+0000 7f1ea5c00700 1 -- 172.21.15.112:0/1522795299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ea00754a0 msgr2=0x7f1ea0075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.782+0000 7f1ea5c00700 1 --2- 172.21.15.112:0/1522795299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ea00754a0 0x7f1ea0075920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009990 tx=0x7f1e88009ca0 comp rx=0 tx=0).stop 2024-04-25T18:46:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.783+0000 7f1ea5c00700 1 -- 172.21.15.112:0/1522795299 shutdown_connections 2024-04-25T18:46:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.783+0000 7f1ea5c00700 1 --2- 172.21.15.112:0/1522795299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ea010a980 0x7f1ea010ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.783+0000 7f1ea5c00700 1 --2- 172.21.15.112:0/1522795299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea0076a70 0x7f1ea0074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.783+0000 7f1ea5c00700 1 --2- 172.21.15.112:0/1522795299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ea00754a0 0x7f1ea0075920 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.783+0000 7f1ea5c00700 1 -- 172.21.15.112:0/1522795299 >> 172.21.15.112:0/1522795299 conn(0x7f1ea00fe010 msgr2=0x7f1ea0100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:26.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.783+0000 7f1ea5c00700 1 -- 172.21.15.112:0/1522795299 shutdown_connections 2024-04-25T18:46:26.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.783+0000 7f1ea5c00700 1 -- 172.21.15.112:0/1522795299 wait complete. 2024-04-25T18:46:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.784+0000 7f1ea5c00700 1 Processor -- start 2024-04-25T18:46:26.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.784+0000 7f1ea5c00700 1 -- start start 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.784+0000 7f1ea5c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ea00754a0 0x7f1ea019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.784+0000 7f1ea5c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea0076a70 0x7f1ea019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.784+0000 7f1ea5c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ea010a980 0x7f1ea01a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.784+0000 7f1ea5c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea0110740 con 0x7f1ea0076a70 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.784+0000 7f1ea5c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01105c0 con 0x7f1ea010a980 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.784+0000 7f1ea5c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01108c0 con 0x7f1ea00754a0 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.785+0000 7f1e97e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea0076a70 0x7f1ea019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.785+0000 7f1e9ea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ea00754a0 0x7f1ea019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.785+0000 7f1e97e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea0076a70 0x7f1ea019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35948/0 (socket says 172.21.15.112:35948) 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.785+0000 7f1e97e00700 1 -- 172.21.15.112:0/3438034690 learned_addr learned my addr 172.21.15.112:0/3438034690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.785+0000 7f1e9f400700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ea010a980 0x7f1ea01a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.785+0000 7f1e9ea00700 1 -- 172.21.15.112:0/3438034690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ea010a980 msgr2=0x7f1ea01a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:26.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.785+0000 7f1e9ea00700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ea010a980 0x7f1ea01a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.785+0000 7f1e9ea00700 1 -- 172.21.15.112:0/3438034690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea0076a70 msgr2=0x7f1ea019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.785+0000 7f1e9ea00700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea0076a70 0x7f1ea019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.785+0000 7f1e9ea00700 1 -- 172.21.15.112:0/3438034690 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea01a2a70 con 0x7f1ea00754a0 2024-04-25T18:46:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.785+0000 7f1e9ea00700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ea00754a0 0x7f1ea019ae10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e9f0 tx=0x7f1e9000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.786+0000 7f1e97400700 1 -- 172.21.15.112:0/3438034690 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000c9c0 con 0x7f1ea00754a0 2024-04-25T18:46:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.786+0000 7f1ea5c00700 1 -- 172.21.15.112:0/3438034690 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea01a2d60 con 0x7f1ea00754a0 2024-04-25T18:46:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.786+0000 7f1e97400700 1 -- 172.21.15.112:0/3438034690 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e90010430 con 0x7f1ea00754a0 2024-04-25T18:46:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.786+0000 7f1e97400700 1 -- 172.21.15.112:0/3438034690 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90018680 con 0x7f1ea00754a0 2024-04-25T18:46:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.786+0000 7f1ea5c00700 1 -- 172.21.15.112:0/3438034690 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea01a3200 con 0x7f1ea00754a0 2024-04-25T18:46:26.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.787+0000 7f1ea5c00700 1 -- 172.21.15.112:0/3438034690 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea0069770 con 0x7f1ea00754a0 2024-04-25T18:46:26.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.789+0000 7f1e97400700 1 -- 172.21.15.112:0/3438034690 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f1e900108e0 con 0x7f1ea00754a0 2024-04-25T18:46:26.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.790+0000 7f1e97400700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e840a3cc0 0x7f1e840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:26.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.791+0000 7f1e97400700 1 -- 172.21.15.112:0/3438034690 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f1e900d0b60 con 0x7f1ea00754a0 2024-04-25T18:46:26.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.791+0000 7f1e97e00700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e840a3cc0 0x7f1e840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:26.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.791+0000 7f1e97e00700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e840a3cc0 0x7f1e840a6180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800b2e0 tx=0x7f1e88005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:26.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:26.793+0000 7f1e97400700 1 -- 172.21.15.112:0/3438034690 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e9009d4c0 con 0x7f1ea00754a0 2024-04-25T18:46:27.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.037+0000 7f1ea5c00700 1 -- 172.21.15.112:0/3438034690 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1ea0062e80 con 0x7f1e840a3cc0 2024-04-25T18:46:27.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.039+0000 7f1e97400700 1 -- 172.21.15.112:0/3438034690 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1ea0062e80 con 0x7f1e840a3cc0 2024-04-25T18:46:27.040 INFO:teuthology.orchestra.run.smithi112.stdout:Backend: cephadm 2024-04-25T18:46:27.040 INFO:teuthology.orchestra.run.smithi112.stdout:Available: Yes 2024-04-25T18:46:27.040 INFO:teuthology.orchestra.run.smithi112.stdout:Paused: No 2024-04-25T18:46:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 -- 172.21.15.112:0/3438034690 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e840a3cc0 msgr2=0x7f1e840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e840a3cc0 0x7f1e840a6180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800b2e0 tx=0x7f1e88005960 comp rx=0 tx=0).stop 2024-04-25T18:46:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 -- 172.21.15.112:0/3438034690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ea00754a0 msgr2=0x7f1ea019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ea00754a0 0x7f1ea019ae10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e9f0 tx=0x7f1e9000ed00 comp rx=0 tx=0).stop 2024-04-25T18:46:27.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 -- 172.21.15.112:0/3438034690 shutdown_connections 2024-04-25T18:46:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f1e840a3cc0 0x7f1e840a6180 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ea010a980 0x7f1ea01a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ea00754a0 0x7f1ea019ae10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 --2- 172.21.15.112:0/3438034690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea0076a70 0x7f1ea019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 -- 172.21.15.112:0/3438034690 >> 172.21.15.112:0/3438034690 conn(0x7f1ea00fe010 msgr2=0x7f1ea0106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 -- 172.21.15.112:0/3438034690 shutdown_connections 2024-04-25T18:46:27.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:27.042+0000 7f1ea5c00700 1 -- 172.21.15.112:0/3438034690 wait complete. 2024-04-25T18:46:27.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T18:46:28.409 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:28.559 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:28 smithi112 ceph-mon[27864]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:28.559 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:28 smithi112 ceph-mon[27864]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:28.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:28 smithi171 ceph-mon[33266]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:28.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:28 smithi171 ceph-mon[33266]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:28.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:28 smithi161 ceph-mon[32609]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:28.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:28 smithi161 ceph-mon[32609]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:30.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:30 smithi112 ceph-mon[27864]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:30.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.636+0000 7ff915600700 1 -- 172.21.15.112:0/2617200050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff910108830 msgr2=0x7ff91010ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:30.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.636+0000 7ff915600700 1 --2- 172.21.15.112:0/2617200050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff910108830 0x7ff91010ac20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff90800b0a0 tx=0x7ff90800b3b0 comp rx=0 tx=0).stop 2024-04-25T18:46:30.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.637+0000 7ff915600700 1 -- 172.21.15.112:0/2617200050 shutdown_connections 2024-04-25T18:46:30.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.637+0000 7ff915600700 1 --2- 172.21.15.112:0/2617200050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff910108830 0x7ff91010ac20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:30.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.637+0000 7ff915600700 1 --2- 172.21.15.112:0/2617200050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9101035d0 0x7ff9101059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:30.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.637+0000 7ff915600700 1 --2- 172.21.15.112:0/2617200050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff910105f00 0x7ff9101082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:30.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.637+0000 7ff915600700 1 -- 172.21.15.112:0/2617200050 >> 172.21.15.112:0/2617200050 conn(0x7ff9100fcf40 msgr2=0x7ff9100ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:30.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.637+0000 7ff915600700 1 -- 172.21.15.112:0/2617200050 shutdown_connections 2024-04-25T18:46:30.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.637+0000 7ff915600700 1 -- 172.21.15.112:0/2617200050 wait complete. 2024-04-25T18:46:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.638+0000 7ff915600700 1 Processor -- start 2024-04-25T18:46:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.638+0000 7ff915600700 1 -- start start 2024-04-25T18:46:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.638+0000 7ff915600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9101035d0 0x7ff91019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.638+0000 7ff915600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff910105f00 0x7ff91019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.638+0000 7ff915600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff910108830 0x7ff9101a2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.638+0000 7ff915600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff91010e580 con 0x7ff910108830 2024-04-25T18:46:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.638+0000 7ff915600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff91010e400 con 0x7ff910105f00 2024-04-25T18:46:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.638+0000 7ff915600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff91010e700 con 0x7ff9101035d0 2024-04-25T18:46:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90e000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff910105f00 0x7ff91019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90ea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9101035d0 0x7ff91019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90e000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff910105f00 0x7ff91019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:59434/0 (socket says 172.21.15.112:59434) 2024-04-25T18:46:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90e000700 1 -- 172.21.15.112:0/1064024759 learned_addr learned my addr 172.21.15.112:0/1064024759 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90f400700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff910108830 0x7ff9101a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90ea00700 1 -- 172.21.15.112:0/1064024759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff910105f00 msgr2=0x7ff91019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90ea00700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff910105f00 0x7ff91019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90ea00700 1 -- 172.21.15.112:0/1064024759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff910108830 msgr2=0x7ff9101a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90ea00700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff910108830 0x7ff9101a2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90ea00700 1 -- 172.21.15.112:0/1064024759 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9101a2ad0 con 0x7ff9101035d0 2024-04-25T18:46:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90e000700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff910105f00 0x7ff91019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.639+0000 7ff90f400700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff910108830 0x7ff9101a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.640+0000 7ff90ea00700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9101035d0 0x7ff91019ae70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff90000ea20 tx=0x7ff90000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.640+0000 7ff907400700 1 -- 172.21.15.112:0/1064024759 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90000cbd0 con 0x7ff9101035d0 2024-04-25T18:46:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.640+0000 7ff907400700 1 -- 172.21.15.112:0/1064024759 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff90000ee60 con 0x7ff9101035d0 2024-04-25T18:46:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.640+0000 7ff915600700 1 -- 172.21.15.112:0/1064024759 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9101a2dc0 con 0x7ff9101035d0 2024-04-25T18:46:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.640+0000 7ff915600700 1 -- 172.21.15.112:0/1064024759 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9101a3270 con 0x7ff9101035d0 2024-04-25T18:46:30.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.640+0000 7ff907400700 1 -- 172.21.15.112:0/1064024759 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff900018680 con 0x7ff9101035d0 2024-04-25T18:46:30.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.641+0000 7ff915600700 1 -- 172.21.15.112:0/1064024759 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f0005330 con 0x7ff9101035d0 2024-04-25T18:46:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.644+0000 7ff907400700 1 -- 172.21.15.112:0/1064024759 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7ff900018820 con 0x7ff9101035d0 2024-04-25T18:46:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.645+0000 7ff907400700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff8f40a3cc0 0x7ff8f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:30.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.645+0000 7ff907400700 1 -- 172.21.15.112:0/1064024759 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7ff9000d0670 con 0x7ff9101035d0 2024-04-25T18:46:30.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.645+0000 7ff90e000700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff8f40a3cc0 0x7ff8f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:30.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.646+0000 7ff90e000700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff8f40a3cc0 0x7ff8f40a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0105c0 tx=0x7ff8fc00b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:30.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.647+0000 7ff907400700 1 -- 172.21.15.112:0/1064024759 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff90009cfd0 con 0x7ff9101035d0 2024-04-25T18:46:30.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:30 smithi171 ceph-mon[33266]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:30.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:30 smithi161 ceph-mon[32609]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:30.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.883+0000 7ff915600700 1 -- 172.21.15.112:0/1064024759 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff8f0000bc0 con 0x7ff8f40a3cc0 2024-04-25T18:46:30.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.898+0000 7ff907400700 1 -- 172.21.15.112:0/1064024759 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7ff8f0000bc0 con 0x7ff8f40a3cc0 2024-04-25T18:46:30.900 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T18:46:30.900 INFO:teuthology.orchestra.run.smithi112.stdout:mds.a smithi112 running (3m) 46s ago 3m 18.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d1f85f3692b8 2024-04-25T18:46:30.900 INFO:teuthology.orchestra.run.smithi112.stdout:mds.b smithi161 running (3m) 77s ago 3m 25.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 742d4f112ada 2024-04-25T18:46:30.900 INFO:teuthology.orchestra.run.smithi112.stdout:mds.c smithi171 running (3m) 77s ago 3m 18.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5e7c956c31a1 2024-04-25T18:46:30.900 INFO:teuthology.orchestra.run.smithi112.stdout:mds.d smithi112 running (4m) 46s ago 4m 13.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 aa1c2907f74f 2024-04-25T18:46:30.900 INFO:teuthology.orchestra.run.smithi112.stdout:mds.e smithi161 running (3m) 77s ago 3m 13.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 4b70ac893a33 2024-04-25T18:46:30.900 INFO:teuthology.orchestra.run.smithi112.stdout:mds.f smithi171 running (3m) 77s ago 3m 15.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 36603a081b1a 2024-04-25T18:46:30.900 INFO:teuthology.orchestra.run.smithi112.stdout:mds.g smithi112 running (3m) 46s ago 3m 19.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 0135fb6acc07 2024-04-25T18:46:30.900 INFO:teuthology.orchestra.run.smithi112.stdout:mds.h smithi161 running (3m) 77s ago 3m 18.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 929ece4abbcc 2024-04-25T18:46:30.900 INFO:teuthology.orchestra.run.smithi112.stdout:mds.i smithi171 running (3m) 77s ago 3m 21.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 87941493cbf9 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:mds.j smithi112 running (4m) 46s ago 4m 13.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8bec2ea38049 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:mds.k smithi161 running (4m) 77s ago 4m 16.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 fcfccf6165d1 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:mds.l smithi171 running (4m) 77s ago 4m 16.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a9264b1d0db3 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.x smithi112 *:9283,8765 running (12m) 46s ago 12m 515M - 19.0.0-3156-gd69f13d8 da4b004c09e8 0472f2a3d65a 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.y smithi161 *:8443,8765 running (10m) 77s ago 10m 455M - 19.0.0-3156-gd69f13d8 da4b004c09e8 37d97ce2f9f6 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.z smithi171 *:8443,8765 running (10m) 77s ago 10m 455M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1dd5024bb2eb 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:mon.a smithi112 running (12m) 46s ago 12m 49.1M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 8e659435d39e 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:mon.b smithi161 running (10m) 77s ago 10m 43.7M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 c70d9eb4feb4 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:mon.c smithi171 running (10m) 77s ago 10m 41.0M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 c5d70d8d788b 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (9m) 46s ago 9m 49.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 8be2293b0c13 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi161 running (9m) 77s ago 9m 57.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 fd7d5c6e9602 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi171 running (8m) 77s ago 8m 58.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 d4e856c3f038 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi112 running (8m) 46s ago 8m 54.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b302163a34c8 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi161 running (7m) 77s ago 7m 53.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 641e3ac32333 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi171 running (7m) 77s ago 7m 49.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 501f1344dd21 2024-04-25T18:46:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 smithi112 running (6m) 46s ago 6m 44.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b2ba727fb965 2024-04-25T18:46:30.902 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 smithi161 running (6m) 77s ago 6m 47.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c1f442d35b5a 2024-04-25T18:46:30.902 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 smithi171 running (6m) 77s ago 6m 49.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c34404916c5c 2024-04-25T18:46:30.902 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 smithi112 running (5m) 46s ago 5m 45.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 d659890e9c2a 2024-04-25T18:46:30.902 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 smithi161 running (5m) 77s ago 5m 54.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 060d4db3ff5a 2024-04-25T18:46:30.902 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 smithi171 running (4m) 77s ago 4m 52.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 68e61bfcdb05 2024-04-25T18:46:30.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.901+0000 7ff915600700 1 -- 172.21.15.112:0/1064024759 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff8f40a3cc0 msgr2=0x7ff8f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.901+0000 7ff915600700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff8f40a3cc0 0x7ff8f40a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0105c0 tx=0x7ff8fc00b480 comp rx=0 tx=0).stop 2024-04-25T18:46:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.901+0000 7ff915600700 1 -- 172.21.15.112:0/1064024759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9101035d0 msgr2=0x7ff91019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.901+0000 7ff915600700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9101035d0 0x7ff91019ae70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff90000ea20 tx=0x7ff90000ed30 comp rx=0 tx=0).stop 2024-04-25T18:46:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.901+0000 7ff915600700 1 -- 172.21.15.112:0/1064024759 shutdown_connections 2024-04-25T18:46:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.901+0000 7ff915600700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7ff8f40a3cc0 0x7ff8f40a6180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.901+0000 7ff915600700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff910108830 0x7ff9101a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.901+0000 7ff915600700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9101035d0 0x7ff91019ae70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.901+0000 7ff915600700 1 --2- 172.21.15.112:0/1064024759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff910105f00 0x7ff91019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.901+0000 7ff915600700 1 -- 172.21.15.112:0/1064024759 >> 172.21.15.112:0/1064024759 conn(0x7ff9100fcf40 msgr2=0x7ff910102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.901+0000 7ff915600700 1 -- 172.21.15.112:0/1064024759 shutdown_connections 2024-04-25T18:46:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:30.902+0000 7ff915600700 1 -- 172.21.15.112:0/1064024759 wait complete. 2024-04-25T18:46:31.829 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T18:46:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:32.660 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:32 smithi112 ceph-mon[27864]: from='client.24922 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:32.660 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:32 smithi112 ceph-mon[27864]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:32 smithi171 ceph-mon[33266]: from='client.24922 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:32 smithi171 ceph-mon[33266]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:32.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:32 smithi161 ceph-mon[32609]: from='client.24922 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:32.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:32 smithi161 ceph-mon[32609]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:33.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:33 smithi171 ceph-mon[33266]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:33.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:33 smithi161 ceph-mon[32609]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:33.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:33 smithi112 ceph-mon[27864]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.624+0000 7f8eeee00700 1 -- 172.21.15.112:0/3858078144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee8102ab0 msgr2=0x7f8ee8102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.624+0000 7f8eeee00700 1 --2- 172.21.15.112:0/3858078144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee8102ab0 0x7f8ee8102ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009fc0 tx=0x7f8ed800b330 comp rx=0 tx=0).stop 2024-04-25T18:46:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.625+0000 7f8eeee00700 1 -- 172.21.15.112:0/3858078144 shutdown_connections 2024-04-25T18:46:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.625+0000 7f8eeee00700 1 --2- 172.21.15.112:0/3858078144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8104650 0x7f8ee810af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.625+0000 7f8eeee00700 1 --2- 172.21.15.112:0/3858078144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee8102ab0 0x7f8ee8102ed0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.625+0000 7f8eeee00700 1 --2- 172.21.15.112:0/3858078144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee8103cb0 0x7f8ee8104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.625+0000 7f8eeee00700 1 -- 172.21.15.112:0/3858078144 >> 172.21.15.112:0/3858078144 conn(0x7f8ee80fe010 msgr2=0x7f8ee8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.625+0000 7f8eeee00700 1 -- 172.21.15.112:0/3858078144 shutdown_connections 2024-04-25T18:46:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.625+0000 7f8eeee00700 1 -- 172.21.15.112:0/3858078144 wait complete. 2024-04-25T18:46:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.626+0000 7f8eeee00700 1 Processor -- start 2024-04-25T18:46:34.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.626+0000 7f8eeee00700 1 -- start start 2024-04-25T18:46:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.626+0000 7f8eeee00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102ab0 0x7f8ee819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.626+0000 7f8eeee00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee8103cb0 0x7f8ee819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.626+0000 7f8eeee00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee8104650 0x7f8ee81a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.626+0000 7f8eeee00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee810e540 con 0x7f8ee8102ab0 2024-04-25T18:46:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.626+0000 7f8eeee00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee810e3c0 con 0x7f8ee8104650 2024-04-25T18:46:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.626+0000 7f8eeee00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee810e6c0 con 0x7f8ee8103cb0 2024-04-25T18:46:34.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.627+0000 7f8ee7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102ab0 0x7f8ee819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.627+0000 7f8edf400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee8103cb0 0x7f8ee819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.627+0000 7f8eece00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee8104650 0x7f8ee81a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.627+0000 7f8ee7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102ab0 0x7f8ee819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58714/0 (socket says 172.21.15.112:58714) 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.627+0000 7f8ee7e00700 1 -- 172.21.15.112:0/3734639969 learned_addr learned my addr 172.21.15.112:0/3734639969 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.627+0000 7f8edf400700 1 -- 172.21.15.112:0/3734639969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee8104650 msgr2=0x7f8ee81a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.627+0000 7f8edf400700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee8104650 0x7f8ee81a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.627+0000 7f8edf400700 1 -- 172.21.15.112:0/3734639969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102ab0 msgr2=0x7f8ee819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.627+0000 7f8edf400700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102ab0 0x7f8ee819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.627+0000 7f8edf400700 1 -- 172.21.15.112:0/3734639969 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee81a2a80 con 0x7f8ee8103cb0 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.627+0000 7f8edf400700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee8103cb0 0x7f8ee819b360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000ea20 tx=0x7f8ed000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.628+0000 7f8ee5600700 1 -- 172.21.15.112:0/3734639969 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed000cbd0 con 0x7f8ee8103cb0 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.628+0000 7f8ee5600700 1 -- 172.21.15.112:0/3734639969 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed000ee60 con 0x7f8ee8103cb0 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.628+0000 7f8eeee00700 1 -- 172.21.15.112:0/3734639969 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee81a2d70 con 0x7f8ee8103cb0 2024-04-25T18:46:34.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.628+0000 7f8ee5600700 1 -- 172.21.15.112:0/3734639969 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0005020 con 0x7f8ee8103cb0 2024-04-25T18:46:34.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.628+0000 7f8eeee00700 1 -- 172.21.15.112:0/3734639969 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee81a3250 con 0x7f8ee8103cb0 2024-04-25T18:46:34.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.629+0000 7f8eeee00700 1 -- 172.21.15.112:0/3734639969 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee8069730 con 0x7f8ee8103cb0 2024-04-25T18:46:34.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.631+0000 7f8ee5600700 1 -- 172.21.15.112:0/3734639969 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f8ed00051c0 con 0x7f8ee8103cb0 2024-04-25T18:46:34.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.632+0000 7f8ee5600700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8ecc0a3cc0 0x7f8ecc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:34.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.632+0000 7f8ee5600700 1 -- 172.21.15.112:0/3734639969 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f8ed00d03d0 con 0x7f8ee8103cb0 2024-04-25T18:46:34.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.632+0000 7f8ee7e00700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8ecc0a3cc0 0x7f8ecc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:34.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.633+0000 7f8ee7e00700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8ecc0a3cc0 0x7f8ecc0a6180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009f90 tx=0x7f8ed8007eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:34.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.635+0000 7f8ee5600700 1 -- 172.21.15.112:0/3734639969 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ed009cdc0 con 0x7f8ee8103cb0 2024-04-25T18:46:34.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.872+0000 7f8eeee00700 1 -- 172.21.15.112:0/3734639969 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f8ee8076390 con 0x7f8ecc0a3cc0 2024-04-25T18:46:34.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.894+0000 7f8ee5600700 1 -- 172.21.15.112:0/3734639969 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f8ee8076390 con 0x7f8ecc0a3cc0 2024-04-25T18:46:34.897 INFO:teuthology.orchestra.run.smithi112.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T18:46:34.897 INFO:teuthology.orchestra.run.smithi112.stdout:mds.all 12/12 81s ago 4m smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-25T18:46:34.897 INFO:teuthology.orchestra.run.smithi112.stdout:mgr 3/3 81s ago 10m smithi112=x;smithi161=y;smithi171=z;count:3 2024-04-25T18:46:34.897 INFO:teuthology.orchestra.run.smithi112.stdout:mon 3/3 81s ago 10m smithi112:172.21.15.112=a;smithi161:172.21.15.161=b;smithi171:172.21.15.171=c;count:3 2024-04-25T18:46:34.897 INFO:teuthology.orchestra.run.smithi112.stdout:osd 12 81s ago - 2024-04-25T18:46:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.897+0000 7f8eeee00700 1 -- 172.21.15.112:0/3734639969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8ecc0a3cc0 msgr2=0x7f8ecc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.897+0000 7f8eeee00700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8ecc0a3cc0 0x7f8ecc0a6180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009f90 tx=0x7f8ed8007eb0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.898+0000 7f8eeee00700 1 -- 172.21.15.112:0/3734639969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee8103cb0 msgr2=0x7f8ee819b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.898+0000 7f8eeee00700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee8103cb0 0x7f8ee819b360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000ea20 tx=0x7f8ed000ed30 comp rx=0 tx=0).stop 2024-04-25T18:46:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.898+0000 7f8eeee00700 1 -- 172.21.15.112:0/3734639969 shutdown_connections 2024-04-25T18:46:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.898+0000 7f8eeee00700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8ecc0a3cc0 0x7f8ecc0a6180 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.898+0000 7f8eeee00700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee8104650 0x7f8ee81a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.898+0000 7f8eeee00700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102ab0 0x7f8ee819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.898+0000 7f8eeee00700 1 --2- 172.21.15.112:0/3734639969 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee8103cb0 0x7f8ee819b360 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.898+0000 7f8eeee00700 1 -- 172.21.15.112:0/3734639969 >> 172.21.15.112:0/3734639969 conn(0x7f8ee80fe010 msgr2=0x7f8ee8074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:34.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.898+0000 7f8eeee00700 1 -- 172.21.15.112:0/3734639969 shutdown_connections 2024-04-25T18:46:34.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:34.898+0000 7f8eeee00700 1 -- 172.21.15.112:0/3734639969 wait complete. 2024-04-25T18:46:35.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T18:46:36.233 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:36.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:36 smithi112 ceph-mon[27864]: from='client.24928 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:36.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:36 smithi112 ceph-mon[27864]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:36.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:36 smithi171 ceph-mon[33266]: from='client.24928 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:36.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:36 smithi171 ceph-mon[33266]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:36.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:36 smithi161 ceph-mon[32609]: from='client.24928 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:36.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:36 smithi161 ceph-mon[32609]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:38.388 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:38 smithi112 ceph-mon[27864]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:38.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.555+0000 7fa5faa00700 1 -- 172.21.15.112:0/3860582486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5f4103cb0 msgr2=0x7fa5f4104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.555+0000 7fa5faa00700 1 --2- 172.21.15.112:0/3860582486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5f4103cb0 0x7fa5f4104110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00098f0 tx=0x7fa5e0009c00 comp rx=0 tx=0).stop 2024-04-25T18:46:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.555+0000 7fa5faa00700 1 -- 172.21.15.112:0/3860582486 shutdown_connections 2024-04-25T18:46:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.555+0000 7fa5faa00700 1 --2- 172.21.15.112:0/3860582486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5f4104650 0x7fa5f410af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.555+0000 7fa5faa00700 1 --2- 172.21.15.112:0/3860582486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f4102ab0 0x7fa5f4102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.555+0000 7fa5faa00700 1 --2- 172.21.15.112:0/3860582486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5f4103cb0 0x7fa5f4104110 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.555+0000 7fa5faa00700 1 -- 172.21.15.112:0/3860582486 >> 172.21.15.112:0/3860582486 conn(0x7fa5f40fe010 msgr2=0x7fa5f4100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.556+0000 7fa5faa00700 1 -- 172.21.15.112:0/3860582486 shutdown_connections 2024-04-25T18:46:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.556+0000 7fa5faa00700 1 -- 172.21.15.112:0/3860582486 wait complete. 2024-04-25T18:46:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.557+0000 7fa5faa00700 1 Processor -- start 2024-04-25T18:46:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.557+0000 7fa5faa00700 1 -- start start 2024-04-25T18:46:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5faa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5f4102ab0 0x7fa5f40767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5faa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f4103cb0 0x7fa5f4074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5faa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5f4104650 0x7fa5f4075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5faa00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f410e590 con 0x7fa5f4104650 2024-04-25T18:46:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5faa00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f410e410 con 0x7fa5f4103cb0 2024-04-25T18:46:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5faa00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f410e710 con 0x7fa5f4102ab0 2024-04-25T18:46:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f3e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5f4102ab0 0x7fa5f40767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f3e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5f4102ab0 0x7fa5f40767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:35350/0 (socket says 172.21.15.112:35350) 2024-04-25T18:46:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f3e00700 1 -- 172.21.15.112:0/369605701 learned_addr learned my addr 172.21.15.112:0/369605701 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f3400700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f4103cb0 0x7fa5f4074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f8a00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5f4104650 0x7fa5f4075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f3e00700 1 -- 172.21.15.112:0/369605701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f4103cb0 msgr2=0x7fa5f4074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f3e00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f4103cb0 0x7fa5f4074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f3e00700 1 -- 172.21.15.112:0/369605701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5f4104650 msgr2=0x7fa5f4075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f3e00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5f4104650 0x7fa5f4075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f3e00700 1 -- 172.21.15.112:0/369605701 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f4075c20 con 0x7fa5f4102ab0 2024-04-25T18:46:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f3400700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f4103cb0 0x7fa5f4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.558+0000 7fa5f8a00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5f4104650 0x7fa5f4075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.559+0000 7fa5f3e00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5f4102ab0 0x7fa5f40767e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e950 tx=0x7fa5e400ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.559+0000 7fa5f0c00700 1 -- 172.21.15.112:0/369605701 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4004230 con 0x7fa5f4102ab0 2024-04-25T18:46:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.559+0000 7fa5f0c00700 1 -- 172.21.15.112:0/369605701 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e40043d0 con 0x7fa5f4102ab0 2024-04-25T18:46:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.559+0000 7fa5f0c00700 1 -- 172.21.15.112:0/369605701 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4003d60 con 0x7fa5f4102ab0 2024-04-25T18:46:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.559+0000 7fa5faa00700 1 -- 172.21.15.112:0/369605701 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f41a2e70 con 0x7fa5f4102ab0 2024-04-25T18:46:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.559+0000 7fa5faa00700 1 -- 172.21.15.112:0/369605701 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f41a3210 con 0x7fa5f4102ab0 2024-04-25T18:46:38.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.560+0000 7fa5faa00700 1 -- 172.21.15.112:0/369605701 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f4069730 con 0x7fa5f4102ab0 2024-04-25T18:46:38.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.562+0000 7fa5f0c00700 1 -- 172.21.15.112:0/369605701 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fa5e400c7e0 con 0x7fa5f4102ab0 2024-04-25T18:46:38.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.564+0000 7fa5f0c00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa5d80a3c40 0x7fa5d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:38.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.564+0000 7fa5f0c00700 1 -- 172.21.15.112:0/369605701 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa5e40d06f0 con 0x7fa5f4102ab0 2024-04-25T18:46:38.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.564+0000 7fa5f3400700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa5d80a3c40 0x7fa5d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:38.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.565+0000 7fa5f3400700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa5d80a3c40 0x7fa5d80a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa5e000fb20 tx=0x7fa5e0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.566+0000 7fa5f0c00700 1 -- 172.21.15.112:0/369605701 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5e409d050 con 0x7fa5f4102ab0 2024-04-25T18:46:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:38 smithi171 ceph-mon[33266]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:38 smithi161 ceph-mon[32609]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:38.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.801+0000 7fa5faa00700 1 -- 172.21.15.112:0/369605701 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fa5f4062e80 con 0x7fa5d80a3c40 2024-04-25T18:46:38.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.804+0000 7fa5f0c00700 1 -- 172.21.15.112:0/369605701 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fa5f4062e80 con 0x7fa5d80a3c40 2024-04-25T18:46:38.806 INFO:teuthology.orchestra.run.smithi112.stdout:HOST ADDR LABELS STATUS 2024-04-25T18:46:38.807 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 172.21.15.112 2024-04-25T18:46:38.807 INFO:teuthology.orchestra.run.smithi112.stdout:smithi161 172.21.15.161 2024-04-25T18:46:38.807 INFO:teuthology.orchestra.run.smithi112.stdout:smithi171 172.21.15.171 2024-04-25T18:46:38.807 INFO:teuthology.orchestra.run.smithi112.stdout:3 hosts in cluster 2024-04-25T18:46:38.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.807+0000 7fa5faa00700 1 -- 172.21.15.112:0/369605701 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa5d80a3c40 msgr2=0x7fa5d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:38.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.807+0000 7fa5faa00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa5d80a3c40 0x7fa5d80a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa5e000fb20 tx=0x7fa5e0019040 comp rx=0 tx=0).stop 2024-04-25T18:46:38.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.807+0000 7fa5faa00700 1 -- 172.21.15.112:0/369605701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5f4102ab0 msgr2=0x7fa5f40767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:38.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.807+0000 7fa5faa00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5f4102ab0 0x7fa5f40767e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e950 tx=0x7fa5e400ec60 comp rx=0 tx=0).stop 2024-04-25T18:46:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.807+0000 7fa5faa00700 1 -- 172.21.15.112:0/369605701 shutdown_connections 2024-04-25T18:46:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.807+0000 7fa5faa00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa5d80a3c40 0x7fa5d80a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.807+0000 7fa5faa00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5f4104650 0x7fa5f4075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.807+0000 7fa5faa00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5f4102ab0 0x7fa5f40767e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.807+0000 7fa5faa00700 1 --2- 172.21.15.112:0/369605701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f4103cb0 0x7fa5f4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.807+0000 7fa5faa00700 1 -- 172.21.15.112:0/369605701 >> 172.21.15.112:0/369605701 conn(0x7fa5f40fe010 msgr2=0x7fa5f4107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.807+0000 7fa5faa00700 1 -- 172.21.15.112:0/369605701 shutdown_connections 2024-04-25T18:46:38.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:38.808+0000 7fa5faa00700 1 -- 172.21.15.112:0/369605701 wait complete. 2024-04-25T18:46:39.712 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T18:46:40.156 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:40.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:40 smithi112 ceph-mon[27864]: from='client.24934 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:40.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:40 smithi112 ceph-mon[27864]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:46:40.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:40 smithi171 ceph-mon[33266]: from='client.24934 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:40.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:40 smithi171 ceph-mon[33266]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:46:40.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:40 smithi161 ceph-mon[32609]: from='client.24934 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:40.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:40 smithi161 ceph-mon[32609]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:46:42.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.471+0000 7f8de1200700 1 -- 172.21.15.112:0/2661398661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ddc105de0 msgr2=0x7f8ddc106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:42.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.471+0000 7f8de1200700 1 --2- 172.21.15.112:0/2661398661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ddc105de0 0x7f8ddc106240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40099c0 tx=0x7f8dc4009cd0 comp rx=0 tx=0).stop 2024-04-25T18:46:42.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.471+0000 7f8de1200700 1 -- 172.21.15.112:0/2661398661 shutdown_connections 2024-04-25T18:46:42.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.471+0000 7f8de1200700 1 --2- 172.21.15.112:0/2661398661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ddc106780 0x7f8ddc10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:42.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.471+0000 7f8de1200700 1 --2- 172.21.15.112:0/2661398661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ddc104be0 0x7f8ddc105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:42.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.471+0000 7f8de1200700 1 --2- 172.21.15.112:0/2661398661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ddc105de0 0x7f8ddc106240 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:42.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.471+0000 7f8de1200700 1 -- 172.21.15.112:0/2661398661 >> 172.21.15.112:0/2661398661 conn(0x7f8ddc100180 msgr2=0x7f8ddc1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:42.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.471+0000 7f8de1200700 1 -- 172.21.15.112:0/2661398661 shutdown_connections 2024-04-25T18:46:42.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.471+0000 7f8de1200700 1 -- 172.21.15.112:0/2661398661 wait complete. 2024-04-25T18:46:42.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.472+0000 7f8de1200700 1 Processor -- start 2024-04-25T18:46:42.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.472+0000 7f8de1200700 1 -- start start 2024-04-25T18:46:42.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.472+0000 7f8de1200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ddc104be0 0x7f8ddc19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:42.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.472+0000 7f8de1200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ddc105de0 0x7f8ddc19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:42.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.472+0000 7f8de1200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ddc106780 0x7f8ddc1a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:42.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.472+0000 7f8de1200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc1106a0 con 0x7f8ddc104be0 2024-04-25T18:46:42.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.472+0000 7f8de1200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc110520 con 0x7f8ddc105de0 2024-04-25T18:46:42.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.472+0000 7f8de1200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc110820 con 0x7f8ddc106780 2024-04-25T18:46:42.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.473+0000 7f8ddb400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ddc104be0 0x7f8ddc19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:42.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.473+0000 7f8ddb400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ddc104be0 0x7f8ddc19ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44546/0 (socket says 172.21.15.112:44546) 2024-04-25T18:46:42.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.473+0000 7f8ddb400700 1 -- 172.21.15.112:0/3151473155 learned_addr learned my addr 172.21.15.112:0/3151473155 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:42.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.473+0000 7f8dd3e00700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ddc105de0 0x7f8ddc19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:42.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.473+0000 7f8ddbe00700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ddc106780 0x7f8ddc1a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:42.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.473+0000 7f8ddb400700 1 -- 172.21.15.112:0/3151473155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ddc106780 msgr2=0x7f8ddc1a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:42.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.473+0000 7f8ddb400700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ddc106780 0x7f8ddc1a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:42.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.473+0000 7f8ddb400700 1 -- 172.21.15.112:0/3151473155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ddc105de0 msgr2=0x7f8ddc19b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:42.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.473+0000 7f8ddb400700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ddc105de0 0x7f8ddc19b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:42.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.473+0000 7f8ddb400700 1 -- 172.21.15.112:0/3151473155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ddc1a2950 con 0x7f8ddc104be0 2024-04-25T18:46:42.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.474+0000 7f8ddb400700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ddc104be0 0x7f8ddc19ad00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00e970 tx=0x7f8dcc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:42.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.474+0000 7f8dd8c00700 1 -- 172.21.15.112:0/3151473155 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc004220 con 0x7f8ddc104be0 2024-04-25T18:46:42.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.474+0000 7f8dd8c00700 1 -- 172.21.15.112:0/3151473155 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dcc0043c0 con 0x7f8ddc104be0 2024-04-25T18:46:42.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.474+0000 7f8de1200700 1 -- 172.21.15.112:0/3151473155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ddc1a2c40 con 0x7f8ddc104be0 2024-04-25T18:46:42.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.474+0000 7f8dd8c00700 1 -- 172.21.15.112:0/3151473155 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc003ce0 con 0x7f8ddc104be0 2024-04-25T18:46:42.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.474+0000 7f8de1200700 1 -- 172.21.15.112:0/3151473155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ddc1a3180 con 0x7f8ddc104be0 2024-04-25T18:46:42.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.475+0000 7f8de1200700 1 -- 172.21.15.112:0/3151473155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ddc069730 con 0x7f8ddc104be0 2024-04-25T18:46:42.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.477+0000 7f8dd8c00700 1 -- 172.21.15.112:0/3151473155 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f8dcc00c7e0 con 0x7f8ddc104be0 2024-04-25T18:46:42.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.478+0000 7f8dd8c00700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8dc00a3c40 0x7f8dc00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:42.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.479+0000 7f8dd8c00700 1 -- 172.21.15.112:0/3151473155 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f8dcc0d0690 con 0x7f8ddc104be0 2024-04-25T18:46:42.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.479+0000 7f8dd3e00700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8dc00a3c40 0x7f8dc00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:42.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.480+0000 7f8dd3e00700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8dc00a3c40 0x7f8dc00a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4000c00 tx=0x7f8dc4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.483+0000 7f8dd8c00700 1 -- 172.21.15.112:0/3151473155 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8dcc09cff0 con 0x7f8ddc104be0 2024-04-25T18:46:42.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:42 smithi171 ceph-mon[33266]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:42.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:42 smithi161 ceph-mon[32609]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:42.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.719+0000 7f8de1200700 1 -- 172.21.15.112:0/3151473155 --> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8ddc062e80 con 0x7f8dc00a3c40 2024-04-25T18:46:42.722 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:42 smithi112 ceph-mon[27864]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:42.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.724+0000 7f8dd8c00700 1 -- 172.21.15.112:0/3151473155 <== mgr.14152 v2:172.21.15.112:6800/605196912 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8ddc062e80 con 0x7f8dc00a3c40 2024-04-25T18:46:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T18:46:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-04-25T18:46:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-04-25T18:46:42.726 INFO:teuthology.orchestra.run.smithi112.stdout:smithi171 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001S400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.726+0000 7f8de1200700 1 -- 172.21.15.112:0/3151473155 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8dc00a3c40 msgr2=0x7f8dc00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.726+0000 7f8de1200700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8dc00a3c40 0x7f8dc00a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4000c00 tx=0x7f8dc4005fb0 comp rx=0 tx=0).stop 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.726+0000 7f8de1200700 1 -- 172.21.15.112:0/3151473155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ddc104be0 msgr2=0x7f8ddc19ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.726+0000 7f8de1200700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ddc104be0 0x7f8ddc19ad00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00e970 tx=0x7f8dcc00ec80 comp rx=0 tx=0).stop 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.727+0000 7f8de1200700 1 -- 172.21.15.112:0/3151473155 shutdown_connections 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.727+0000 7f8de1200700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f8dc00a3c40 0x7f8dc00a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.727+0000 7f8de1200700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ddc106780 0x7f8ddc1a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.727+0000 7f8de1200700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ddc104be0 0x7f8ddc19ad00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.727+0000 7f8de1200700 1 --2- 172.21.15.112:0/3151473155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ddc105de0 0x7f8ddc19b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.727+0000 7f8de1200700 1 -- 172.21.15.112:0/3151473155 >> 172.21.15.112:0/3151473155 conn(0x7f8ddc100180 msgr2=0x7f8ddc1090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.727+0000 7f8de1200700 1 -- 172.21.15.112:0/3151473155 shutdown_connections 2024-04-25T18:46:42.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:42.727+0000 7f8de1200700 1 -- 172.21.15.112:0/3151473155 wait complete. 2024-04-25T18:46:43.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:43 smithi112 ceph-mon[27864]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:43.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:43 smithi112 ceph-mon[27864]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:43.629 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T18:46:43.637 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T18:46:43.637 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T18:46:43.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:43 smithi171 ceph-mon[33266]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:43.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:43 smithi171 ceph-mon[33266]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:43.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:43 smithi161 ceph-mon[32609]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:43.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:43 smithi161 ceph-mon[32609]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:44.071 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:46.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.352+0000 7f29c1600700 1 -- 172.21.15.112:0/98907402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29bc103cb0 msgr2=0x7f29bc104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:46.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.352+0000 7f29c1600700 1 --2- 172.21.15.112:0/98907402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29bc103cb0 0x7f29bc104110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f29a80099c0 tx=0x7f29a8009cd0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.352+0000 7f29c1600700 1 -- 172.21.15.112:0/98907402 shutdown_connections 2024-04-25T18:46:46.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.352+0000 7f29c1600700 1 --2- 172.21.15.112:0/98907402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29bc104650 0x7f29bc10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.352+0000 7f29c1600700 1 --2- 172.21.15.112:0/98907402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc102ab0 0x7f29bc102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.352+0000 7f29c1600700 1 --2- 172.21.15.112:0/98907402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29bc103cb0 0x7f29bc104110 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.352+0000 7f29c1600700 1 -- 172.21.15.112:0/98907402 >> 172.21.15.112:0/98907402 conn(0x7f29bc0fe010 msgr2=0x7f29bc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:46.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.352+0000 7f29c1600700 1 -- 172.21.15.112:0/98907402 shutdown_connections 2024-04-25T18:46:46.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.352+0000 7f29c1600700 1 -- 172.21.15.112:0/98907402 wait complete. 2024-04-25T18:46:46.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.353+0000 7f29c1600700 1 Processor -- start 2024-04-25T18:46:46.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.353+0000 7f29c1600700 1 -- start start 2024-04-25T18:46:46.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29c1600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29bc102ab0 0x7f29bc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:46.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29c1600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29bc103cb0 0x7f29bc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:46.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29c1600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc104650 0x7f29bc1a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:46.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29c1600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc10e590 con 0x7f29bc102ab0 2024-04-25T18:46:46.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29c1600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc10e410 con 0x7f29bc103cb0 2024-04-25T18:46:46.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29c1600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc10e710 con 0x7f29bc104650 2024-04-25T18:46:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29ba000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29bc103cb0 0x7f29bc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29baa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29bc102ab0 0x7f29bc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29bb400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc104650 0x7f29bc1a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29ba000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29bc103cb0 0x7f29bc19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:42324/0 (socket says 172.21.15.112:42324) 2024-04-25T18:46:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29bb400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc104650 0x7f29bc1a23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:48474/0 (socket says 172.21.15.112:48474) 2024-04-25T18:46:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29ba000700 1 -- 172.21.15.112:0/1697344511 learned_addr learned my addr 172.21.15.112:0/1697344511 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29ba000700 1 -- 172.21.15.112:0/1697344511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc104650 msgr2=0x7f29bc1a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29ba000700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc104650 0x7f29bc1a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29ba000700 1 -- 172.21.15.112:0/1697344511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29bc102ab0 msgr2=0x7f29bc19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29ba000700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29bc102ab0 0x7f29bc19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.354+0000 7f29ba000700 1 -- 172.21.15.112:0/1697344511 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc1a2a70 con 0x7f29bc103cb0 2024-04-25T18:46:46.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.355+0000 7f29ba000700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29bc103cb0 0x7f29bc19b350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009990 tx=0x7f29a800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:46.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.355+0000 7f29b3400700 1 -- 172.21.15.112:0/1697344511 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8009e60 con 0x7f29bc103cb0 2024-04-25T18:46:46.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.355+0000 7f29b3400700 1 -- 172.21.15.112:0/1697344511 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a80038c0 con 0x7f29bc103cb0 2024-04-25T18:46:46.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.355+0000 7f29c1600700 1 -- 172.21.15.112:0/1697344511 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc1a2d00 con 0x7f29bc103cb0 2024-04-25T18:46:46.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.355+0000 7f29b3400700 1 -- 172.21.15.112:0/1697344511 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8005670 con 0x7f29bc103cb0 2024-04-25T18:46:46.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.355+0000 7f29c1600700 1 -- 172.21.15.112:0/1697344511 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc1a31b0 con 0x7f29bc103cb0 2024-04-25T18:46:46.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.356+0000 7f29c1600700 1 -- 172.21.15.112:0/1697344511 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc069730 con 0x7f29bc103cb0 2024-04-25T18:46:46.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.358+0000 7f29b3400700 1 -- 172.21.15.112:0/1697344511 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f29a800fb00 con 0x7f29bc103cb0 2024-04-25T18:46:46.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.360+0000 7f29b3400700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f29a00a3cc0 0x7f29a00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:46.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.360+0000 7f29b3400700 1 -- 172.21.15.112:0/1697344511 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f29a80d1130 con 0x7f29bc103cb0 2024-04-25T18:46:46.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.360+0000 7f29baa00700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f29a00a3cc0 0x7f29a00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:46.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.361+0000 7f29baa00700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f29a00a3cc0 0x7f29a00a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f29bc103b10 tx=0x7f29ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:46.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.364+0000 7f29b3400700 1 -- 172.21.15.112:0/1697344511 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29a8013070 con 0x7f29bc103cb0 2024-04-25T18:46:46.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:46 smithi112 ceph-mon[27864]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:46.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.642+0000 7f29c1600700 1 -- 172.21.15.112:0/1697344511 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f29bc1a33d0 con 0x7f29bc103cb0 2024-04-25T18:46:46.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.642+0000 7f29b3400700 1 -- 172.21.15.112:0/1697344511 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3066 (secure 0 0 0) 0x7f29a809db20 con 0x7f29bc103cb0 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:e25 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 25 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-04-25T18:43:19.347467+0000 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-04-25T18:43:45.212327+0000 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-04-25T18:46:46.645 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 300 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 0 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 3 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1,2 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=24512,1=24511,2=24493} 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-04-25T18:46:46.646 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data disabled 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 0 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 24512 members: 24511,24512,24493 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b{0:24512} state up:active seq 7 addr [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.g{0:24488} state up:standby-replay seq 1 addr [v2:172.21.15.112:6838/3241331981,v1:172.21.15.112:6839/3241331981] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a{1:24511} state up:active seq 12 addr [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c{1:24478} state up:standby-replay seq 1 addr [v2:172.21.15.171:6836/3522712423,v1:172.21.15.171:6837/3522712423] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.i{2:24493} state up:active seq 15 addr [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.h{2:24476} state up:standby-replay seq 1 addr [v2:172.21.15.161:6836/4007863322,v1:172.21.15.161:6837/4007863322] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:46:46.647 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-04-25T18:46:46.648 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:46:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.l{-1:24427} state up:standby seq 1 addr [v2:172.21.15.171:6832/1467985525,v1:172.21.15.171:6833/1467985525] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.k{-1:24431} state up:standby seq 1 addr [v2:172.21.15.161:6832/3214695167,v1:172.21.15.161:6833/3214695167] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.j{-1:24443} state up:standby seq 1 addr [v2:172.21.15.112:6834/139408672,v1:172.21.15.112:6835/139408672] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d{-1:24454} state up:standby seq 1 addr [v2:172.21.15.112:6836/2558651587,v1:172.21.15.112:6837/2558651587] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.e{-1:24461} state up:standby seq 1 addr [v2:172.21.15.161:6834/3751012536,v1:172.21.15.161:6835/3751012536] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.648 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.f{-1:24466} state up:standby seq 1 addr [v2:172.21.15.171:6834/2423338358,v1:172.21.15.171:6835/2423338358] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:46:46.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 -- 172.21.15.112:0/1697344511 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f29a00a3cc0 msgr2=0x7f29a00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:46.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f29a00a3cc0 0x7f29a00a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f29bc103b10 tx=0x7f29ac009260 comp rx=0 tx=0).stop 2024-04-25T18:46:46.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 -- 172.21.15.112:0/1697344511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29bc103cb0 msgr2=0x7f29bc19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:46.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29bc103cb0 0x7f29bc19b350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009990 tx=0x7f29a800f690 comp rx=0 tx=0).stop 2024-04-25T18:46:46.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 -- 172.21.15.112:0/1697344511 shutdown_connections 2024-04-25T18:46:46.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f29a00a3cc0 0x7f29a00a6180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29bc104650 0x7f29bc1a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29bc102ab0 0x7f29bc19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 --2- 172.21.15.112:0/1697344511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29bc103cb0 0x7f29bc19b350 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 -- 172.21.15.112:0/1697344511 >> 172.21.15.112:0/1697344511 conn(0x7f29bc0fe010 msgr2=0x7f29bc074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:46.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 -- 172.21.15.112:0/1697344511 shutdown_connections 2024-04-25T18:46:46.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:46.647+0000 7f29c1600700 1 -- 172.21.15.112:0/1697344511 wait complete. 2024-04-25T18:46:46.650 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-04-25T18:46:46.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:46 smithi171 ceph-mon[33266]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:46.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:46 smithi161 ceph-mon[32609]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:47.442 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:46:47.442 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:47 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1697344511' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T18:46:47.442 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.442 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.442 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.442 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:47 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.486 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T18:46:47.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:46:47.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:47 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1697344511' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T18:46:47.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:47 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:46:47.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:47 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1697344511' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T18:46:47.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:47 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:47.928 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:48.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:48 smithi112 ceph-mon[27864]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:48.399 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:46:48.399 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:46:48.399 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:48 smithi112 ceph-mon[27864]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:48.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:48 smithi171 ceph-mon[33266]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:48.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:46:48.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:46:48.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:48 smithi171 ceph-mon[33266]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:48.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:48 smithi161 ceph-mon[32609]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:48.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:46:48.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:46:48.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:48 smithi161 ceph-mon[32609]: from='mgr.14152 172.21.15.112:0/3149724266' entity='mgr.x' 2024-04-25T18:46:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.104+0000 7fe003e00700 1 -- 172.21.15.112:0/3471368397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe004104be0 msgr2=0x7fe004105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.104+0000 7fe003e00700 1 --2- 172.21.15.112:0/3471368397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe004104be0 0x7fe004105000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdff40099c0 tx=0x7fdff4009cd0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.104+0000 7fe003e00700 1 -- 172.21.15.112:0/3471368397 shutdown_connections 2024-04-25T18:46:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.104+0000 7fe003e00700 1 --2- 172.21.15.112:0/3471368397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe004106780 0x7fe00410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.104+0000 7fe003e00700 1 --2- 172.21.15.112:0/3471368397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe004104be0 0x7fe004105000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.104+0000 7fe003e00700 1 --2- 172.21.15.112:0/3471368397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe004105de0 0x7fe004106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.104+0000 7fe003e00700 1 -- 172.21.15.112:0/3471368397 >> 172.21.15.112:0/3471368397 conn(0x7fe004100160 msgr2=0x7fe0041025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:50.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.104+0000 7fe003e00700 1 -- 172.21.15.112:0/3471368397 shutdown_connections 2024-04-25T18:46:50.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.105+0000 7fe003e00700 1 -- 172.21.15.112:0/3471368397 wait complete. 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.106+0000 7fe003e00700 1 Processor -- start 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.106+0000 7fe003e00700 1 -- start start 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.106+0000 7fe003e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe004104be0 0x7fe00419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.106+0000 7fe003e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe004105de0 0x7fe00419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.106+0000 7fe003e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe004106780 0x7fe0041a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.106+0000 7fe003e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041106c0 con 0x7fe004105de0 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.106+0000 7fe003e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe004110540 con 0x7fe004104be0 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.106+0000 7fe003e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe004110840 con 0x7fe004106780 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.106+0000 7fe002000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe004105de0 0x7fe00419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.107+0000 7fe002000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe004105de0 0x7fe00419b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53378/0 (socket says 172.21.15.112:53378) 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.107+0000 7fe002000700 1 -- 172.21.15.112:0/1682776263 learned_addr learned my addr 172.21.15.112:0/1682776263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.107+0000 7fe002a00700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe004104be0 0x7fe00419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.107+0000 7fe003400700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe004106780 0x7fe0041a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.107+0000 7fe002a00700 1 -- 172.21.15.112:0/1682776263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe004106780 msgr2=0x7fe0041a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.107+0000 7fe002a00700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe004106780 0x7fe0041a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.107+0000 7fe002a00700 1 -- 172.21.15.112:0/1682776263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe004105de0 msgr2=0x7fe00419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.107+0000 7fe002a00700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe004105de0 0x7fe00419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.107+0000 7fe002a00700 1 -- 172.21.15.112:0/1682776263 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0041a29a0 con 0x7fe004104be0 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.107+0000 7fe002a00700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe004104be0 0x7fe00419ad40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdff40098e0 tx=0x7fdff4003af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.108+0000 7fdffb400700 1 -- 172.21.15.112:0/1682776263 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40050c0 con 0x7fe004104be0 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.108+0000 7fdffb400700 1 -- 172.21.15.112:0/1682776263 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdff4022460 con 0x7fe004104be0 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.108+0000 7fe003e00700 1 -- 172.21.15.112:0/1682776263 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0041a2c30 con 0x7fe004104be0 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.108+0000 7fdffb400700 1 -- 172.21.15.112:0/1682776263 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff400f6e0 con 0x7fe004104be0 2024-04-25T18:46:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.108+0000 7fe003e00700 1 -- 172.21.15.112:0/1682776263 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0041a3080 con 0x7fe004104be0 2024-04-25T18:46:50.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.109+0000 7fe003e00700 1 -- 172.21.15.112:0/1682776263 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe004069730 con 0x7fe004104be0 2024-04-25T18:46:50.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.111+0000 7fdffb400700 1 -- 172.21.15.112:0/1682776263 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fdff4022610 con 0x7fe004104be0 2024-04-25T18:46:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.112+0000 7fdffb400700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdfe80a3c40 0x7fdfe80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.112+0000 7fdffb400700 1 -- 172.21.15.112:0/1682776263 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fdff40d10c0 con 0x7fe004104be0 2024-04-25T18:46:50.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.113+0000 7fe002000700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdfe80a3c40 0x7fdfe80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:50.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.113+0000 7fe002000700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdfe80a3c40 0x7fdfe80a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009d60 tx=0x7fdff0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:50.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.114+0000 7fdffb400700 1 -- 172.21.15.112:0/1682776263 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdff405dff0 con 0x7fe004104be0 2024-04-25T18:46:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.344+0000 7fe003e00700 1 -- 172.21.15.112:0/1682776263 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe00402e0a0 con 0x7fe004104be0 2024-04-25T18:46:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.345+0000 7fdffb400700 1 -- 172.21.15.112:0/1682776263 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5346 (secure 0 0 0) 0x7fdff4013070 con 0x7fe004104be0 2024-04-25T18:46:50.348 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 94 2024-04-25T18:46:50.348 INFO:teuthology.orchestra.run.smithi112.stdout:fsid 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T18:46:50.348 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-04-25T18:33:37.379089+0000 2024-04-25T18:46:50.348 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-04-25T18:43:56.620122+0000 2024-04-25T18:46:50.348 INFO:teuthology.orchestra.run.smithi112.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T18:46:50.348 INFO:teuthology.orchestra.run.smithi112.stdout:crush_version 26 2024-04-25T18:46:50.348 INFO:teuthology.orchestra.run.smithi112.stdout:full_ratio 0.95 2024-04-25T18:46:50.348 INFO:teuthology.orchestra.run.smithi112.stdout:backfillfull_ratio 0.9 2024-04-25T18:46:50.348 INFO:teuthology.orchestra.run.smithi112.stdout:nearfull_ratio 0.85 2024-04-25T18:46:50.348 INFO:teuthology.orchestra.run.smithi112.stdout:require_min_compat_client luminous 2024-04-25T18:46:50.348 INFO:teuthology.orchestra.run.smithi112.stdout:min_compat_client luminous 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:require_osd_release squid 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:stretch_mode_enabled false 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 92 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 92 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:max_osd 12 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6802/232254535,v1:172.21.15.112:6803/232254535] [v2:172.21.15.112:6804/232254535,v1:172.21.15.112:6805/232254535] exists,up efb4bc01-e29b-4402-8a7a-3387d1731ec4 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/664224744,v1:172.21.15.161:6801/664224744] [v2:172.21.15.161:6802/664224744,v1:172.21.15.161:6803/664224744] exists,up 65a83fa2-96a6-4743-8c17-21531f5e885e 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/345172073,v1:172.21.15.171:6801/345172073] [v2:172.21.15.171:6802/345172073,v1:172.21.15.171:6803/345172073] exists,up 7562ec4c-c126-48ec-a51c-56fa461a2e05 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 up in weight 1 up_from 27 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6810/1472407156,v1:172.21.15.112:6811/1472407156] [v2:172.21.15.112:6812/1472407156,v1:172.21.15.112:6813/1472407156] exists,up d7aae8cd-fec7-46d9-8773-25eaaca3cc78 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 up in weight 1 up_from 35 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/373204306,v1:172.21.15.161:6809/373204306] [v2:172.21.15.161:6810/373204306,v1:172.21.15.161:6811/373204306] exists,up b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 up in weight 1 up_from 40 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/2643512313,v1:172.21.15.171:6809/2643512313] [v2:172.21.15.171:6810/2643512313,v1:172.21.15.171:6811/2643512313] exists,up 56124d12-bc1f-4cba-8430-938248851776 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 up in weight 1 up_from 48 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6818/1933654933,v1:172.21.15.112:6819/1933654933] [v2:172.21.15.112:6820/1933654933,v1:172.21.15.112:6821/1933654933] exists,up 9563a9d3-352b-4686-975c-282ec167248b 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 up in weight 1 up_from 54 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/3060201138,v1:172.21.15.161:6817/3060201138] [v2:172.21.15.161:6818/3060201138,v1:172.21.15.161:6819/3060201138] exists,up f2d06e86-6aac-40f4-956e-18fe0b0e3543 2024-04-25T18:46:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 up in weight 1 up_from 62 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/835756571,v1:172.21.15.171:6817/835756571] [v2:172.21.15.171:6818/835756571,v1:172.21.15.171:6819/835756571] exists,up f006e45f-2bb9-45d0-9020-be7c33c6a825 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 up in weight 1 up_from 70 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6826/349758528,v1:172.21.15.112:6827/349758528] [v2:172.21.15.112:6828/349758528,v1:172.21.15.112:6829/349758528] exists,up c5a96491-4151-4633-b3bc-2c214fe51c43 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 up in weight 1 up_from 76 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/3013292765,v1:172.21.15.161:6825/3013292765] [v2:172.21.15.161:6826/3013292765,v1:172.21.15.161:6827/3013292765] exists,up 244eda3e-25a0-490e-879f-c1e2726c1be5 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 up in weight 1 up_from 84 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/1010336458,v1:172.21.15.171:6825/1010336458] [v2:172.21.15.171:6826/1010336458,v1:172.21.15.171:6827/1010336458] exists,up 356e5c54-bbc3-45ee-bda2-85164ce3cb92 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.0 [8,4] 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.6 [8,11] 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.c [2,4] 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.12 [5,11] 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.1d [2,1] 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.3 [4,9] 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.13 [4,9] 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.18 [3,9] 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.39 [5,6] 2024-04-25T18:46:50.350 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.3a [4,8] 2024-04-25T18:46:50.351 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/733062511 expires 2024-04-26T18:34:54.948120+0000 2024-04-25T18:46:50.351 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/733062511 expires 2024-04-26T18:34:54.948120+0000 2024-04-25T18:46:50.351 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/181862246 expires 2024-04-26T18:34:54.948120+0000 2024-04-25T18:46:50.351 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3393454453 expires 2024-04-26T18:34:20.605775+0000 2024-04-25T18:46:50.351 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3195850527 expires 2024-04-26T18:34:20.605775+0000 2024-04-25T18:46:50.351 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/3468460974 expires 2024-04-26T18:34:20.605775+0000 2024-04-25T18:46:50.351 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3392048544 expires 2024-04-26T18:34:20.605775+0000 2024-04-25T18:46:50.351 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1329430933 expires 2024-04-26T18:34:54.948120+0000 2024-04-25T18:46:50.351 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/735578423 expires 2024-04-26T18:34:54.948120+0000 2024-04-25T18:46:50.351 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/3468460974 expires 2024-04-26T18:34:20.605775+0000 2024-04-25T18:46:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 -- 172.21.15.112:0/1682776263 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdfe80a3c40 msgr2=0x7fdfe80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdfe80a3c40 0x7fdfe80a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009d60 tx=0x7fdff0009260 comp rx=0 tx=0).stop 2024-04-25T18:46:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 -- 172.21.15.112:0/1682776263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe004104be0 msgr2=0x7fe00419ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe004104be0 0x7fe00419ad40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdff40098e0 tx=0x7fdff4003af0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 -- 172.21.15.112:0/1682776263 shutdown_connections 2024-04-25T18:46:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fdfe80a3c40 0x7fdfe80a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe004106780 0x7fe0041a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe004104be0 0x7fe00419ad40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 --2- 172.21.15.112:0/1682776263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe004105de0 0x7fe00419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 -- 172.21.15.112:0/1682776263 >> 172.21.15.112:0/1682776263 conn(0x7fe004100160 msgr2=0x7fe004109100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:50.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 -- 172.21.15.112:0/1682776263 shutdown_connections 2024-04-25T18:46:50.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:50.349+0000 7fe003e00700 1 -- 172.21.15.112:0/1682776263 wait complete. 2024-04-25T18:46:50.361 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:50 smithi112 ceph-mon[27864]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:46:50.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:50 smithi161 ceph-mon[32609]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:46:50.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:50 smithi171 ceph-mon[33266]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:46:51.221 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T18:46:51.233 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:46:51.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:51 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1682776263' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T18:46:51.680 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:51.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:51 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1682776263' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T18:46:51.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:51 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1682776263' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T18:46:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:52 smithi112 ceph-mon[27864]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:46:52.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:52 smithi171 ceph-mon[33266]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:46:52.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:52 smithi161 ceph-mon[32609]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:46:53.377 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:53 smithi112 ceph-mon[27864]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:53.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:53 smithi171 ceph-mon[33266]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:53.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:53 smithi161 ceph-mon[32609]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:46:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.904+0000 7faafca00700 1 -- 172.21.15.112:0/3627979220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8076f70 msgr2=0x7faaf8077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.904+0000 7faafca00700 1 --2- 172.21.15.112:0/3627979220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8076f70 0x7faaf8077350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faae8009990 tx=0x7faae8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:46:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.905+0000 7faafca00700 1 -- 172.21.15.112:0/3627979220 shutdown_connections 2024-04-25T18:46:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.905+0000 7faafca00700 1 --2- 172.21.15.112:0/3627979220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaf81137f0 0x7faaf8115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.905+0000 7faafca00700 1 --2- 172.21.15.112:0/3627979220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8076f70 0x7faaf8077350 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.905+0000 7faafca00700 1 --2- 172.21.15.112:0/3627979220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaf8077890 0x7faaf81132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.905+0000 7faafca00700 1 -- 172.21.15.112:0/3627979220 >> 172.21.15.112:0/3627979220 conn(0x7faaf8100680 msgr2=0x7faaf8102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.905+0000 7faafca00700 1 -- 172.21.15.112:0/3627979220 shutdown_connections 2024-04-25T18:46:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.905+0000 7faafca00700 1 -- 172.21.15.112:0/3627979220 wait complete. 2024-04-25T18:46:53.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.906+0000 7faafca00700 1 Processor -- start 2024-04-25T18:46:53.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.906+0000 7faafca00700 1 -- start start 2024-04-25T18:46:53.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.906+0000 7faafca00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8076f70 0x7faaf819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:53.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.906+0000 7faafca00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaf8077890 0x7faaf819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.906+0000 7faafca00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaf81137f0 0x7faaf81a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.906+0000 7faafca00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faaf81191b0 con 0x7faaf8076f70 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.906+0000 7faafca00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faaf8119030 con 0x7faaf8077890 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.906+0000 7faafca00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faaf8119330 con 0x7faaf81137f0 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.907+0000 7faaf6800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8076f70 0x7faaf819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.907+0000 7faaf6800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8076f70 0x7faaf819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53382/0 (socket says 172.21.15.112:53382) 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.907+0000 7faaf6800700 1 -- 172.21.15.112:0/1782099607 learned_addr learned my addr 172.21.15.112:0/1782099607 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.907+0000 7faaefe00700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaf8077890 0x7faaf819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.907+0000 7faaf7200700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaf81137f0 0x7faaf81a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.907+0000 7faaf6800700 1 -- 172.21.15.112:0/1782099607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaf81137f0 msgr2=0x7faaf81a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.907+0000 7faaf6800700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaf81137f0 0x7faaf81a3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:53.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.907+0000 7faaf6800700 1 -- 172.21.15.112:0/1782099607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaf8077890 msgr2=0x7faaf819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.907+0000 7faaf6800700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaf8077890 0x7faaf819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.907+0000 7faaf6800700 1 -- 172.21.15.112:0/1782099607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf81a4690 con 0x7faaf8076f70 2024-04-25T18:46:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.907+0000 7faaf7200700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaf81137f0 0x7faaf81a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:46:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.908+0000 7faaf6800700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8076f70 0x7faaf819f680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faae8000c00 tx=0x7faae80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.908+0000 7faaef400700 1 -- 172.21.15.112:0/1782099607 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8027070 con 0x7faaf8076f70 2024-04-25T18:46:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.908+0000 7faaef400700 1 -- 172.21.15.112:0/1782099607 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae8022470 con 0x7faaf8076f70 2024-04-25T18:46:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.908+0000 7faafca00700 1 -- 172.21.15.112:0/1782099607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf81a4920 con 0x7faaf8076f70 2024-04-25T18:46:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.908+0000 7faaef400700 1 -- 172.21.15.112:0/1782099607 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8017650 con 0x7faaf8076f70 2024-04-25T18:46:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.908+0000 7faafca00700 1 -- 172.21.15.112:0/1782099607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf81ac200 con 0x7faaf8076f70 2024-04-25T18:46:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.909+0000 7faafca00700 1 -- 172.21.15.112:0/1782099607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf80786b0 con 0x7faaf8076f70 2024-04-25T18:46:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.911+0000 7faaef400700 1 -- 172.21.15.112:0/1782099607 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7faae802a070 con 0x7faaf8076f70 2024-04-25T18:46:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.912+0000 7faaef400700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7faadc0a3c40 0x7faadc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.912+0000 7faaef400700 1 -- 172.21.15.112:0/1782099607 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7faae80d1260 con 0x7faaf8076f70 2024-04-25T18:46:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.913+0000 7faaefe00700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7faadc0a3c40 0x7faadc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.913+0000 7faaefe00700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7faadc0a3c40 0x7faadc0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faaf8079a70 tx=0x7faae0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:53.917+0000 7faaef400700 1 -- 172.21.15.112:0/1782099607 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faae8013070 con 0x7faaf8076f70 2024-04-25T18:46:54.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.188+0000 7faafca00700 1 -- 172.21.15.112:0/1782099607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faaf81a09e0 con 0x7faaf8076f70 2024-04-25T18:46:54.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.190+0000 7faaef400700 1 -- 172.21.15.112:0/1782099607 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7faae809dc50 con 0x7faaf8076f70 2024-04-25T18:46:54.193 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:46:54.193 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:43:45.212327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24511,24512,24493]},"id":1}]} 2024-04-25T18:46:54.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.193+0000 7faafca00700 1 -- 172.21.15.112:0/1782099607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7faadc0a3c40 msgr2=0x7faadc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:54.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.193+0000 7faafca00700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7faadc0a3c40 0x7faadc0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faaf8079a70 tx=0x7faae0009260 comp rx=0 tx=0).stop 2024-04-25T18:46:54.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.193+0000 7faafca00700 1 -- 172.21.15.112:0/1782099607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8076f70 msgr2=0x7faaf819f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:54.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.193+0000 7faafca00700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8076f70 0x7faaf819f680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faae8000c00 tx=0x7faae80057c0 comp rx=0 tx=0).stop 2024-04-25T18:46:54.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.194+0000 7faafca00700 1 -- 172.21.15.112:0/1782099607 shutdown_connections 2024-04-25T18:46:54.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.194+0000 7faafca00700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7faadc0a3c40 0x7faadc0a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:54.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.194+0000 7faafca00700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faaf81137f0 0x7faaf81a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:54.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.194+0000 7faafca00700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaf8076f70 0x7faaf819f680 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:54.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.194+0000 7faafca00700 1 --2- 172.21.15.112:0/1782099607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaf8077890 0x7faaf819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:54.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.194+0000 7faafca00700 1 -- 172.21.15.112:0/1782099607 >> 172.21.15.112:0/1782099607 conn(0x7faaf8100680 msgr2=0x7faaf8102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:54.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.194+0000 7faafca00700 1 -- 172.21.15.112:0/1782099607 shutdown_connections 2024-04-25T18:46:54.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:54.194+0000 7faafca00700 1 -- 172.21.15.112:0/1782099607 wait complete. 2024-04-25T18:46:54.197 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-04-25T18:46:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:54 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1782099607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:54.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:54 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1782099607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:54.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:54 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1782099607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:55.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:46:55.546 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:46:55.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:55 smithi171 ceph-mon[33266]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:55.694 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:55 smithi112 ceph-mon[27864]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:55.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:55 smithi161 ceph-mon[32609]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:46:57.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.910+0000 7f18d5c00700 1 -- 172.21.15.112:0/3538814639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d0109c80 msgr2=0x7f18d010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:57.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.910+0000 7f18d5c00700 1 --2- 172.21.15.112:0/3538814639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d0109c80 0x7f18d010dcd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18b80099c0 tx=0x7f18b8009cd0 comp rx=0 tx=0).stop 2024-04-25T18:46:57.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.910+0000 7f18d5c00700 1 -- 172.21.15.112:0/3538814639 shutdown_connections 2024-04-25T18:46:57.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.910+0000 7f18d5c00700 1 --2- 172.21.15.112:0/3538814639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18d010e3b0 0x7f18d0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:57.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.910+0000 7f18d5c00700 1 --2- 172.21.15.112:0/3538814639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d01092d0 0x7f18d01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.910+0000 7f18d5c00700 1 --2- 172.21.15.112:0/3538814639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d0109c80 0x7f18d010dcd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.910+0000 7f18d5c00700 1 -- 172.21.15.112:0/3538814639 >> 172.21.15.112:0/3538814639 conn(0x7f18d0076210 msgr2=0x7f18d0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.910+0000 7f18d5c00700 1 -- 172.21.15.112:0/3538814639 shutdown_connections 2024-04-25T18:46:57.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.910+0000 7f18d5c00700 1 -- 172.21.15.112:0/3538814639 wait complete. 2024-04-25T18:46:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.911+0000 7f18d5c00700 1 Processor -- start 2024-04-25T18:46:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.911+0000 7f18d5c00700 1 -- start start 2024-04-25T18:46:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18d5c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18d01092d0 0x7f18d019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18d5c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d0109c80 0x7f18d019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18d5c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d010e3b0 0x7f18d01a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18d5c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0119290 con 0x7f18d0109c80 2024-04-25T18:46:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18d5c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0119110 con 0x7f18d010e3b0 2024-04-25T18:46:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18d5c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0119410 con 0x7f18d01092d0 2024-04-25T18:46:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18c6000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d0109c80 0x7f18d019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18cea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18d01092d0 0x7f18d019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:57.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18cea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18d01092d0 0x7f18d019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:58672/0 (socket says 172.21.15.112:58672) 2024-04-25T18:46:57.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18cea00700 1 -- 172.21.15.112:0/1004322072 learned_addr learned my addr 172.21.15.112:0/1004322072 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:46:57.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18c6000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d0109c80 0x7f18d019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53386/0 (socket says 172.21.15.112:53386) 2024-04-25T18:46:57.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.912+0000 7f18cf400700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d010e3b0 0x7f18d01a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:57.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18c6000700 1 -- 172.21.15.112:0/1004322072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18d01092d0 msgr2=0x7f18d019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:57.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18c6000700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18d01092d0 0x7f18d019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:57.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18c6000700 1 -- 172.21.15.112:0/1004322072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d010e3b0 msgr2=0x7f18d01a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:57.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18c6000700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d010e3b0 0x7f18d01a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:57.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18c6000700 1 -- 172.21.15.112:0/1004322072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d01a4730 con 0x7f18d0109c80 2024-04-25T18:46:57.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18cea00700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18d01092d0 0x7f18d019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:46:57.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18c6000700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d0109c80 0x7f18d019fc60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f18b80098e0 tx=0x7f18b80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:57.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18c7e00700 1 -- 172.21.15.112:0/1004322072 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8009e60 con 0x7f18d0109c80 2024-04-25T18:46:57.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18c7e00700 1 -- 172.21.15.112:0/1004322072 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b80047c0 con 0x7f18d0109c80 2024-04-25T18:46:57.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18c7e00700 1 -- 172.21.15.112:0/1004322072 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8005290 con 0x7f18d0109c80 2024-04-25T18:46:57.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18d5c00700 1 -- 172.21.15.112:0/1004322072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d01a49c0 con 0x7f18d0109c80 2024-04-25T18:46:57.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.913+0000 7f18d5c00700 1 -- 172.21.15.112:0/1004322072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01a4df0 con 0x7f18d0109c80 2024-04-25T18:46:57.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.914+0000 7f18d5c00700 1 -- 172.21.15.112:0/1004322072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d0069730 con 0x7f18d0109c80 2024-04-25T18:46:57.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.916+0000 7f18c7e00700 1 -- 172.21.15.112:0/1004322072 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f18b8028060 con 0x7f18d0109c80 2024-04-25T18:46:57.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.918+0000 7f18c7e00700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b40a3c40 0x7f18b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:57.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.918+0000 7f18c7e00700 1 -- 172.21.15.112:0/1004322072 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f18b80d0b90 con 0x7f18d0109c80 2024-04-25T18:46:57.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.918+0000 7f18cea00700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b40a3c40 0x7f18b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:57.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.919+0000 7f18cea00700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b40a3c40 0x7f18b40a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009fd0 tx=0x7f18c0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:57.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:57.922+0000 7f18c7e00700 1 -- 172.21.15.112:0/1004322072 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18b8013070 con 0x7f18d0109c80 2024-04-25T18:46:58.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.200+0000 7f18d5c00700 1 -- 172.21.15.112:0/1004322072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18d002df80 con 0x7f18d0109c80 2024-04-25T18:46:58.203 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:46:58.203 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:43:45.212327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24511,24512,24493]},"id":1}]} 2024-04-25T18:46:58.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.201+0000 7f18c7e00700 1 -- 172.21.15.112:0/1004322072 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11893 (secure 0 0 0) 0x7f18b809d580 con 0x7f18d0109c80 2024-04-25T18:46:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 -- 172.21.15.112:0/1004322072 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b40a3c40 msgr2=0x7f18b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b40a3c40 0x7f18b40a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009fd0 tx=0x7f18c0009040 comp rx=0 tx=0).stop 2024-04-25T18:46:58.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 -- 172.21.15.112:0/1004322072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d0109c80 msgr2=0x7f18d019fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:58.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d0109c80 0x7f18d019fc60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f18b80098e0 tx=0x7f18b80045e0 comp rx=0 tx=0).stop 2024-04-25T18:46:58.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 -- 172.21.15.112:0/1004322072 shutdown_connections 2024-04-25T18:46:58.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f18b40a3c40 0x7f18b40a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:58.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d010e3b0 0x7f18d01a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:58.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18d01092d0 0x7f18d019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:58.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 --2- 172.21.15.112:0/1004322072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18d0109c80 0x7f18d019fc60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:58.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 -- 172.21.15.112:0/1004322072 >> 172.21.15.112:0/1004322072 conn(0x7f18d0076210 msgr2=0x7f18d0076830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:58.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 -- 172.21.15.112:0/1004322072 shutdown_connections 2024-04-25T18:46:58.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:46:58.205+0000 7f18d5c00700 1 -- 172.21.15.112:0/1004322072 wait complete. 2024-04-25T18:46:58.208 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-04-25T18:46:58.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:58 smithi112 ceph-mon[27864]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:58.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:46:58 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1004322072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:58 smithi171 ceph-mon[33266]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:46:58 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1004322072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:58.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:58 smithi161 ceph-mon[32609]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:46:58.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:46:58 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1004322072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:46:59.044 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:46:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:00.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:00 smithi112 ceph-mon[27864]: pgmap v446: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:47:00.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:00 smithi171 ceph-mon[33266]: pgmap v446: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:47:00.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:00 smithi161 ceph-mon[32609]: pgmap v446: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:47:01.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.974+0000 7fa874e00700 1 -- 172.21.15.112:0/2781424827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa870102a90 msgr2=0x7fa870102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:01.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.974+0000 7fa874e00700 1 --2- 172.21.15.112:0/2781424827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa870102a90 0x7fa870102f10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa858009990 tx=0x7fa858009ca0 comp rx=0 tx=0).stop 2024-04-25T18:47:01.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.974+0000 7fa874e00700 1 -- 172.21.15.112:0/2781424827 shutdown_connections 2024-04-25T18:47:01.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.974+0000 7fa874e00700 1 --2- 172.21.15.112:0/2781424827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa870103450 0x7fa87010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:01.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.974+0000 7fa874e00700 1 --2- 172.21.15.112:0/2781424827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa870108a50 0x7fa870108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:01.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.974+0000 7fa874e00700 1 --2- 172.21.15.112:0/2781424827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa870102a90 0x7fa870102f10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:01.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.974+0000 7fa874e00700 1 -- 172.21.15.112:0/2781424827 >> 172.21.15.112:0/2781424827 conn(0x7fa870079ba0 msgr2=0x7fa870079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:01.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.974+0000 7fa874e00700 1 -- 172.21.15.112:0/2781424827 shutdown_connections 2024-04-25T18:47:01.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.974+0000 7fa874e00700 1 -- 172.21.15.112:0/2781424827 wait complete. 2024-04-25T18:47:01.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.976+0000 7fa874e00700 1 Processor -- start 2024-04-25T18:47:01.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.976+0000 7fa874e00700 1 -- start start 2024-04-25T18:47:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.977+0000 7fa874e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa870102a90 0x7fa870199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.977+0000 7fa874e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa870103450 0x7fa870199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.977+0000 7fa874e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa870108a50 0x7fa87019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.977+0000 7fa874e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa870112c20 con 0x7fa870102a90 2024-04-25T18:47:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.977+0000 7fa874e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa870112aa0 con 0x7fa870108a50 2024-04-25T18:47:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.977+0000 7fa874e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa870112da0 con 0x7fa870103450 2024-04-25T18:47:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.977+0000 7fa86de00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa870102a90 0x7fa870199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.977+0000 7fa867e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa870103450 0x7fa870199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.977+0000 7fa86e800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa870108a50 0x7fa87019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.977+0000 7fa867e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa870103450 0x7fa870199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:32840/0 (socket says 172.21.15.112:32840) 2024-04-25T18:47:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.977+0000 7fa867e00700 1 -- 172.21.15.112:0/643780366 learned_addr learned my addr 172.21.15.112:0/643780366 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:01.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.978+0000 7fa867e00700 1 -- 172.21.15.112:0/643780366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa870108a50 msgr2=0x7fa87019d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:01.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.978+0000 7fa867e00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa870108a50 0x7fa87019d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:01.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.978+0000 7fa867e00700 1 -- 172.21.15.112:0/643780366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa870102a90 msgr2=0x7fa870199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:01.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.978+0000 7fa867e00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa870102a90 0x7fa870199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:01.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.978+0000 7fa867e00700 1 -- 172.21.15.112:0/643780366 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87019e120 con 0x7fa870103450 2024-04-25T18:47:01.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.978+0000 7fa867e00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa870103450 0x7fa870199650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa8580057c0 tx=0x7fa858009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:01.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.978+0000 7fa866a00700 1 -- 172.21.15.112:0/643780366 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858026070 con 0x7fa870103450 2024-04-25T18:47:01.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.978+0000 7fa866a00700 1 -- 172.21.15.112:0/643780366 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa858021410 con 0x7fa870103450 2024-04-25T18:47:01.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.978+0000 7fa874e00700 1 -- 172.21.15.112:0/643780366 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87019e3b0 con 0x7fa870103450 2024-04-25T18:47:01.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.978+0000 7fa866a00700 1 -- 172.21.15.112:0/643780366 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858017650 con 0x7fa870103450 2024-04-25T18:47:01.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.978+0000 7fa874e00700 1 -- 172.21.15.112:0/643780366 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8701a5c90 con 0x7fa870103450 2024-04-25T18:47:01.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.980+0000 7fa874e00700 1 -- 172.21.15.112:0/643780366 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa870069730 con 0x7fa870103450 2024-04-25T18:47:01.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.982+0000 7fa866a00700 1 -- 172.21.15.112:0/643780366 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fa858029070 con 0x7fa870103450 2024-04-25T18:47:01.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.983+0000 7fa866a00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa8540a3c40 0x7fa8540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:01.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.983+0000 7fa866a00700 1 -- 172.21.15.112:0/643780366 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa8580d14c0 con 0x7fa870103450 2024-04-25T18:47:01.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.983+0000 7fa86de00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa8540a3c40 0x7fa8540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:01.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.984+0000 7fa86de00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa8540a3c40 0x7fa8540a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa860009be0 tx=0x7fa860009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:01.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:01.985+0000 7fa866a00700 1 -- 172.21.15.112:0/643780366 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa858030080 con 0x7fa870103450 2024-04-25T18:47:02.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.213+0000 7fa874e00700 1 -- 172.21.15.112:0/643780366 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa87002df80 con 0x7fa870103450 2024-04-25T18:47:02.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.215+0000 7fa866a00700 1 -- 172.21.15.112:0/643780366 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+22709 (secure 0 0 0) 0x7fa858013070 con 0x7fa870103450 2024-04-25T18:47:02.217 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:47:02.217 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","created":"2024-04-25T18:33:37.379089+0000","modified":"2024-04-25T18:43:56.620122+0000","last_up_change":"2024-04-25T18:42:00.961646+0000","last_in_change":"2024-04-25T18:41:40.647162+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:37:45.106199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:43:11.017975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:43:15.116917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efb4bc01-e29b-4402-8a7a-3387d1731ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6803","nonce":232254535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6805","nonce":232254535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6809","nonce":232254535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6807","nonce":232254535}]},"public_addr":"172.21.15.112:6803/232254535","cluster_addr":"172.21.15.112:6805/232254535","heartbeat_back_addr":"172.21.15.112:6809/232254535","heartbeat_front_addr":"172.21.15.112:6807/232254535","state":["exists","up"]},{"osd":1,"uuid":"65a83fa2-96a6-4743-8c17-21531f5e885e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6801","nonce":664224744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6803","nonce":664224744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6807","nonce":664224744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6805","nonce":664224744}]},"public_addr":"172.21.15.161:6801/664224744","cluster_addr":"172.21.15.161:6803/664224744","heartbeat_back_addr":"172.21.15.161:6807/664224744","heartbeat_front_addr":"172.21.15.161:6805/664224744","state":["exists","up"]},{"osd":2,"uuid":"7562ec4c-c126-48ec-a51c-56fa461a2e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6801","nonce":345172073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6803","nonce":345172073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6807","nonce":345172073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6805","nonce":345172073}]},"public_addr":"172.21.15.171:6801/345172073","cluster_addr":"172.21.15.171:6803/345172073","heartbeat_back_addr":"172.21.15.171:6807/345172073","heartbeat_front_addr":"172.21.15.171:6805/345172073","state":["exists","up"]},{"osd":3,"uuid":"d7aae8cd-fec7-46d9-8773-25eaaca3cc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6811","nonce":1472407156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6813","nonce":1472407156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6817","nonce":1472407156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6815","nonce":1472407156}]},"public_addr":"172.21.15.112:6811/1472407156","cluster_addr":"172.21.15.112:6813/1472407156","heartbeat_back_addr":"172.21.15.112:6817/1472407156","heartbeat_front_addr":"172.21.15.112:6815/1472407156","state":["exists","up"]},{"osd":4,"uuid":"b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6809","nonce":373204306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6811","nonce":373204306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6815","nonce":373204306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6813","nonce":373204306}]},"public_addr":"172.21.15.161:6809/373204306","cluster_addr":"172.21.15.161:6811/373204306","heartbeat_back_addr":"172.21.15.161:6815/373204306","heartbeat_front_addr":"172.21.15.161:6813/373204306","state":["exists","up"]},{"osd":5,"uuid":"56124d12-bc1f-4cba-8430-938248851776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6809","nonce":2643512313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6811","nonce":2643512313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6815","nonce":2643512313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6813","nonce":2643512313}]},"public_addr":"172.21.15.171:6809/2643512313","cluster_addr":"172.21.15.171:6811/2643512313","heartbeat_back_addr":"172.21.15.171:6815/2643512313","heartbeat_front_addr":"172.21.15.171:6813/2643512313","state":["exists","up"]},{"osd":6,"uuid":"9563a9d3-352b-4686-975c-282ec167248b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6819","nonce":1933654933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6821","nonce":1933654933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6825","nonce":1933654933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6823","nonce":1933654933}]},"public_addr":"172.21.15.112:6819/1933654933","cluster_addr":"172.21.15.112:6821/1933654933","heartbeat_back_addr":"172.21.15.112:6825/1933654933","heartbeat_front_addr":"172.21.15.112:6823/1933654933","state":["exists","up"]},{"osd":7,"uuid":"f2d06e86-6aac-40f4-956e-18fe0b0e3543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6817","nonce":3060201138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6819","nonce":3060201138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6823","nonce":3060201138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6821","nonce":3060201138}]},"public_addr":"172.21.15.161:6817/3060201138","cluster_addr":"172.21.15.161:6819/3060201138","heartbeat_back_addr":"172.21.15.161:6823/3060201138","heartbeat_front_addr":"172.21.15.161:6821/3060201138","state":["exists","up"]},{"osd":8,"uuid":"f006e45f-2bb9-45d0-9020-be7c33c6a825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6817","nonce":835756571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6819","nonce":835756571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6823","nonce":835756571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6821","nonce":835756571}]},"public_addr":"172.21.15.171:6817/835756571","cluster_addr":"172.21.15.171:6819/835756571","heartbeat_back_addr":"172.21.15.171:6823/835756571","heartbeat_front_addr":"172.21.15.171:6821/835756571","state":["exists","up"]},{"osd":9,"uuid":"c5a96491-4151-4633-b3bc-2c214fe51c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6827","nonce":349758528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6829","nonce":349758528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6833","nonce":349758528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6831","nonce":349758528}]},"public_addr":"172.21.15.112:6827/349758528","cluster_addr":"172.21.15.112:6829/349758528","heartbeat_back_addr":"172.21.15.112:6833/349758528","heartbeat_front_addr":"172.21.15.112:6831/349758528","state":["exists","up"]},{"osd":10,"uuid":"244eda3e-25a0-490e-879f-c1e2726c1be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6825","nonce":3013292765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6827","nonce":3013292765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6831","nonce":3013292765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6829","nonce":3013292765}]},"public_addr":"172.21.15.161:6825/3013292765","cluster_addr":"172.21.15.161:6827/3013292765","heartbeat_back_addr":"172.21.15.161:6831/3013292765","heartbeat_front_addr":"172.21.15.161:6829/3013292765","state":["exists","up"]},{"osd":11,"uuid":"356e5c54-bbc3-45ee-bda2-85164ce3cb92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6825","nonce":1010336458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6827","nonce":1010336458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6831","nonce":1010336458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6829","nonce":1010336458}]},"public_addr":"172.21.15.171:6825/1010336458","cluster_addr":"172.21.15.171:6827/1010336458","heartbeat_back_addr":"172.21.15.171:6831/1010336458","heartbeat_front_addr":"172.21.15.171:6829/1010336458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:36:51.024650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:17.026638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:41.239726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:15.148968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:41.029118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:06.225858+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:41.639444+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:07.753400+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:32.292085+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:07.159926+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:33.749782+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:58.594447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/181862246":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/3393454453":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3195850527":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6801/3468460974":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3392048544":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/1329430933":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/735578423":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/3468460974":"2024-04-26T18:34:20.605775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:47:02.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.219+0000 7fa874e00700 1 -- 172.21.15.112:0/643780366 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa8540a3c40 msgr2=0x7fa8540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:02.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.219+0000 7fa874e00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa8540a3c40 0x7fa8540a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa860009be0 tx=0x7fa860009260 comp rx=0 tx=0).stop 2024-04-25T18:47:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.219+0000 7fa874e00700 1 -- 172.21.15.112:0/643780366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa870103450 msgr2=0x7fa870199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.219+0000 7fa874e00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa870103450 0x7fa870199650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa8580057c0 tx=0x7fa858009580 comp rx=0 tx=0).stop 2024-04-25T18:47:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.219+0000 7fa874e00700 1 -- 172.21.15.112:0/643780366 shutdown_connections 2024-04-25T18:47:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.219+0000 7fa874e00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fa8540a3c40 0x7fa8540a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.219+0000 7fa874e00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa870108a50 0x7fa87019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.219+0000 7fa874e00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa870102a90 0x7fa870199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.219+0000 7fa874e00700 1 --2- 172.21.15.112:0/643780366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa870103450 0x7fa870199650 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.219+0000 7fa874e00700 1 -- 172.21.15.112:0/643780366 >> 172.21.15.112:0/643780366 conn(0x7fa870079ba0 msgr2=0x7fa8701018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.220+0000 7fa874e00700 1 -- 172.21.15.112:0/643780366 shutdown_connections 2024-04-25T18:47:02.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:02.220+0000 7fa874e00700 1 -- 172.21.15.112:0/643780366 wait complete. 2024-04-25T18:47:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:02 smithi112 ceph-mon[27864]: pgmap v447: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:47:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:02 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/643780366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:47:02.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:02 smithi171 ceph-mon[33266]: pgmap v447: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:47:02.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:02 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/643780366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:47:02.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:02 smithi161 ceph-mon[32609]: pgmap v447: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:47:02.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:02 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/643780366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:47:03.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T18:43:19.347467+0000', 'modified': '2024-04-25T18:43:45.212327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24511, 'mds_2': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/4007863322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 4007863322}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 4007863322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/3522712423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 3522712423}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 3522712423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'g', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6839/3241331981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 3241331981}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 3241331981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.171:6839/1642228003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 1642228003}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 1642228003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.112:6841/1028859850', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6840', 'nonce': 1028859850}, {'type': 'v1', 'addr': '172.21.15.112:6841', 'nonce': 1028859850}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.161:6839/2315334000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2315334000}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2315334000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24511, 24512, 24493]} 2024-04-25T18:47:03.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T18:47:03.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T18:47:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:04.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:04 smithi112 ceph-mon[27864]: pgmap v448: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:47:04.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:04 smithi171 ceph-mon[33266]: pgmap v448: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:47:04.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:04 smithi161 ceph-mon[32609]: pgmap v448: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:47:05.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:05 smithi112 ceph-mon[27864]: pgmap v449: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:47:05.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:05 smithi171 ceph-mon[33266]: pgmap v449: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:47:05.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:05 smithi161 ceph-mon[32609]: pgmap v449: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:47:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.060+0000 7f03d1c00700 1 -- 172.21.15.112:0/791561178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03cc102a90 msgr2=0x7f03cc102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.060+0000 7f03d1c00700 1 --2- 172.21.15.112:0/791561178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03cc102a90 0x7f03cc102f10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009990 tx=0x7f03b4009ca0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.060+0000 7f03d1c00700 1 -- 172.21.15.112:0/791561178 shutdown_connections 2024-04-25T18:47:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.060+0000 7f03d1c00700 1 --2- 172.21.15.112:0/791561178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc103450 0x7f03cc10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.060+0000 7f03d1c00700 1 --2- 172.21.15.112:0/791561178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03cc108a50 0x7f03cc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.060+0000 7f03d1c00700 1 --2- 172.21.15.112:0/791561178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03cc102a90 0x7f03cc102f10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.060+0000 7f03d1c00700 1 -- 172.21.15.112:0/791561178 >> 172.21.15.112:0/791561178 conn(0x7f03cc079ba0 msgr2=0x7f03cc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.061+0000 7f03d1c00700 1 -- 172.21.15.112:0/791561178 shutdown_connections 2024-04-25T18:47:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.061+0000 7f03d1c00700 1 -- 172.21.15.112:0/791561178 wait complete. 2024-04-25T18:47:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.062+0000 7f03d1c00700 1 Processor -- start 2024-04-25T18:47:06.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.062+0000 7f03d1c00700 1 -- start start 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.062+0000 7f03d1c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03cc102a90 0x7f03cc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.062+0000 7f03d1c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03cc103450 0x7f03cc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.062+0000 7f03d1c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc108a50 0x7f03cc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.062+0000 7f03d1c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc112c50 con 0x7f03cc102a90 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.062+0000 7f03d1c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc112ad0 con 0x7f03cc103450 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.062+0000 7f03d1c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc112dd0 con 0x7f03cc108a50 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.063+0000 7f03caa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03cc102a90 0x7f03cc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.063+0000 7f03caa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03cc102a90 0x7f03cc074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50486/0 (socket says 172.21.15.112:50486) 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.063+0000 7f03caa00700 1 -- 172.21.15.112:0/3019658607 learned_addr learned my addr 172.21.15.112:0/3019658607 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.063+0000 7f03c2000700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03cc103450 0x7f03cc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.063+0000 7f03cb400700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc108a50 0x7f03cc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.063+0000 7f03caa00700 1 -- 172.21.15.112:0/3019658607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc108a50 msgr2=0x7f03cc0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.063+0000 7f03caa00700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc108a50 0x7f03cc0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.063+0000 7f03caa00700 1 -- 172.21.15.112:0/3019658607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03cc103450 msgr2=0x7f03cc075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:06.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.063+0000 7f03caa00700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03cc103450 0x7f03cc075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.063+0000 7f03caa00700 1 -- 172.21.15.112:0/3019658607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc1a76d0 con 0x7f03cc102a90 2024-04-25T18:47:06.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.063+0000 7f03cb400700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc108a50 0x7f03cc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:47:06.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.064+0000 7f03caa00700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03cc102a90 0x7f03cc074d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00e9a0 tx=0x7f03bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:06.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.064+0000 7f03c3e00700 1 -- 172.21.15.112:0/3019658607 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc00cbd0 con 0x7f03cc102a90 2024-04-25T18:47:06.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.064+0000 7f03c3e00700 1 -- 172.21.15.112:0/3019658607 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03bc00cd70 con 0x7f03cc102a90 2024-04-25T18:47:06.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.064+0000 7f03c3e00700 1 -- 172.21.15.112:0/3019658607 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc018680 con 0x7f03cc102a90 2024-04-25T18:47:06.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.064+0000 7f03d1c00700 1 -- 172.21.15.112:0/3019658607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc1a79c0 con 0x7f03cc102a90 2024-04-25T18:47:06.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.064+0000 7f03d1c00700 1 -- 172.21.15.112:0/3019658607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc1a7eb0 con 0x7f03cc102a90 2024-04-25T18:47:06.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.067+0000 7f03c3e00700 1 -- 172.21.15.112:0/3019658607 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f03bc010430 con 0x7f03cc102a90 2024-04-25T18:47:06.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.068+0000 7f03c3e00700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f03b00a3cc0 0x7f03b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.069+0000 7f03dc4c0b80 1 Processor -- start 2024-04-25T18:47:06.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.069+0000 7f03c3e00700 1 -- 172.21.15.112:0/3019658607 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f03bc0d05b0 con 0x7f03cc102a90 2024-04-25T18:47:06.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.069+0000 7f03dc4c0b80 1 -- start start 2024-04-25T18:47:06.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.069+0000 7f03c2000700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f03b00a3cc0 0x7f03b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.069+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033af2f0 0x562f033af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.069+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033ba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.069+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033baab0 0x562f033bcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.069+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f03302e40 con 0x562f033af2f0 2024-04-25T18:47:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.069+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562f032a96e0 con 0x562f033baab0 2024-04-25T18:47:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.069+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562f032fee00 con 0x562f033afc10 2024-04-25T18:47:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03caa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033af2f0 0x562f033af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03c2000700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f03b00a3cc0 0x7f03b00a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f03cc07a2f0 tx=0x7f03b400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03cb400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033baab0 0x562f033bcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03c2000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03caa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033af2f0 0x562f033af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50494/0 (socket says 172.21.15.112:50494) 2024-04-25T18:47:06.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03caa00700 1 -- 172.21.15.112:0/1447098354 learned_addr learned my addr 172.21.15.112:0/1447098354 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:06.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03caa00700 1 -- 172.21.15.112:0/1447098354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 msgr2=0x562f033ba570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:06.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03caa00700 1 --2- 172.21.15.112:0/1447098354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033ba570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03caa00700 1 -- 172.21.15.112:0/1447098354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033baab0 msgr2=0x562f033bcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03caa00700 1 --2- 172.21.15.112:0/1447098354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033baab0 0x562f033bcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03caa00700 1 -- 172.21.15.112:0/1447098354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f0329f630 con 0x562f033af2f0 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.070+0000 7f03caa00700 1 --2- 172.21.15.112:0/1447098354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033af2f0 0x562f033af6d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f03bc09c510 tx=0x7f03bc09c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0feff1926424af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.071+0000 7f03bbe00700 1 -- 172.21.15.112:0/1447098354 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc09c810 con 0x562f033af2f0 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.071+0000 7f03bbe00700 1 -- 172.21.15.112:0/1447098354 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03bc0a0bc0 con 0x562f033af2f0 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.071+0000 7f03bbe00700 1 -- 172.21.15.112:0/1447098354 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0a5280 con 0x562f033af2f0 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.071+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1447098354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033af2f0 msgr2=0x562f033af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.071+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1447098354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033af2f0 0x562f033af6d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f03bc09c510 tx=0x7f03bc09c170 comp rx=0 tx=0).stop 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.072+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1447098354 shutdown_connections 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.072+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1447098354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033baab0 0x562f033bcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.072+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1447098354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033af2f0 0x562f033af6d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.072+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1447098354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033ba570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.072+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1447098354 >> 172.21.15.112:0/1447098354 conn(0x562f032de550 msgr2=0x562f033ab790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:06.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.072+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1447098354 shutdown_connections 2024-04-25T18:47:06.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.072+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1447098354 wait complete. 2024-04-25T18:47:06.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.072+0000 7f03dc4c0b80 1 -- start start 2024-04-25T18:47:06.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.073+0000 7f03dc4c0b80 1 Processor -- start 2024-04-25T18:47:06.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.073+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033ba220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.073+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033b7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.073+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033baab0 0x562f033b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.073+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f033bd540 con 0x562f033baab0 2024-04-25T18:47:06.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.073+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562f032bc810 con 0x562f033af2f0 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.073+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562f033bd6c0 con 0x562f033afc10 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.073+0000 7f03dc4c0b80 10 client.0 _ll_register_callbacks cb 0x562f0329b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.073+0000 7f03caa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033ba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03caa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033ba220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:60910/0 (socket says 172.21.15.112:60910) 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03caa00700 1 -- 172.21.15.112:0/1097066959 learned_addr learned my addr 172.21.15.112:0/1097066959 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03cb400700 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033baab0 0x562f033b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03c2000700 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03cb400700 1 -- 172.21.15.112:0/1097066959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 msgr2=0x562f033b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03cb400700 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03cb400700 1 -- 172.21.15.112:0/1097066959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 msgr2=0x562f033ba220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03cb400700 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033ba220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03cb400700 1 -- 172.21.15.112:0/1097066959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f03250880 con 0x562f033baab0 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03caa00700 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033ba220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:06.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03cb400700 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033baab0 0x562f033b7a60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f03c4007a60 tx=0x7f03c4007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba6d25c3bdf36ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:06.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.074+0000 7f03b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:47:06.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.075+0000 7f03b8c00700 1 -- 172.21.15.112:0/1097066959 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c401a030 con 0x562f033baab0 2024-04-25T18:47:06.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.075+0000 7f03b8c00700 1 -- 172.21.15.112:0/1097066959 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f03c4003c10 con 0x562f033baab0 2024-04-25T18:47:06.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.075+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1097066959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03bc0a0bc0 con 0x562f033baab0 2024-04-25T18:47:06.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.075+0000 7f03b8c00700 1 -- 172.21.15.112:0/1097066959 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03c4003e10 con 0x562f033baab0 2024-04-25T18:47:06.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.075+0000 7f03dc4c0b80 10 client.14865 fetch_fsmap learned FSMap version 25 2024-04-25T18:47:06.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.075+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1097066959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562f03299aa0 con 0x562f033baab0 2024-04-25T18:47:06.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.075+0000 7f03b8c00700 1 -- 172.21.15.112:0/1097066959 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03bc0a0bc0 con 0x562f033baab0 2024-04-25T18:47:06.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.075+0000 7f03b8c00700 1 -- 172.21.15.112:0/1097066959 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f03c4004480 con 0x562f033baab0 2024-04-25T18:47:06.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.075+0000 7f03dc4c0b80 10 client.14865 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:47:06.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.075+0000 7f03dc4c0b80 10 client.14865 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:47:06.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.076+0000 7f03dc4c0b80 20 client.14865 populate_metadata read hostname 'smithi112' 2024-04-25T18:47:06.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.076+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x562f033ca310 0x562f033cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.076+0000 7f03dc4c0b80 4 client.14865 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:47:06.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.076+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1097066959 --> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f03310630 con 0x562f033ca310 2024-04-25T18:47:06.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.076+0000 7f03caa00700 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x562f033ca310 0x562f033cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.077+0000 7f03caa00700 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x562f033ca310 0x562f033cb780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79e542ee0d86e813 server_cookie=1597f6cd8138d7b4 in_seq=0 out_seq=0 2024-04-25T18:47:06.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.077+0000 7f03b8c00700 10 client.14865 ms_handle_connect on v2:172.21.15.171:6838/1642228003 2024-04-25T18:47:06.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03b8c00700 1 -- 172.21.15.112:0/1097066959 <== mds.2 v2:172.21.15.171:6838/1642228003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f03310630 con 0x562f033ca310 2024-04-25T18:47:06.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03b8c00700 10 client.14865 handle_command_reply: tid=0 2024-04-25T18:47:06.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03dc4c0b80 1 client.14865 shutdown 2024-04-25T18:47:06.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1097066959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033baab0 msgr2=0x562f033b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:06.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033baab0 0x562f033b7a60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f03c4007a60 tx=0x7f03c4007d70 comp rx=0 tx=0).stop 2024-04-25T18:47:06.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1097066959 shutdown_connections 2024-04-25T18:47:06.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x562f033ca310 0x562f033cb780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033baab0 0x562f033b7a60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033ba220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1097066959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1097066959 >> 172.21.15.112:0/1097066959 conn(0x562f032de550 msgr2=0x562f033b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:06.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03b8c00700 0 client.14865 ms_handle_reset on v2:172.21.15.171:6838/1642228003 2024-04-25T18:47:06.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1097066959 shutdown_connections 2024-04-25T18:47:06.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.078+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1097066959 wait complete. 2024-04-25T18:47:06.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.079+0000 7f03dc4c0b80 20 client.14865 trim_cache size 0 max 16384 2024-04-25T18:47:06.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.100+0000 7f03dc4c0b80 1 Processor -- start 2024-04-25T18:47:06.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03dc4c0b80 1 -- start start 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033baab0 0x562f033ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f033c9ac0 con 0x562f033baab0 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562f033ca190 con 0x562f033af2f0 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562f033ca860 con 0x562f033afc10 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03cb400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033baab0 0x562f033ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03caa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03caa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:60924/0 (socket says 172.21.15.112:60924) 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03caa00700 1 -- 172.21.15.112:0/1483139135 learned_addr learned my addr 172.21.15.112:0/1483139135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.101+0000 7f03c2000700 1 --2- 172.21.15.112:0/1483139135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.102+0000 7f03caa00700 1 -- 172.21.15.112:0/1483139135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 msgr2=0x562f033ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.102+0000 7f03caa00700 1 --2- 172.21.15.112:0/1483139135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.102+0000 7f03caa00700 1 -- 172.21.15.112:0/1483139135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033baab0 msgr2=0x562f033ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:06.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.102+0000 7f03caa00700 1 --2- 172.21.15.112:0/1483139135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033baab0 0x562f033ab610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.102+0000 7f03caa00700 1 -- 172.21.15.112:0/1483139135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f033cbaf0 con 0x562f033af2f0 2024-04-25T18:47:06.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.102+0000 7f03c2000700 1 --2- 172.21.15.112:0/1483139135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:47:06.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.102+0000 7f03caa00700 1 --2- 172.21.15.112:0/1483139135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033b2c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f03bc09c7a0 tx=0x7f03bc010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e823a647588bcaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:06.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.102+0000 7f03bb400700 1 -- 172.21.15.112:0/1483139135 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc09ab10 con 0x562f033af2f0 2024-04-25T18:47:06.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.102+0000 7f03bb400700 1 -- 172.21.15.112:0/1483139135 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03bc0870b0 con 0x562f033af2f0 2024-04-25T18:47:06.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.102+0000 7f03bb400700 1 -- 172.21.15.112:0/1483139135 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc09bc60 con 0x562f033af2f0 2024-04-25T18:47:06.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.103+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1483139135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 msgr2=0x562f033b2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:06.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.103+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1483139135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033b2c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f03bc09c7a0 tx=0x7f03bc010cb0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.103+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1483139135 shutdown_connections 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.103+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1483139135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033baab0 0x562f033ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.103+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1483139135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033af2f0 0x562f033b2c20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.103+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/1483139135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033afc10 0x562f033ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.103+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1483139135 >> 172.21.15.112:0/1483139135 conn(0x562f032de550 msgr2=0x562f033b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.103+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1483139135 shutdown_connections 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.103+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/1483139135 wait complete. 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.104+0000 7f03dc4c0b80 1 -- start start 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.104+0000 7f03dc4c0b80 1 Processor -- start 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.104+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033af2f0 0x562f033bc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.104+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033afc10 0x562f033bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.104+0000 7f03dc4c0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033baab0 0x562f033d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.104+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562f033cb600 con 0x562f033afc10 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.104+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562f033caf30 con 0x562f033baab0 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.104+0000 7f03dc4c0b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562f033cbcd0 con 0x562f033af2f0 2024-04-25T18:47:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.104+0000 7f03dc4c0b80 10 client.0 _ll_register_callbacks cb 0x562f0329b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:47:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03c2000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033afc10 0x562f033bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03caa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033af2f0 0x562f033bc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03c2000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033afc10 0x562f033bc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50510/0 (socket says 172.21.15.112:50510) 2024-04-25T18:47:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03caa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033af2f0 0x562f033bc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:32902/0 (socket says 172.21.15.112:32902) 2024-04-25T18:47:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03c2000700 1 -- 172.21.15.112:0/3018684827 learned_addr learned my addr 172.21.15.112:0/3018684827 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03cb400700 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033baab0 0x562f033d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03caa00700 1 -- 172.21.15.112:0/3018684827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033baab0 msgr2=0x562f033d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:06.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03caa00700 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033baab0 0x562f033d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03caa00700 1 -- 172.21.15.112:0/3018684827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033afc10 msgr2=0x562f033bc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:06.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03caa00700 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033afc10 0x562f033bc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03caa00700 1 -- 172.21.15.112:0/3018684827 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f033d1850 con 0x562f033af2f0 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03caa00700 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033af2f0 0x562f033bc300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009340 tx=0x7f03bc010bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e6bbc792cb1d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.105+0000 7f03b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.106+0000 7f03b8c00700 1 -- 172.21.15.112:0/3018684827 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc09c810 con 0x562f033af2f0 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.106+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/3018684827 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03bc0870b0 con 0x562f033af2f0 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.106+0000 7f03b8c00700 1 -- 172.21.15.112:0/3018684827 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f03bc010720 con 0x562f033af2f0 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.106+0000 7f03b8c00700 1 -- 172.21.15.112:0/3018684827 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03bc010920 con 0x562f033af2f0 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.106+0000 7f03dc4c0b80 10 client.24988 fetch_fsmap learned FSMap version 25 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.106+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/3018684827 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562f03431ca0 con 0x562f033af2f0 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.106+0000 7f03b8c00700 1 -- 172.21.15.112:0/3018684827 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03bc0870b0 con 0x562f033af2f0 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.107+0000 7f03b8c00700 1 -- 172.21.15.112:0/3018684827 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f03bc098560 con 0x562f033af2f0 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.107+0000 7f03dc4c0b80 10 client.24988 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.107+0000 7f03dc4c0b80 10 client.24988 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:47:06.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.107+0000 7f03dc4c0b80 20 client.24988 populate_metadata read hostname 'smithi112' 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.107+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x562f034361c0 0x562f03434ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.107+0000 7f03dc4c0b80 4 client.24988 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.107+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/3018684827 --> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] -- command(tid 0: {"prefix": "status"}) -- 0x562f0330a6d0 con 0x562f034361c0 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.107+0000 7f03c2000700 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x562f034361c0 0x562f03434ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03c2000700 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x562f034361c0 0x562f03434ed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=592570da6d93d0b3 server_cookie=9987b4d652c67288 in_seq=0 out_seq=0 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03b8c00700 10 client.24988 ms_handle_connect on v2:172.21.15.171:6838/1642228003 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03b8c00700 1 -- 172.21.15.112:0/3018684827 <== mds.2 v2:172.21.15.171:6838/1642228003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562f0330a6d0 con 0x562f034361c0 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03b8c00700 10 client.24988 handle_command_reply: tid=0 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03dc4c0b80 1 client.24988 shutdown 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/3018684827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033af2f0 msgr2=0x562f033bc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033af2f0 0x562f033bc300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009340 tx=0x7f03bc010bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/3018684827 shutdown_connections 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x562f034361c0 0x562f03434ed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562f033baab0 0x562f033d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562f033af2f0 0x562f033bc300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03dc4c0b80 1 --2- 172.21.15.112:0/3018684827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562f033afc10 0x562f033bc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/3018684827 >> 172.21.15.112:0/3018684827 conn(0x562f032de550 msgr2=0x562f033b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:06.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03b8c00700 0 client.24988 ms_handle_reset on v2:172.21.15.171:6838/1642228003 2024-04-25T18:47:06.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/3018684827 shutdown_connections 2024-04-25T18:47:06.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.108+0000 7f03dc4c0b80 1 -- 172.21.15.112:0/3018684827 wait complete. 2024-04-25T18:47:06.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.109+0000 7f03dc4c0b80 20 client.24988 trim_cache size 0 max 16384 2024-04-25T18:47:06.112 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:47:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:47:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-04-25T18:47:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24493, 2024-04-25T18:47:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T18:47:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T18:47:06.113 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T18:47:06.113 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 207.887100414, 2024-04-25T18:47:06.113 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 25, 2024-04-25T18:47:06.113 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 94, 2024-04-25T18:47:06.113 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:47:06.113 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 262.22231990199998 2024-04-25T18:47:06.113 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:47:06.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.109+0000 7f03d1c00700 1 -- 172.21.15.112:0/3019658607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f03b00a3cc0 msgr2=0x7f03b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.109+0000 7f03d1c00700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f03b00a3cc0 0x7f03b00a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f03cc07a2f0 tx=0x7f03b400b260 comp rx=0 tx=0).stop 2024-04-25T18:47:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.109+0000 7f03d1c00700 1 -- 172.21.15.112:0/3019658607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03cc102a90 msgr2=0x7f03cc074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.109+0000 7f03d1c00700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03cc102a90 0x7f03cc074d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00e9a0 tx=0x7f03bc00ecb0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.109+0000 7f03d1c00700 1 -- 172.21.15.112:0/3019658607 shutdown_connections 2024-04-25T18:47:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.109+0000 7f03d1c00700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f03b00a3cc0 0x7f03b00a6180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.109+0000 7f03d1c00700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc108a50 0x7f03cc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.109+0000 7f03d1c00700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03cc102a90 0x7f03cc074d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.109+0000 7f03d1c00700 1 --2- 172.21.15.112:0/3019658607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03cc103450 0x7f03cc075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.109+0000 7f03d1c00700 1 -- 172.21.15.112:0/3019658607 >> 172.21.15.112:0/3019658607 conn(0x7f03cc079ba0 msgr2=0x7f03cc101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.110+0000 7f03d1c00700 1 -- 172.21.15.112:0/3019658607 shutdown_connections 2024-04-25T18:47:06.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:06.110+0000 7f03d1c00700 1 -- 172.21.15.112:0/3019658607 wait complete. 2024-04-25T18:47:07.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.a status 2024-04-25T18:47:07.470 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:08.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:08 smithi112 ceph-mon[27864]: pgmap v450: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:47:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:08 smithi171 ceph-mon[33266]: pgmap v450: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:47:08.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:08 smithi161 ceph-mon[32609]: pgmap v450: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:47:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.898+0000 7f6c60000700 1 -- 172.21.15.112:0/1940434734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c581092d0 msgr2=0x7f6c581096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.898+0000 7f6c60000700 1 --2- 172.21.15.112:0/1940434734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c581092d0 0x7f6c581096b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0099c0 tx=0x7f6c4c009cd0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.898+0000 7f6c60000700 1 -- 172.21.15.112:0/1940434734 shutdown_connections 2024-04-25T18:47:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.898+0000 7f6c60000700 1 --2- 172.21.15.112:0/1940434734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c5810e3b0 0x7f6c58115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.898+0000 7f6c60000700 1 --2- 172.21.15.112:0/1940434734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c581092d0 0x7f6c581096b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.898+0000 7f6c60000700 1 --2- 172.21.15.112:0/1940434734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c58109c80 0x7f6c5810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.898+0000 7f6c60000700 1 -- 172.21.15.112:0/1940434734 >> 172.21.15.112:0/1940434734 conn(0x7f6c58076210 msgr2=0x7f6c58107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:09.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.898+0000 7f6c60000700 1 -- 172.21.15.112:0/1940434734 shutdown_connections 2024-04-25T18:47:09.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.899+0000 7f6c60000700 1 -- 172.21.15.112:0/1940434734 wait complete. 2024-04-25T18:47:09.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.899+0000 7f6c60000700 1 Processor -- start 2024-04-25T18:47:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.900+0000 7f6c60000700 1 -- start start 2024-04-25T18:47:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.900+0000 7f6c60000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c581092d0 0x7f6c581a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.900+0000 7f6c60000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58109c80 0x7f6c581a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.900+0000 7f6c60000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c5810e3b0 0x7f6c5819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.900+0000 7f6c60000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c581192b0 con 0x7f6c5810e3b0 2024-04-25T18:47:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.900+0000 7f6c60000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58119130 con 0x7f6c58109c80 2024-04-25T18:47:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.900+0000 7f6c60000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58119430 con 0x7f6c581092d0 2024-04-25T18:47:09.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.900+0000 7f6c5d600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c581092d0 0x7f6c581a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.900+0000 7f6c5d600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c581092d0 0x7f6c581a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:58334/0 (socket says 172.21.15.112:58334) 2024-04-25T18:47:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.900+0000 7f6c5d600700 1 -- 172.21.15.112:0/45045960 learned_addr learned my addr 172.21.15.112:0/45045960 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.901+0000 7f6c5cc00700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58109c80 0x7f6c581a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.901+0000 7f6c5e000700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c5810e3b0 0x7f6c5819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.901+0000 7f6c5d600700 1 -- 172.21.15.112:0/45045960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58109c80 msgr2=0x7f6c581a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.901+0000 7f6c5d600700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58109c80 0x7f6c581a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.901+0000 7f6c5d600700 1 -- 172.21.15.112:0/45045960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c5810e3b0 msgr2=0x7f6c5819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.901+0000 7f6c5d600700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c5810e3b0 0x7f6c5819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.901+0000 7f6c5d600700 1 -- 172.21.15.112:0/45045960 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5819fd50 con 0x7f6c581092d0 2024-04-25T18:47:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.901+0000 7f6c5cc00700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58109c80 0x7f6c581a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:09.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.901+0000 7f6c5d600700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c581092d0 0x7f6c581a53e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c005910 tx=0x7f6c4c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.902+0000 7f6c56000700 1 -- 172.21.15.112:0/45045960 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c026070 con 0x7f6c581092d0 2024-04-25T18:47:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.902+0000 7f6c60000700 1 -- 172.21.15.112:0/45045960 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5819ffe0 con 0x7f6c581092d0 2024-04-25T18:47:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.902+0000 7f6c56000700 1 -- 172.21.15.112:0/45045960 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c4c003c70 con 0x7f6c581092d0 2024-04-25T18:47:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.902+0000 7f6c56000700 1 -- 172.21.15.112:0/45045960 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00fae0 con 0x7f6c581092d0 2024-04-25T18:47:09.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.902+0000 7f6c60000700 1 -- 172.21.15.112:0/45045960 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c581ac2b0 con 0x7f6c581092d0 2024-04-25T18:47:09.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.905+0000 7f6c56000700 1 -- 172.21.15.112:0/45045960 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f6c4c029070 con 0x7f6c581092d0 2024-04-25T18:47:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.906+0000 7f6c6a933b80 1 Processor -- start 2024-04-25T18:47:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.906+0000 7f6c6a933b80 1 -- start start 2024-04-25T18:47:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c56000700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6c400a3c40 0x7f6c400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c0d2f0 0x561a37c0d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c56000700 1 -- 172.21.15.112:0/45045960 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f6c4c0d0be0 con 0x7f6c581092d0 2024-04-25T18:47:09.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0dc10 0x561a37c18570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c18ab0 0x561a37c1aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c5e000700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6c400a3c40 0x7f6c400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a37b60e40 con 0x561a37c0dc10 2024-04-25T18:47:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561a37b076e0 con 0x561a37c18ab0 2024-04-25T18:47:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561a37b5ce00 con 0x561a37c0d2f0 2024-04-25T18:47:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c5cc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0dc10 0x561a37c18570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c5cc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0dc10 0x561a37c18570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59954/0 (socket says 172.21.15.112:59954) 2024-04-25T18:47:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c5cc00700 1 -- 172.21.15.112:0/2545568704 learned_addr learned my addr 172.21.15.112:0/2545568704 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.907+0000 7f6c5d600700 1 --2- 172.21.15.112:0/2545568704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c0d2f0 0x561a37c0d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c5e000700 1 --2- 172.21.15.112:0/2545568704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c18ab0 0x561a37c1aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c5d600700 1 -- 172.21.15.112:0/2545568704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c18ab0 msgr2=0x561a37c1aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c5d600700 1 --2- 172.21.15.112:0/2545568704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c18ab0 0x561a37c1aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c5d600700 1 -- 172.21.15.112:0/2545568704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0dc10 msgr2=0x561a37c18570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c5d600700 1 --2- 172.21.15.112:0/2545568704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0dc10 0x561a37c18570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c5d600700 1 -- 172.21.15.112:0/2545568704 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a37afd630 con 0x561a37c0d2f0 2024-04-25T18:47:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c5cc00700 1 --2- 172.21.15.112:0/2545568704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0dc10 0x561a37c18570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c5e000700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6c400a3c40 0x7f6c400a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009fd0 tx=0x7f6c5000fc60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c5d600700 1 --2- 172.21.15.112:0/2545568704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c0d2f0 0x561a37c0d6d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009fd0 tx=0x7f6c4c0036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a66c29b974b265d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c4aa00700 1 -- 172.21.15.112:0/2545568704 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c09c830 con 0x561a37c0d2f0 2024-04-25T18:47:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c4aa00700 1 -- 172.21.15.112:0/2545568704 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c4c003e20 con 0x561a37c0d2f0 2024-04-25T18:47:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.908+0000 7f6c4aa00700 1 -- 172.21.15.112:0/2545568704 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c09d350 con 0x561a37c0d2f0 2024-04-25T18:47:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.909+0000 7f6c6a933b80 1 -- 172.21.15.112:0/2545568704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c0d2f0 msgr2=0x561a37c0d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.909+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/2545568704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c0d2f0 0x561a37c0d6d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009fd0 tx=0x7f6c4c0036f0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.909+0000 7f6c6a933b80 1 -- 172.21.15.112:0/2545568704 shutdown_connections 2024-04-25T18:47:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.909+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/2545568704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c18ab0 0x561a37c1aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.909+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/2545568704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c0d2f0 0x561a37c0d6d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.909+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/2545568704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0dc10 0x561a37c18570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.909+0000 7f6c6a933b80 1 -- 172.21.15.112:0/2545568704 >> 172.21.15.112:0/2545568704 conn(0x561a37b3c550 msgr2=0x561a37c09790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.909+0000 7f6c6a933b80 1 -- 172.21.15.112:0/2545568704 shutdown_connections 2024-04-25T18:47:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.909+0000 7f6c6a933b80 1 -- 172.21.15.112:0/2545568704 wait complete. 2024-04-25T18:47:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.910+0000 7f6c6a933b80 1 -- start start 2024-04-25T18:47:09.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.910+0000 7f6c6a933b80 1 Processor -- start 2024-04-25T18:47:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c0d2f0 0x561a37c18280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c15580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c18ab0 0x561a37c15ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a37c1b3e0 con 0x561a37c18ab0 2024-04-25T18:47:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561a37b1a810 con 0x561a37c0dc10 2024-04-25T18:47:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561a37c1b560 con 0x561a37c0d2f0 2024-04-25T18:47:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c6a933b80 10 client.0 _ll_register_callbacks cb 0x561a37af96d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:47:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c5e000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c18ab0 0x561a37c15ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c5e000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c18ab0 0x561a37c15ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59962/0 (socket says 172.21.15.112:59962) 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c5e000700 1 -- 172.21.15.112:0/524635678 learned_addr learned my addr 172.21.15.112:0/524635678 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c5cc00700 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c15580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c5d600700 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c0d2f0 0x561a37c18280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c5e000700 1 -- 172.21.15.112:0/524635678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c0d2f0 msgr2=0x561a37c18280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c5e000700 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c0d2f0 0x561a37c18280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c5e000700 1 -- 172.21.15.112:0/524635678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 msgr2=0x561a37c15580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c5e000700 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c15580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.911+0000 7f6c5e000700 1 -- 172.21.15.112:0/524635678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a37aae880 con 0x561a37c18ab0 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.912+0000 7f6c5e000700 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c18ab0 0x561a37c15ac0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6c500188c0 tx=0x7f6c50004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac22177133dc576b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.912+0000 7f6c3f400700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.912+0000 7f6c3f400700 1 -- 172.21.15.112:0/524635678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50004230 con 0x561a37c18ab0 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.912+0000 7f6c3f400700 1 -- 172.21.15.112:0/524635678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f6c500043d0 con 0x561a37c18ab0 2024-04-25T18:47:09.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.912+0000 7f6c6a933b80 1 -- 172.21.15.112:0/524635678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a37b6e630 con 0x561a37c18ab0 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.912+0000 7f6c3f400700 1 -- 172.21.15.112:0/524635678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6c50004610 con 0x561a37c18ab0 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.912+0000 7f6c6a933b80 10 client.14883 fetch_fsmap learned FSMap version 25 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.912+0000 7f6c6a933b80 1 -- 172.21.15.112:0/524635678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561a37af7aa0 con 0x561a37c18ab0 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.913+0000 7f6c3f400700 1 -- 172.21.15.112:0/524635678 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561a37b6e630 con 0x561a37c18ab0 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.913+0000 7f6c3f400700 1 -- 172.21.15.112:0/524635678 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6c5001a440 con 0x561a37c18ab0 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.913+0000 7f6c6a933b80 10 client.14883 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.913+0000 7f6c6a933b80 10 client.14883 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.913+0000 7f6c6a933b80 20 client.14883 populate_metadata read hostname 'smithi112' 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.913+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x561a37c280c0 0x561a37c29530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.913+0000 7f6c6a933b80 4 client.14883 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.913+0000 7f6c6a933b80 1 -- 172.21.15.112:0/524635678 --> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a37b686d0 con 0x561a37c280c0 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.913+0000 7f6c5d600700 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x561a37c280c0 0x561a37c29530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.914+0000 7f6c5d600700 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x561a37c280c0 0x561a37c29530 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21324e2d568c455b server_cookie=cc663d6fe7017487 in_seq=0 out_seq=0 2024-04-25T18:47:09.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.914+0000 7f6c3f400700 10 client.14883 ms_handle_connect on v2:172.21.15.112:6840/1028859850 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c3f400700 1 -- 172.21.15.112:0/524635678 <== mds.1 v2:172.21.15.112:6840/1028859850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a37b686d0 con 0x561a37c280c0 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c3f400700 10 client.14883 handle_command_reply: tid=0 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c6a933b80 1 client.14883 shutdown 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c6a933b80 1 -- 172.21.15.112:0/524635678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c18ab0 msgr2=0x561a37c15ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c18ab0 0x561a37c15ac0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6c500188c0 tx=0x7f6c50004b40 comp rx=0 tx=0).stop 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c6a933b80 1 -- 172.21.15.112:0/524635678 shutdown_connections 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c18ab0 0x561a37c15ac0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c0d2f0 0x561a37c18280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x561a37c280c0 0x561a37c29530 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/524635678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c15580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c6a933b80 1 -- 172.21.15.112:0/524635678 >> 172.21.15.112:0/524635678 conn(0x561a37b3c550 msgr2=0x561a37c0fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:09.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c3f400700 0 client.14883 ms_handle_reset on v2:172.21.15.112:6840/1028859850 2024-04-25T18:47:09.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c6a933b80 1 -- 172.21.15.112:0/524635678 shutdown_connections 2024-04-25T18:47:09.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.915+0000 7f6c6a933b80 1 -- 172.21.15.112:0/524635678 wait complete. 2024-04-25T18:47:09.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.916+0000 7f6c6a933b80 20 client.14883 trim_cache size 0 max 16384 2024-04-25T18:47:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.938+0000 7f6c6a933b80 1 Processor -- start 2024-04-25T18:47:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.938+0000 7f6c6a933b80 1 -- start start 2024-04-25T18:47:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.938+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 0x561a37c15bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.938+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c098e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.938+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c18ab0 0x561a37c23f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.938+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a37c27870 con 0x561a37c0d2f0 2024-04-25T18:47:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.938+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561a37c27f40 con 0x561a37c0dc10 2024-04-25T18:47:09.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.938+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561a37c28610 con 0x561a37c18ab0 2024-04-25T18:47:09.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.939+0000 7f6c5cc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.939+0000 7f6c5e000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c18ab0 0x561a37c23f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.939+0000 7f6c5cc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c098e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:46414/0 (socket says 172.21.15.112:46414) 2024-04-25T18:47:09.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.939+0000 7f6c5cc00700 1 -- 172.21.15.112:0/1572329761 learned_addr learned my addr 172.21.15.112:0/1572329761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:09.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.939+0000 7f6c5d600700 1 --2- 172.21.15.112:0/1572329761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 0x561a37c15bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.939+0000 7f6c5cc00700 1 -- 172.21.15.112:0/1572329761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c18ab0 msgr2=0x561a37c23f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.939+0000 7f6c5cc00700 1 --2- 172.21.15.112:0/1572329761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c18ab0 0x561a37c23f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.939+0000 7f6c5cc00700 1 -- 172.21.15.112:0/1572329761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 msgr2=0x561a37c15bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.939+0000 7f6c5cc00700 1 --2- 172.21.15.112:0/1572329761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 0x561a37c15bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.939+0000 7f6c5cc00700 1 -- 172.21.15.112:0/1572329761 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a37c00f70 con 0x561a37c0dc10 2024-04-25T18:47:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.939+0000 7f6c5cc00700 1 --2- 172.21.15.112:0/1572329761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c098e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6c440129e0 tx=0x7f6c44009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2858f2a42de0bc17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c4a000700 1 -- 172.21.15.112:0/1572329761 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44012190 con 0x561a37c0dc10 2024-04-25T18:47:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c4a000700 1 -- 172.21.15.112:0/1572329761 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c44004ce0 con 0x561a37c0dc10 2024-04-25T18:47:09.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c4a000700 1 -- 172.21.15.112:0/1572329761 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400f5c0 con 0x561a37c0dc10 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c6a933b80 1 -- 172.21.15.112:0/1572329761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 msgr2=0x561a37c098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/1572329761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c098e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6c440129e0 tx=0x7f6c44009260 comp rx=0 tx=0).stop 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c6a933b80 1 -- 172.21.15.112:0/1572329761 shutdown_connections 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/1572329761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c18ab0 0x561a37c23f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/1572329761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 0x561a37c15bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/1572329761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c098e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c6a933b80 1 -- 172.21.15.112:0/1572329761 >> 172.21.15.112:0/1572329761 conn(0x561a37b3c550 msgr2=0x561a37c091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c6a933b80 1 -- 172.21.15.112:0/1572329761 shutdown_connections 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.940+0000 7f6c6a933b80 1 -- 172.21.15.112:0/1572329761 wait complete. 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.941+0000 7f6c6a933b80 1 -- start start 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.941+0000 7f6c6a933b80 1 Processor -- start 2024-04-25T18:47:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 0x561a37c2c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c2c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c6a933b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c18ab0 0x561a37c2cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a37c293b0 con 0x561a37c0d2f0 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561a37c28ce0 con 0x561a37c0dc10 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c6a933b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561a37c29a80 con 0x561a37c18ab0 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c6a933b80 10 client.0 _ll_register_callbacks cb 0x561a37af96d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c5d600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 0x561a37c2c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c5d600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 0x561a37c2c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59988/0 (socket says 172.21.15.112:59988) 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c5d600700 1 -- 172.21.15.112:0/169157462 learned_addr learned my addr 172.21.15.112:0/169157462 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c5cc00700 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c2c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c5e000700 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c18ab0 0x561a37c2cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c5d600700 1 -- 172.21.15.112:0/169157462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c18ab0 msgr2=0x561a37c2cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c5d600700 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c18ab0 0x561a37c2cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c5d600700 1 -- 172.21.15.112:0/169157462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 msgr2=0x561a37c2c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:09.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c5d600700 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c2c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.942+0000 7f6c5d600700 1 -- 172.21.15.112:0/169157462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a37c30ac0 con 0x561a37c0d2f0 2024-04-25T18:47:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.943+0000 7f6c5d600700 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 0x561a37c2c300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c099330 tx=0x7f6c4c099220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbbc64c04c3faef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.943+0000 7f6c3f400700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:47:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.943+0000 7f6c3f400700 1 -- 172.21.15.112:0/169157462 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c0b5070 con 0x561a37c0d2f0 2024-04-25T18:47:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.943+0000 7f6c3f400700 1 -- 172.21.15.112:0/169157462 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f6c4c09b1d0 con 0x561a37c0d2f0 2024-04-25T18:47:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.943+0000 7f6c6a933b80 1 -- 172.21.15.112:0/169157462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c44004ce0 con 0x561a37c0d2f0 2024-04-25T18:47:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.943+0000 7f6c3f400700 1 -- 172.21.15.112:0/169157462 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6c4c09cca0 con 0x561a37c0d2f0 2024-04-25T18:47:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.943+0000 7f6c6a933b80 10 client.14889 fetch_fsmap learned FSMap version 25 2024-04-25T18:47:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.943+0000 7f6c6a933b80 1 -- 172.21.15.112:0/169157462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561a37c8f470 con 0x561a37c0d2f0 2024-04-25T18:47:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.943+0000 7f6c3f400700 1 -- 172.21.15.112:0/169157462 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c44004ce0 con 0x561a37c0d2f0 2024-04-25T18:47:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.944+0000 7f6c3f400700 1 -- 172.21.15.112:0/169157462 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6c4c09d130 con 0x561a37c0d2f0 2024-04-25T18:47:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.944+0000 7f6c6a933b80 10 client.14889 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.944+0000 7f6c6a933b80 10 client.14889 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.944+0000 7f6c6a933b80 20 client.14889 populate_metadata read hostname 'smithi112' 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.944+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x561a37c93de0 0x561a37c92af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.944+0000 7f6c6a933b80 4 client.14889 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.944+0000 7f6c6a933b80 1 -- 172.21.15.112:0/169157462 --> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] -- command(tid 0: {"prefix": "status"}) -- 0x561a379e1200 con 0x561a37c93de0 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.944+0000 7f6c5cc00700 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x561a37c93de0 0x561a37c92af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c5cc00700 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x561a37c93de0 0x561a37c92af0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8d1b117da3f95c5 server_cookie=9d09b070da19404a in_seq=0 out_seq=0 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c3f400700 10 client.14889 ms_handle_connect on v2:172.21.15.112:6840/1028859850 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c3f400700 1 -- 172.21.15.112:0/169157462 <== mds.1 v2:172.21.15.112:6840/1028859850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561a379e1200 con 0x561a37c93de0 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c3f400700 10 client.14889 handle_command_reply: tid=0 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c6a933b80 1 client.14889 shutdown 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c6a933b80 1 -- 172.21.15.112:0/169157462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 msgr2=0x561a37c2c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 0x561a37c2c300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c099330 tx=0x7f6c4c099220 comp rx=0 tx=0).stop 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c6a933b80 1 -- 172.21.15.112:0/169157462 shutdown_connections 2024-04-25T18:47:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a37c18ab0 0x561a37c2cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a37c0d2f0 0x561a37c2c300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x561a37c93de0 0x561a37c92af0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c6a933b80 1 --2- 172.21.15.112:0/169157462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a37c0dc10 0x561a37c2c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c6a933b80 1 -- 172.21.15.112:0/169157462 >> 172.21.15.112:0/169157462 conn(0x561a37b3c550 msgr2=0x561a37c08950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c3f400700 0 client.14889 ms_handle_reset on v2:172.21.15.112:6840/1028859850 2024-04-25T18:47:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.945+0000 7f6c6a933b80 1 -- 172.21.15.112:0/169157462 shutdown_connections 2024-04-25T18:47:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.946+0000 7f6c6a933b80 1 -- 172.21.15.112:0/169157462 wait complete. 2024-04-25T18:47:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.946+0000 7f6c6a933b80 20 client.14889 trim_cache size 0 max 16384 2024-04-25T18:47:09.949 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:47:09.949 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:47:09.949 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-04-25T18:47:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24511, 2024-04-25T18:47:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T18:47:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T18:47:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T18:47:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 212.73565382999999, 2024-04-25T18:47:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 25, 2024-04-25T18:47:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 94, 2024-04-25T18:47:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:47:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 256.31989669000001 2024-04-25T18:47:09.950 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:47:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.946+0000 7f6c60000700 1 -- 172.21.15.112:0/45045960 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6c400a3c40 msgr2=0x7f6c400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.946+0000 7f6c60000700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6c400a3c40 0x7f6c400a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009fd0 tx=0x7f6c5000fc60 comp rx=0 tx=0).stop 2024-04-25T18:47:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.946+0000 7f6c60000700 1 -- 172.21.15.112:0/45045960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c581092d0 msgr2=0x7f6c581a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.946+0000 7f6c60000700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c581092d0 0x7f6c581a53e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c005910 tx=0x7f6c4c004270 comp rx=0 tx=0).stop 2024-04-25T18:47:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.947+0000 7f6c60000700 1 -- 172.21.15.112:0/45045960 shutdown_connections 2024-04-25T18:47:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.947+0000 7f6c60000700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f6c400a3c40 0x7f6c400a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.947+0000 7f6c60000700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c5810e3b0 0x7f6c5819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.947+0000 7f6c60000700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c581092d0 0x7f6c581a53e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.947+0000 7f6c60000700 1 --2- 172.21.15.112:0/45045960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c58109c80 0x7f6c581a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.947+0000 7f6c60000700 1 -- 172.21.15.112:0/45045960 >> 172.21.15.112:0/45045960 conn(0x7f6c58076210 msgr2=0x7f6c58077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.947+0000 7f6c60000700 1 -- 172.21.15.112:0/45045960 shutdown_connections 2024-04-25T18:47:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:09.947+0000 7f6c60000700 1 -- 172.21.15.112:0/45045960 wait complete. 2024-04-25T18:47:10.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:10 smithi171 ceph-mon[33266]: pgmap v451: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:47:10.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:10 smithi161 ceph-mon[32609]: pgmap v451: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:47:10.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:10 smithi112 ceph-mon[27864]: pgmap v451: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:47:10.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T18:47:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:12.576 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:12 smithi112 ceph-mon[27864]: pgmap v452: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:47:12.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:12 smithi171 ceph-mon[33266]: pgmap v452: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:47:12.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:12 smithi161 ceph-mon[32609]: pgmap v452: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:47:13.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.641+0000 7f3731200700 1 -- 172.21.15.112:0/4256802490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c06a750 msgr2=0x7f372c06ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.641+0000 7f3731200700 1 --2- 172.21.15.112:0/4256802490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c06a750 0x7f372c06ab30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f371c009990 tx=0x7f371c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.641+0000 7f3731200700 1 -- 172.21.15.112:0/4256802490 shutdown_connections 2024-04-25T18:47:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.641+0000 7f3731200700 1 --2- 172.21.15.112:0/4256802490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c10f4f0 0x7f372c1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.641+0000 7f3731200700 1 --2- 172.21.15.112:0/4256802490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c06a750 0x7f372c06ab30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.641+0000 7f3731200700 1 --2- 172.21.15.112:0/4256802490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f372c06b070 0x7f372c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.641+0000 7f3731200700 1 -- 172.21.15.112:0/4256802490 >> 172.21.15.112:0/4256802490 conn(0x7f372c0788f0 msgr2=0x7f372c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.641+0000 7f3731200700 1 -- 172.21.15.112:0/4256802490 shutdown_connections 2024-04-25T18:47:13.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.641+0000 7f3731200700 1 -- 172.21.15.112:0/4256802490 wait complete. 2024-04-25T18:47:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.642+0000 7f3731200700 1 Processor -- start 2024-04-25T18:47:13.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.642+0000 7f3731200700 1 -- start start 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.643+0000 7f3731200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c06a750 0x7f372c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.643+0000 7f3731200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f372c06b070 0x7f372c19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.643+0000 7f3731200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c10f4f0 0x7f372c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.643+0000 7f3731200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f372c114e50 con 0x7f372c06a750 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.643+0000 7f3731200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f372c114cd0 con 0x7f372c10f4f0 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.643+0000 7f3731200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f372c114fd0 con 0x7f372c06b070 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.643+0000 7f3729e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c06a750 0x7f372c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.643+0000 7f3729e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c06a750 0x7f372c19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60012/0 (socket says 172.21.15.112:60012) 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.643+0000 7f3729e00700 1 -- 172.21.15.112:0/3672741947 learned_addr learned my addr 172.21.15.112:0/3672741947 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.643+0000 7f3729400700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f372c06b070 0x7f372c19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.643+0000 7f372a800700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c10f4f0 0x7f372c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.644+0000 7f3729e00700 1 -- 172.21.15.112:0/3672741947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f372c06b070 msgr2=0x7f372c19b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.644+0000 7f3729e00700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f372c06b070 0x7f372c19b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.644+0000 7f3729e00700 1 -- 172.21.15.112:0/3672741947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c10f4f0 msgr2=0x7f372c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.644+0000 7f3729e00700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c10f4f0 0x7f372c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.644+0000 7f3729e00700 1 -- 172.21.15.112:0/3672741947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c1a0430 con 0x7f372c06a750 2024-04-25T18:47:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.644+0000 7f3729e00700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c06a750 0x7f372c19b420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f371c005910 tx=0x7f371c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.644+0000 7f3722a00700 1 -- 172.21.15.112:0/3672741947 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c027070 con 0x7f372c06a750 2024-04-25T18:47:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.644+0000 7f3722a00700 1 -- 172.21.15.112:0/3672741947 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f371c022470 con 0x7f372c06a750 2024-04-25T18:47:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.644+0000 7f3722a00700 1 -- 172.21.15.112:0/3672741947 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c017630 con 0x7f372c06a750 2024-04-25T18:47:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.644+0000 7f3731200700 1 -- 172.21.15.112:0/3672741947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c1a06c0 con 0x7f372c06a750 2024-04-25T18:47:13.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.645+0000 7f3731200700 1 -- 172.21.15.112:0/3672741947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c1a7fa0 con 0x7f372c06a750 2024-04-25T18:47:13.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.648+0000 7f3722a00700 1 -- 172.21.15.112:0/3672741947 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f371c02a070 con 0x7f372c06a750 2024-04-25T18:47:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.649+0000 7f3722a00700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f37100a3c40 0x7f37100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.649+0000 7f3722a00700 1 -- 172.21.15.112:0/3672741947 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f371c0d1090 con 0x7f372c06a750 2024-04-25T18:47:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.650+0000 7f373bacfb80 1 Processor -- start 2024-04-25T18:47:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.650+0000 7f3729400700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f37100a3c40 0x7f37100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.650+0000 7f373bacfb80 1 -- start start 2024-04-25T18:47:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.650+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.650+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d30a0 0x55a9ff0cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.650+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.650+0000 7f373bacfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9ff01ee40 con 0x55a9ff0d30a0 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.650+0000 7f373bacfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a9fefc56e0 con 0x55a9ff0ccef0 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.650+0000 7f373bacfb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a9ff01ae00 con 0x55a9ff0d2780 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.650+0000 7f3729400700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f37100a3c40 0x7f37100a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f372c103050 tx=0x7f3718009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f3729e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f3729e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:58408/0 (socket says 172.21.15.112:58408) 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f3729e00700 1 -- 172.21.15.112:0/3788473338 learned_addr learned my addr 172.21.15.112:0/3788473338 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f3729400700 1 --2- 172.21.15.112:0/3788473338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d30a0 0x55a9ff0cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f372a800700 1 --2- 172.21.15.112:0/3788473338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f3729e00700 1 -- 172.21.15.112:0/3788473338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0ccef0 msgr2=0x55a9ff0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f3729e00700 1 --2- 172.21.15.112:0/3788473338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f3729e00700 1 -- 172.21.15.112:0/3788473338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d30a0 msgr2=0x55a9ff0cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f3729e00700 1 --2- 172.21.15.112:0/3788473338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d30a0 0x55a9ff0cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f3729e00700 1 -- 172.21.15.112:0/3788473338 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9fefbb630 con 0x55a9ff0d2780 2024-04-25T18:47:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f372a800700 1 --2- 172.21.15.112:0/3788473338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:47:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f3729400700 1 --2- 172.21.15.112:0/3788473338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d30a0 0x55a9ff0cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:47:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.651+0000 7f3729e00700 1 --2- 172.21.15.112:0/3788473338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d2b60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f371c09d370 tx=0x7f371c09d230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc5b265356f94128 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.652+0000 7f3717400700 1 -- 172.21.15.112:0/3788473338 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c0b6070 con 0x55a9ff0d2780 2024-04-25T18:47:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.652+0000 7f3717400700 1 -- 172.21.15.112:0/3788473338 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f371c09dae0 con 0x55a9ff0d2780 2024-04-25T18:47:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.652+0000 7f3717400700 1 -- 172.21.15.112:0/3788473338 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c09d640 con 0x55a9ff0d2780 2024-04-25T18:47:13.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.652+0000 7f373bacfb80 1 -- 172.21.15.112:0/3788473338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 msgr2=0x55a9ff0d2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:13.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.652+0000 7f373bacfb80 1 --2- 172.21.15.112:0/3788473338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d2b60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f371c09d370 tx=0x7f371c09d230 comp rx=0 tx=0).stop 2024-04-25T18:47:13.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.653+0000 7f373bacfb80 1 -- 172.21.15.112:0/3788473338 shutdown_connections 2024-04-25T18:47:13.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.653+0000 7f373bacfb80 1 --2- 172.21.15.112:0/3788473338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.653+0000 7f373bacfb80 1 --2- 172.21.15.112:0/3788473338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d2b60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.653+0000 7f373bacfb80 1 --2- 172.21.15.112:0/3788473338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d30a0 0x55a9ff0cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.653+0000 7f373bacfb80 1 -- 172.21.15.112:0/3788473338 >> 172.21.15.112:0/3788473338 conn(0x55a9feffaf00 msgr2=0x55a9ff0c78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.653+0000 7f373bacfb80 1 -- 172.21.15.112:0/3788473338 shutdown_connections 2024-04-25T18:47:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.653+0000 7f373bacfb80 1 -- 172.21.15.112:0/3788473338 wait complete. 2024-04-25T18:47:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.653+0000 7f373bacfb80 1 -- start start 2024-04-25T18:47:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.653+0000 7f373bacfb80 1 Processor -- start 2024-04-25T18:47:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0d8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d6530 0x55a9ff0d69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f373bacfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9ff0d96d0 con 0x55a9ff0ccef0 2024-04-25T18:47:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f373bacfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a9fefd8810 con 0x55a9ff0d2780 2024-04-25T18:47:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f373bacfb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a9ff0d9850 con 0x55a9ff0d6530 2024-04-25T18:47:13.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f373bacfb80 10 client.0 _ll_register_callbacks cb 0x55a9fefb76d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f3729e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0d8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f3729e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0d8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60034/0 (socket says 172.21.15.112:60034) 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f3729e00700 1 -- 172.21.15.112:0/354013238 learned_addr learned my addr 172.21.15.112:0/354013238 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f372a800700 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d6530 0x55a9ff0d69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.654+0000 7f3729400700 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.655+0000 7f3729e00700 1 -- 172.21.15.112:0/354013238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d6530 msgr2=0x55a9ff0d69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.655+0000 7f3729e00700 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d6530 0x55a9ff0d69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.655+0000 7f3729e00700 1 -- 172.21.15.112:0/354013238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0d2780 msgr2=0x55a9ff0d5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.655+0000 7f3729e00700 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.655+0000 7f3729e00700 1 -- 172.21.15.112:0/354013238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9fef6c880 con 0x55a9ff0ccef0 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.655+0000 7f372a800700 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d6530 0x55a9ff0d69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.655+0000 7f3729e00700 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0d8c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f371c09d0e0 tx=0x7f371c09d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a300f6c89cb5427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:13.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.655+0000 7f370be00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.655+0000 7f370be00700 1 -- 172.21.15.112:0/354013238 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c0b6070 con 0x55a9ff0ccef0 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.655+0000 7f370be00700 1 -- 172.21.15.112:0/354013238 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f371c09b6e0 con 0x55a9ff0ccef0 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.655+0000 7f373bacfb80 1 -- 172.21.15.112:0/354013238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371c09dae0 con 0x55a9ff0ccef0 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.656+0000 7f370be00700 1 -- 172.21.15.112:0/354013238 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f371c09d640 con 0x55a9ff0ccef0 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.656+0000 7f373bacfb80 10 client.14901 fetch_fsmap learned FSMap version 25 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.656+0000 7f373bacfb80 1 -- 172.21.15.112:0/354013238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a9fefb5aa0 con 0x55a9ff0ccef0 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.656+0000 7f370be00700 1 -- 172.21.15.112:0/354013238 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f371c09dae0 con 0x55a9ff0ccef0 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.656+0000 7f370be00700 1 -- 172.21.15.112:0/354013238 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f371c099130 con 0x55a9ff0ccef0 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.656+0000 7f373bacfb80 10 client.14901 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.656+0000 7f373bacfb80 10 client.14901 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.656+0000 7f373bacfb80 20 client.14901 populate_metadata read hostname 'smithi112' 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.656+0000 7f373bacfb80 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55a9ff0e62e0 0x55a9ff0e7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.656+0000 7f373bacfb80 4 client.14901 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:47:13.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.656+0000 7f373bacfb80 1 -- 172.21.15.112:0/354013238 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9ff02c630 con 0x55a9ff0e62e0 2024-04-25T18:47:13.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.657+0000 7f3729400700 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55a9ff0e62e0 0x55a9ff0e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.657+0000 7f3729400700 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55a9ff0e62e0 0x55a9ff0e7750 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c354ffd6f58365d server_cookie=2f9b286a2888a0c5 in_seq=0 out_seq=0 2024-04-25T18:47:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.657+0000 7f370be00700 10 client.14901 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f370be00700 1 -- 172.21.15.112:0/354013238 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9ff02c630 con 0x55a9ff0e62e0 2024-04-25T18:47:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f370be00700 10 client.14901 handle_command_reply: tid=0 2024-04-25T18:47:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f373bacfb80 1 client.14901 shutdown 2024-04-25T18:47:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f373bacfb80 1 -- 172.21.15.112:0/354013238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 msgr2=0x55a9ff0d8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f373bacfb80 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0d8c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f371c09d0e0 tx=0x7f371c09d4f0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f373bacfb80 1 -- 172.21.15.112:0/354013238 shutdown_connections 2024-04-25T18:47:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f373bacfb80 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55a9ff0e62e0 0x55a9ff0e7750 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f373bacfb80 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d6530 0x55a9ff0d69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f373bacfb80 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0d8c80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f373bacfb80 1 --2- 172.21.15.112:0/354013238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f373bacfb80 1 -- 172.21.15.112:0/354013238 >> 172.21.15.112:0/354013238 conn(0x55a9feffaf00 msgr2=0x55a9ff0c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f370be00700 0 client.14901 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f373bacfb80 1 -- 172.21.15.112:0/354013238 shutdown_connections 2024-04-25T18:47:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.658+0000 7f373bacfb80 1 -- 172.21.15.112:0/354013238 wait complete. 2024-04-25T18:47:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.659+0000 7f373bacfb80 20 client.14901 trim_cache size 0 max 16384 2024-04-25T18:47:13.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.681+0000 7f373bacfb80 1 Processor -- start 2024-04-25T18:47:13.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.681+0000 7f373bacfb80 1 -- start start 2024-04-25T18:47:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.681+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.681+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.681+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d6530 0x55a9ff0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.681+0000 7f373bacfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9ff0e5a90 con 0x55a9ff0d2780 2024-04-25T18:47:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.681+0000 7f373bacfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a9ff0e6160 con 0x55a9ff0ccef0 2024-04-25T18:47:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.681+0000 7f373bacfb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a9ff0e6830 con 0x55a9ff0d6530 2024-04-25T18:47:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.682+0000 7f3729400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.682+0000 7f3729400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60038/0 (socket says 172.21.15.112:60038) 2024-04-25T18:47:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.682+0000 7f3729400700 1 -- 172.21.15.112:0/1612893992 learned_addr learned my addr 172.21.15.112:0/1612893992 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:13.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.682+0000 7f372a800700 1 --2- 172.21.15.112:0/1612893992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d6530 0x55a9ff0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.682+0000 7f3729e00700 1 --2- 172.21.15.112:0/1612893992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.682+0000 7f3729400700 1 -- 172.21.15.112:0/1612893992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d6530 msgr2=0x55a9ff0c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.682+0000 7f3729400700 1 --2- 172.21.15.112:0/1612893992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d6530 0x55a9ff0c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.682+0000 7f3729400700 1 -- 172.21.15.112:0/1612893992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0ccef0 msgr2=0x55a9ff0e2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.682+0000 7f3729400700 1 --2- 172.21.15.112:0/1612893992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0e2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.682+0000 7f3729400700 1 -- 172.21.15.112:0/1612893992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9fefb5aa0 con 0x55a9ff0d2780 2024-04-25T18:47:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.682+0000 7f3729400700 1 --2- 172.21.15.112:0/1612893992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d5dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3718012fd0 tx=0x7f3718014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de5f6a551b05ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.683+0000 7f3716a00700 1 -- 172.21.15.112:0/1612893992 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37180295b0 con 0x55a9ff0d2780 2024-04-25T18:47:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.683+0000 7f3716a00700 1 -- 172.21.15.112:0/1612893992 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3718004ce0 con 0x55a9ff0d2780 2024-04-25T18:47:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.683+0000 7f3716a00700 1 -- 172.21.15.112:0/1612893992 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718005650 con 0x55a9ff0d2780 2024-04-25T18:47:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.683+0000 7f373bacfb80 1 -- 172.21.15.112:0/1612893992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d2780 msgr2=0x55a9ff0d5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:13.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.683+0000 7f373bacfb80 1 --2- 172.21.15.112:0/1612893992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d5dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3718012fd0 tx=0x7f3718014000 comp rx=0 tx=0).stop 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.683+0000 7f373bacfb80 1 -- 172.21.15.112:0/1612893992 shutdown_connections 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.683+0000 7f373bacfb80 1 --2- 172.21.15.112:0/1612893992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d6530 0x55a9ff0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.683+0000 7f373bacfb80 1 --2- 172.21.15.112:0/1612893992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0e2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.683+0000 7f373bacfb80 1 --2- 172.21.15.112:0/1612893992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0d2780 0x55a9ff0d5dd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.683+0000 7f373bacfb80 1 -- 172.21.15.112:0/1612893992 >> 172.21.15.112:0/1612893992 conn(0x55a9feffaf00 msgr2=0x55a9ff0c3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.684+0000 7f373bacfb80 1 -- 172.21.15.112:0/1612893992 shutdown_connections 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.684+0000 7f373bacfb80 1 -- 172.21.15.112:0/1612893992 wait complete. 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.684+0000 7f373bacfb80 1 -- start start 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.684+0000 7f373bacfb80 1 Processor -- start 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 0x55a9ff0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f373bacfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0d6530 0x55a9ff0eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f373bacfb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9ff0e75d0 con 0x55a9ff0ccef0 2024-04-25T18:47:13.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f373bacfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a9ff0e6f00 con 0x55a9ff0d6530 2024-04-25T18:47:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f373bacfb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a9ff0e7ca0 con 0x55a9ff0d2780 2024-04-25T18:47:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f373bacfb80 10 client.0 _ll_register_callbacks cb 0x55a9fefb76d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:47:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f3729e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f372a800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0d6530 0x55a9ff0eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f3729e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60046/0 (socket says 172.21.15.112:60046) 2024-04-25T18:47:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f3729e00700 1 -- 172.21.15.112:0/2630189401 learned_addr learned my addr 172.21.15.112:0/2630189401 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.685+0000 7f3729400700 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 0x55a9ff0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.686+0000 7f372a800700 1 -- 172.21.15.112:0/2630189401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 msgr2=0x55a9ff0c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.686+0000 7f372a800700 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 0x55a9ff0c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.686+0000 7f372a800700 1 -- 172.21.15.112:0/2630189401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 msgr2=0x55a9ff0c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:13.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.686+0000 7f372a800700 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.686+0000 7f372a800700 1 -- 172.21.15.112:0/2630189401 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9ff0c7a80 con 0x55a9ff0d6530 2024-04-25T18:47:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.686+0000 7f3729e00700 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.686+0000 7f372a800700 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0d6530 0x55a9ff0eefe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f372c06a310 tx=0x7f3724004630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73747236f61ce792 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.686+0000 7f370be00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:47:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.686+0000 7f370be00700 1 -- 172.21.15.112:0/2630189401 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37240088a0 con 0x55a9ff0d6530 2024-04-25T18:47:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.686+0000 7f373bacfb80 1 -- 172.21.15.112:0/2630189401 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3718004ce0 con 0x55a9ff0d6530 2024-04-25T18:47:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.686+0000 7f370be00700 1 -- 172.21.15.112:0/2630189401 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f3724008a40 con 0x55a9ff0d6530 2024-04-25T18:47:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.687+0000 7f370be00700 1 -- 172.21.15.112:0/2630189401 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3724008c40 con 0x55a9ff0d6530 2024-04-25T18:47:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.687+0000 7f373bacfb80 10 client.25016 fetch_fsmap learned FSMap version 25 2024-04-25T18:47:13.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.687+0000 7f373bacfb80 1 -- 172.21.15.112:0/2630189401 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a9ff156270 con 0x55a9ff0d6530 2024-04-25T18:47:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.687+0000 7f370be00700 1 -- 172.21.15.112:0/2630189401 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3718004ce0 con 0x55a9ff0d6530 2024-04-25T18:47:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.687+0000 7f370be00700 1 -- 172.21.15.112:0/2630189401 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3724021420 con 0x55a9ff0d6530 2024-04-25T18:47:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.687+0000 7f373bacfb80 10 client.25016 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:47:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.687+0000 7f373bacfb80 10 client.25016 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T18:47:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.687+0000 7f373bacfb80 20 client.25016 populate_metadata read hostname 'smithi112' 2024-04-25T18:47:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.687+0000 7f373bacfb80 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55a9ff15ae10 0x55a9ff159b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.687+0000 7f373bacfb80 4 client.25016 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:47:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.687+0000 7f373bacfb80 1 -- 172.21.15.112:0/2630189401 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "status"}) -- 0x55a9ff0266d0 con 0x55a9ff15ae10 2024-04-25T18:47:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.688+0000 7f3729e00700 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55a9ff15ae10 0x55a9ff159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.688+0000 7f370be00700 10 client.25016 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:13.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.688+0000 7f3729e00700 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55a9ff15ae10 0x55a9ff159b20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e298f9bfc88aad server_cookie=9a9cb5cae2569cd in_seq=0 out_seq=0 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.688+0000 7f370be00700 1 -- 172.21.15.112:0/2630189401 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a9ff0266d0 con 0x55a9ff15ae10 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.688+0000 7f370be00700 10 client.25016 handle_command_reply: tid=0 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.688+0000 7f373bacfb80 1 client.25016 shutdown 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f373bacfb80 1 -- 172.21.15.112:0/2630189401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0d6530 msgr2=0x55a9ff0eefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f373bacfb80 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0d6530 0x55a9ff0eefe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f372c06a310 tx=0x7f3724004630 comp rx=0 tx=0).stop 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f373bacfb80 1 -- 172.21.15.112:0/2630189401 shutdown_connections 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f373bacfb80 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55a9ff15ae10 0x55a9ff159b20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f373bacfb80 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9ff0d6530 0x55a9ff0eefe0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f370be00700 0 client.25016 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f373bacfb80 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9ff0ccef0 0x55a9ff0c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f373bacfb80 1 --2- 172.21.15.112:0/2630189401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a9ff0d2780 0x55a9ff0c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f373bacfb80 1 -- 172.21.15.112:0/2630189401 >> 172.21.15.112:0/2630189401 conn(0x55a9feffaf00 msgr2=0x55a9ff0d5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f373bacfb80 1 -- 172.21.15.112:0/2630189401 shutdown_connections 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f373bacfb80 1 -- 172.21.15.112:0/2630189401 wait complete. 2024-04-25T18:47:13.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.689+0000 7f373bacfb80 20 client.25016 trim_cache size 0 max 16384 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24512, 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 233.40493655700001, 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 25, 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 94, 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 257.28846297199999 2024-04-25T18:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:47:13.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 -- 172.21.15.112:0/3672741947 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f37100a3c40 msgr2=0x7f37100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:13.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f37100a3c40 0x7f37100a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f372c103050 tx=0x7f3718009260 comp rx=0 tx=0).stop 2024-04-25T18:47:13.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 -- 172.21.15.112:0/3672741947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c06a750 msgr2=0x7f372c19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:13.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c06a750 0x7f372c19b420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f371c005910 tx=0x7f371c00b670 comp rx=0 tx=0).stop 2024-04-25T18:47:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 -- 172.21.15.112:0/3672741947 shutdown_connections 2024-04-25T18:47:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f37100a3c40 0x7f37100a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c10f4f0 0x7f372c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c06a750 0x7f372c19b420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 --2- 172.21.15.112:0/3672741947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f372c06b070 0x7f372c19b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 -- 172.21.15.112:0/3672741947 >> 172.21.15.112:0/3672741947 conn(0x7f372c0788f0 msgr2=0x7f372c110c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 -- 172.21.15.112:0/3672741947 shutdown_connections 2024-04-25T18:47:13.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:13.690+0000 7f3731200700 1 -- 172.21.15.112:0/3672741947 wait complete. 2024-04-25T18:47:14.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:14 smithi112 ceph-mon[27864]: pgmap v453: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:47:14.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:14 smithi171 ceph-mon[33266]: pgmap v453: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:47:14.715 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T18:47:14.725 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T18:47:14.725 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T18:47:14.725 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T18:47:14.725 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T18:47:14.726 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T18:47:14.726 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T18:47:14.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T18:47:14.727 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:14 smithi161 ceph-mon[32609]: pgmap v453: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:47:14.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.931+0000 7f25ce800640 1 -- 172.21.15.112:0/898977548 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25c8104010 msgr2=0x7f25c8104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:14.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.931+0000 7f25ce800640 1 --2- 172.21.15.112:0/898977548 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25c8104010 0x7f25c8104430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009860 tx=0x7f25bc02a340 comp rx=0 tx=0).stop 2024-04-25T18:47:14.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.931+0000 7f25ce800640 1 -- 172.21.15.112:0/898977548 shutdown_connections 2024-04-25T18:47:14.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.931+0000 7f25ce800640 1 --2- 172.21.15.112:0/898977548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25c8105bc0 0x7f25c810c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:14.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.931+0000 7f25ce800640 1 --2- 172.21.15.112:0/898977548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25c8105200 0x7f25c8105680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:14.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.931+0000 7f25ce800640 1 --2- 172.21.15.112:0/898977548 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25c8104010 0x7f25c8104430 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:14.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.931+0000 7f25ce800640 1 -- 172.21.15.112:0/898977548 >> 172.21.15.112:0/898977548 conn(0x7f25c80ff5c0 msgr2=0x7f25c81019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:14.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.931+0000 7f25ce800640 1 -- 172.21.15.112:0/898977548 shutdown_connections 2024-04-25T18:47:14.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.931+0000 7f25ce800640 1 -- 172.21.15.112:0/898977548 wait complete. 2024-04-25T18:47:14.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.932+0000 7f25ce800640 1 Processor -- start 2024-04-25T18:47:14.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25ce800640 1 -- start start 2024-04-25T18:47:14.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25ce800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25c8104010 0x7f25c819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:14.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25ce800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25c8105200 0x7f25c819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:14.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25ce800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25c8105bc0 0x7f25c81a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:14.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25ce800640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25c810fb50 con 0x7f25c8105bc0 2024-04-25T18:47:14.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25ce800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f25c810f9d0 con 0x7f25c8104010 2024-04-25T18:47:14.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25ce800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f25c810fcd0 con 0x7f25c8105200 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25cde00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25c8105bc0 0x7f25c81a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25cde00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25c8105bc0 0x7f25c81a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60066/0 (socket says 172.21.15.112:60066) 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25cd400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25c8104010 0x7f25c819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25cde00640 1 -- 172.21.15.112:0/2152193185 learned_addr learned my addr 172.21.15.112:0/2152193185 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.933+0000 7f25cca00640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25c8105200 0x7f25c819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.934+0000 7f25cde00640 1 -- 172.21.15.112:0/2152193185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25c8105200 msgr2=0x7f25c819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.934+0000 7f25cde00640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25c8105200 0x7f25c819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.934+0000 7f25cde00640 1 -- 172.21.15.112:0/2152193185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25c8104010 msgr2=0x7f25c819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.934+0000 7f25cde00640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25c8104010 0x7f25c819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.934+0000 7f25cde00640 1 -- 172.21.15.112:0/2152193185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c81a1e30 con 0x7f25c8105bc0 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.934+0000 7f25cd400640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25c8104010 0x7f25c819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.934+0000 7f25cde00640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25c8105bc0 0x7f25c81a17c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f25c000eb10 tx=0x7f25c000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.934+0000 7f25c6000640 1 -- 172.21.15.112:0/2152193185 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c000ce20 con 0x7f25c8105bc0 2024-04-25T18:47:14.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.934+0000 7f25ce800640 1 -- 172.21.15.112:0/2152193185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c81a2120 con 0x7f25c8105bc0 2024-04-25T18:47:14.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.934+0000 7f25c6000640 1 -- 172.21.15.112:0/2152193185 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c0005020 con 0x7f25c8105bc0 2024-04-25T18:47:14.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.934+0000 7f25c6000640 1 -- 172.21.15.112:0/2152193185 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0010690 con 0x7f25c8105bc0 2024-04-25T18:47:14.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.935+0000 7f25ce800640 1 -- 172.21.15.112:0/2152193185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c81a26c0 con 0x7f25c8105bc0 2024-04-25T18:47:14.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.936+0000 7f25ce800640 1 -- 172.21.15.112:0/2152193185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c810b350 con 0x7f25c8105bc0 2024-04-25T18:47:14.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.938+0000 7f25c6000640 1 -- 172.21.15.112:0/2152193185 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f25c0005510 con 0x7f25c8105bc0 2024-04-25T18:47:14.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.940+0000 7f25c6000640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f25b00a3ba0 0x7f25b00a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:14.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.940+0000 7f25c6000640 1 -- 172.21.15.112:0/2152193185 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f25c00d09d0 con 0x7f25c8105bc0 2024-04-25T18:47:14.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.941+0000 7f25cd400640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f25b00a3ba0 0x7f25b00a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:14.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.941+0000 7f25cd400640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f25b00a3ba0 0x7f25b00a6060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f25bc02a930 tx=0x7f25bc030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:14.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:14.945+0000 7f25c6000640 1 -- 172.21.15.112:0/2152193185 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25c009d330 con 0x7f25c8105bc0 2024-04-25T18:47:15.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.203+0000 7f25ce800640 1 -- 172.21.15.112:0/2152193185 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f25c8062b50 con 0x7f25c8105bc0 2024-04-25T18:47:15.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.276+0000 7f25c6000640 1 -- 172.21.15.112:0/2152193185 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f25c009cc30 con 0x7f25c8105bc0 2024-04-25T18:47:15.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.280+0000 7f25ce800640 1 -- 172.21.15.112:0/2152193185 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f25b00a3ba0 msgr2=0x7f25b00a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:15.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.280+0000 7f25ce800640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f25b00a3ba0 0x7f25b00a6060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f25bc02a930 tx=0x7f25bc030470 comp rx=0 tx=0).stop 2024-04-25T18:47:15.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.280+0000 7f25ce800640 1 -- 172.21.15.112:0/2152193185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25c8105bc0 msgr2=0x7f25c81a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:15.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.280+0000 7f25ce800640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25c8105bc0 0x7f25c81a17c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f25c000eb10 tx=0x7f25c000ef10 comp rx=0 tx=0).stop 2024-04-25T18:47:15.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.280+0000 7f25ce800640 1 -- 172.21.15.112:0/2152193185 shutdown_connections 2024-04-25T18:47:15.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.280+0000 7f25ce800640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25c8105bc0 0x7f25c81a17c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.280+0000 7f25ce800640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f25b00a3ba0 0x7f25b00a6060 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.280+0000 7f25ce800640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25c8105200 0x7f25c819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.280+0000 7f25ce800640 1 --2- 172.21.15.112:0/2152193185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25c8104010 0x7f25c819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.280+0000 7f25ce800640 1 -- 172.21.15.112:0/2152193185 >> 172.21.15.112:0/2152193185 conn(0x7f25c80ff5c0 msgr2=0x7f25c8108520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:15.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.283+0000 7f25ce800640 1 -- 172.21.15.112:0/2152193185 shutdown_connections 2024-04-25T18:47:15.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.284+0000 7f25ce800640 1 -- 172.21.15.112:0/2152193185 wait complete. 2024-04-25T18:47:15.310 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T18:47:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.488+0000 7fe675800640 1 -- 172.21.15.112:0/1109231654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6701011a0 msgr2=0x7fe670101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.488+0000 7fe675800640 1 --2- 172.21.15.112:0/1109231654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6701011a0 0x7fe670101600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe658009900 tx=0x7fe65802a3f0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.489+0000 7fe675800640 1 -- 172.21.15.112:0/1109231654 shutdown_connections 2024-04-25T18:47:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.489+0000 7fe675800640 1 --2- 172.21.15.112:0/1109231654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe670101b40 0x7fe670101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.489+0000 7fe675800640 1 --2- 172.21.15.112:0/1109231654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6701011a0 0x7fe670101600 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.489+0000 7fe675800640 1 --2- 172.21.15.112:0/1109231654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe670100840 0x7fe670100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.489+0000 7fe675800640 1 -- 172.21.15.112:0/1109231654 >> 172.21.15.112:0/1109231654 conn(0x7fe6700fc460 msgr2=0x7fe6700fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.489+0000 7fe675800640 1 -- 172.21.15.112:0/1109231654 shutdown_connections 2024-04-25T18:47:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.489+0000 7fe675800640 1 -- 172.21.15.112:0/1109231654 wait complete. 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe675800640 1 Processor -- start 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe675800640 1 -- start start 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe675800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe670100840 0x7fe67019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe675800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6701011a0 0x7fe67019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe675800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe670101b40 0x7fe6701a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe675800640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe67010fd40 con 0x7fe670101b40 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe675800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe67010fbc0 con 0x7fe6701011a0 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe675800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe67010fec0 con 0x7fe670100840 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe666000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6701011a0 0x7fe67019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe666000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6701011a0 0x7fe67019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:46512/0 (socket says 172.21.15.112:46512) 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe666000640 1 -- 172.21.15.112:0/2094912794 learned_addr learned my addr 172.21.15.112:0/2094912794 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe66f400640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe670101b40 0x7fe6701a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:15.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.490+0000 7fe66ea00640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe670100840 0x7fe67019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.491+0000 7fe666000640 1 -- 172.21.15.112:0/2094912794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe670100840 msgr2=0x7fe67019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.491+0000 7fe666000640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe670100840 0x7fe67019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.491+0000 7fe666000640 1 -- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe670101b40 msgr2=0x7fe6701a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.491+0000 7fe666000640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe670101b40 0x7fe6701a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.491+0000 7fe666000640 1 -- 172.21.15.112:0/2094912794 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6701a1fe0 con 0x7fe6701011a0 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.491+0000 7fe66ea00640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe670100840 0x7fe67019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.491+0000 7fe666000640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6701011a0 0x7fe67019a900 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe658005ab0 tx=0x7fe658003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.491+0000 7fe667e00640 1 -- 172.21.15.112:0/2094912794 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65803f070 con 0x7fe6701011a0 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.491+0000 7fe667e00640 1 -- 172.21.15.112:0/2094912794 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe658031c50 con 0x7fe6701011a0 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.491+0000 7fe675800640 1 -- 172.21.15.112:0/2094912794 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6701a2270 con 0x7fe6701011a0 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.492+0000 7fe667e00640 1 -- 172.21.15.112:0/2094912794 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658035030 con 0x7fe6701011a0 2024-04-25T18:47:15.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.492+0000 7fe675800640 1 -- 172.21.15.112:0/2094912794 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6701a27b0 con 0x7fe6701011a0 2024-04-25T18:47:15.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.493+0000 7fe675800640 1 -- 172.21.15.112:0/2094912794 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67010b590 con 0x7fe6701011a0 2024-04-25T18:47:15.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.496+0000 7fe667e00640 1 -- 172.21.15.112:0/2094912794 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7fe658042070 con 0x7fe6701011a0 2024-04-25T18:47:15.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.497+0000 7fe667e00640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6540a3d20 0x7fe6540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:15.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.497+0000 7fe66ea00640 1 -- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6540a3d20 msgr2=0x7fe6540a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/605196912 2024-04-25T18:47:15.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.497+0000 7fe667e00640 1 -- 172.21.15.112:0/2094912794 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fe6580f0ad0 con 0x7fe6701011a0 2024-04-25T18:47:15.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.498+0000 7fe66ea00640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6540a3d20 0x7fe6540a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:47:15.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.501+0000 7fe667e00640 1 -- 172.21.15.112:0/2094912794 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6580bd490 con 0x7fe6701011a0 2024-04-25T18:47:15.593 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: ignoring --setuser ceph since I am not root 2024-04-25T18:47:15.593 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: ignoring --setgroup ceph since I am not root 2024-04-25T18:47:15.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:15 smithi112 ceph-mon[27864]: pgmap v454: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:47:15.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:15 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2152193185' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T18:47:15.643 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:15 smithi171 ceph-mon[33266]: pgmap v454: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:47:15.643 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:15 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2152193185' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T18:47:15.644 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: ignoring --setuser ceph since I am not root 2024-04-25T18:47:15.644 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: ignoring --setgroup ceph since I am not root 2024-04-25T18:47:15.644 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:15 smithi171 ceph-mgr[35147]: -- 172.21.15.171:0/1760334129 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56044d7125a0 con 0x56044d6e7000 2024-04-25T18:47:15.644 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:15.456+0000 7fb81647e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:47:15.644 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:15.642+0000 7fb81647e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:47:15.648 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:15 smithi161 ceph-mon[32609]: pgmap v454: 97 pgs: 97 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:47:15.648 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:15 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2152193185' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T18:47:15.648 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: ignoring --setuser ceph since I am not root 2024-04-25T18:47:15.648 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: ignoring --setgroup ceph since I am not root 2024-04-25T18:47:15.648 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:15 smithi161 ceph-mgr[33999]: -- 172.21.15.161:0/3739510082 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5636ed5ec4e0 con 0x5636ed5c2c00 2024-04-25T18:47:15.648 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:15 smithi161 ceph-mgr[33999]: -- 172.21.15.161:0/3739510082 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636ed5ee5a0 con 0x5636ed5c2c00 2024-04-25T18:47:15.649 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:15.461+0000 7f460fd4c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:47:15.649 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:15.646+0000 7f460fd4c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:47:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.699+0000 7fe66ea00640 1 -- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6540a3d20 msgr2=0x7fe6540a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/605196912 2024-04-25T18:47:15.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.699+0000 7fe66ea00640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6540a3d20 0x7fe6540a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:47:15.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.709+0000 7fe675800640 1 -- 172.21.15.112:0/2094912794 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fe67002d660 con 0x7fe6701011a0 2024-04-25T18:47:15.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.717+0000 7fe667e00640 1 -- 172.21.15.112:0/2094912794 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7fe658031e00 con 0x7fe6701011a0 2024-04-25T18:47:15.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.719+0000 7fe675800640 1 -- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6540a3d20 msgr2=0x7fe6540a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:47:15.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.719+0000 7fe675800640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6540a3d20 0x7fe6540a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.719+0000 7fe675800640 1 -- 172.21.15.112:0/2094912794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6701011a0 msgr2=0x7fe67019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:15.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.719+0000 7fe675800640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6701011a0 0x7fe67019a900 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe658005ab0 tx=0x7fe658003b90 comp rx=0 tx=0).stop 2024-04-25T18:47:15.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.719+0000 7fe675800640 1 -- 172.21.15.112:0/2094912794 shutdown_connections 2024-04-25T18:47:15.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.719+0000 7fe675800640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe670101b40 0x7fe6701a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.719+0000 7fe675800640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6701011a0 0x7fe67019a900 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.719+0000 7fe675800640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7fe6540a3d20 0x7fe6540a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.719+0000 7fe675800640 1 --2- 172.21.15.112:0/2094912794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe670100840 0x7fe67019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.719+0000 7fe675800640 1 -- 172.21.15.112:0/2094912794 >> 172.21.15.112:0/2094912794 conn(0x7fe6700fc460 msgr2=0x7fe6700fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:15.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.720+0000 7fe675800640 1 -- 172.21.15.112:0/2094912794 shutdown_connections 2024-04-25T18:47:15.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.720+0000 7fe675800640 1 -- 172.21.15.112:0/2094912794 wait complete. 2024-04-25T18:47:15.740 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T18:47:15.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.915+0000 7f986a000640 1 -- 172.21.15.112:0/2004289160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864100cd0 msgr2=0x7f98641010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:15.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.915+0000 7f986a000640 1 --2- 172.21.15.112:0/2004289160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864100cd0 0x7f98641010f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9858009900 tx=0x7f985802a3f0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.915+0000 7f986a000640 1 -- 172.21.15.112:0/2004289160 shutdown_connections 2024-04-25T18:47:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.915+0000 7f986a000640 1 --2- 172.21.15.112:0/2004289160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9864102880 0x7f9864109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.915+0000 7f986a000640 1 --2- 172.21.15.112:0/2004289160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9864101ec0 0x7f9864102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.915+0000 7f986a000640 1 --2- 172.21.15.112:0/2004289160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864100cd0 0x7f98641010f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.915+0000 7f986a000640 1 -- 172.21.15.112:0/2004289160 >> 172.21.15.112:0/2004289160 conn(0x7f98640fc240 msgr2=0x7f98640fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.915+0000 7f986a000640 1 -- 172.21.15.112:0/2004289160 shutdown_connections 2024-04-25T18:47:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.915+0000 7f986a000640 1 -- 172.21.15.112:0/2004289160 wait complete. 2024-04-25T18:47:15.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.916+0000 7f986a000640 1 Processor -- start 2024-04-25T18:47:15.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.916+0000 7f986a000640 1 -- start start 2024-04-25T18:47:15.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f986a000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9864100cd0 0x7f9864198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:15.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f986a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864101ec0 0x7f98641985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:15.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f986a000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9864102880 0x7f986419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:15.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f986a000640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f986410c840 con 0x7f9864102880 2024-04-25T18:47:15.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f986a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f986410c6c0 con 0x7f9864101ec0 2024-04-25T18:47:15.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f986a000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f986410c9c0 con 0x7f9864100cd0 2024-04-25T18:47:15.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f9863e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864101ec0 0x7f98641985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f9869600640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9864102880 0x7f986419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f9868c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9864100cd0 0x7f9864198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f9863e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864101ec0 0x7f98641985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:46520/0 (socket says 172.21.15.112:46520) 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f9863e00640 1 -- 172.21.15.112:0/385972976 learned_addr learned my addr 172.21.15.112:0/385972976 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f9863e00640 1 -- 172.21.15.112:0/385972976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9864100cd0 msgr2=0x7f9864198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f9863e00640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9864100cd0 0x7f9864198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f9863e00640 1 -- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9864102880 msgr2=0x7f986419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f9863e00640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9864102880 0x7f986419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f9863e00640 1 -- 172.21.15.112:0/385972976 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986419fc90 con 0x7f9864101ec0 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.917+0000 7f9869600640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9864102880 0x7f986419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.918+0000 7f9863e00640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864101ec0 0x7f98641985b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f985000e9f0 tx=0x7f985000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.918+0000 7f9861600640 1 -- 172.21.15.112:0/385972976 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985000cd80 con 0x7f9864101ec0 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.918+0000 7f9861600640 1 -- 172.21.15.112:0/385972976 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98500054d0 con 0x7f9864101ec0 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.918+0000 7f986a000640 1 -- 172.21.15.112:0/385972976 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9864103140 con 0x7f9864101ec0 2024-04-25T18:47:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.918+0000 7f9861600640 1 -- 172.21.15.112:0/385972976 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850010690 con 0x7f9864101ec0 2024-04-25T18:47:15.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.918+0000 7f986a000640 1 -- 172.21.15.112:0/385972976 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9864103640 con 0x7f9864101ec0 2024-04-25T18:47:15.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.919+0000 7f986a000640 1 -- 172.21.15.112:0/385972976 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9864103900 con 0x7f9864101ec0 2024-04-25T18:47:15.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.921+0000 7f9861600640 1 -- 172.21.15.112:0/385972976 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f9850004e90 con 0x7f9864101ec0 2024-04-25T18:47:15.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.923+0000 7f9861600640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f984c0a3bf0 0x7f984c0a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:15.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.923+0000 7f9868c00640 1 -- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f984c0a3bf0 msgr2=0x7f984c0a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/605196912 2024-04-25T18:47:15.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.923+0000 7f9861600640 1 -- 172.21.15.112:0/385972976 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f98500d08d0 con 0x7f9864101ec0 2024-04-25T18:47:15.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.923+0000 7f9868c00640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f984c0a3bf0 0x7f984c0a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:47:15.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:15.927+0000 7f9861600640 1 -- 172.21.15.112:0/385972976 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f985009d230 con 0x7f9864101ec0 2024-04-25T18:47:15.932 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:15.745+0000 7fb81647e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:47:15.971 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:15.750+0000 7f460fd4c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:47:15.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:15.591+0000 7fca67a0a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:47:15.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:15.837+0000 7fca67a0a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:47:16.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.124+0000 7f9868c00640 1 -- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f984c0a3bf0 msgr2=0x7f984c0a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/605196912 2024-04-25T18:47:16.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.124+0000 7f9868c00640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f984c0a3bf0 0x7f984c0a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:47:16.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.130+0000 7f986a000640 1 -- 172.21.15.112:0/385972976 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f9864104170 con 0x7f9864101ec0 2024-04-25T18:47:16.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.137+0000 7f9861600640 1 -- 172.21.15.112:0/385972976 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f985009cb30 con 0x7f9864101ec0 2024-04-25T18:47:16.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 -- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f984c0a3bf0 msgr2=0x7f984c0a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:47:16.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f984c0a3bf0 0x7f984c0a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:16.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 -- 172.21.15.112:0/385972976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864101ec0 msgr2=0x7f98641985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:16.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864101ec0 0x7f98641985b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f985000e9f0 tx=0x7f985000edf0 comp rx=0 tx=0).stop 2024-04-25T18:47:16.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 -- 172.21.15.112:0/385972976 shutdown_connections 2024-04-25T18:47:16.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9864102880 0x7f986419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:16.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864101ec0 0x7f98641985b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:16.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f984c0a3bf0 0x7f984c0a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:16.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 --2- 172.21.15.112:0/385972976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9864100cd0 0x7f9864198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:16.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 -- 172.21.15.112:0/385972976 >> 172.21.15.112:0/385972976 conn(0x7f98640fc240 msgr2=0x7f9864069600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:16.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 -- 172.21.15.112:0/385972976 shutdown_connections 2024-04-25T18:47:16.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:16.139+0000 7f986a000640 1 -- 172.21.15.112:0/385972976 wait complete. 2024-04-25T18:47:16.159 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T18:47:16.180 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T18:47:16.180 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T18:47:16.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:47:16.181 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T18:47:16.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:47:16.231 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T18:47:16.270 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:15.979+0000 7fca67a0a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:47:16.343 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-04-25T18:47:16.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:47:16.347 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T18:47:16.347 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T18:47:16.347 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T18:47:16.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T18:47:16.432 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:16 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:16.053+0000 7fb81647e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:47:16.432 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:16 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:16.137+0000 7fb81647e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:47:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:16 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2152193185' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T18:47:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:16 smithi171 ceph-mon[33266]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-25T18:47:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:16 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' 2024-04-25T18:47:16.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:16 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' 2024-04-25T18:47:16.471 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:16 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:16.060+0000 7f460fd4c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:47:16.471 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:16 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:16.145+0000 7f460fd4c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:47:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:16 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2152193185' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T18:47:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:16 smithi161 ceph-mon[32609]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-25T18:47:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:16 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' 2024-04-25T18:47:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:16 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' 2024-04-25T18:47:16.554 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:16 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:16.433+0000 7fca67a0a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:47:16.554 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:16 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2152193185' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T18:47:16.555 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:16 smithi112 ceph-mon[27864]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-25T18:47:16.555 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:16 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' 2024-04-25T18:47:16.555 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:16 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' 2024-04-25T18:47:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:16.911 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:16 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:16.552+0000 7fca67a0a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:47:17.671 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:17 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:17.331+0000 7f460fd4c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:47:17.671 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:17 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:17.498+0000 7f460fd4c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:47:17.671 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:17 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:17.669+0000 7f460fd4c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:47:17.682 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:17 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:17.380+0000 7fb81647e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:47:17.683 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:17 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:17.555+0000 7fb81647e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:47:18.024 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:17 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:17.734+0000 7fb81647e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:47:18.221 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:17 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:17.958+0000 7f460fd4c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:47:18.221 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:18.048+0000 7f460fd4c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:47:18.320 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:18.023+0000 7fb81647e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:47:18.320 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:18.115+0000 7fb81647e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:47:18.320 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:18.318+0000 7fb81647e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:47:18.381 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:18 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:18.150+0000 7fca67a0a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:47:18.381 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:18 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:18.378+0000 7fca67a0a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:47:18.682 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:18.512+0000 7fb81647e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:47:18.721 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:18.252+0000 7f460fd4c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:47:18.721 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:18.444+0000 7f460fd4c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:47:18.724 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:18 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:18.607+0000 7fca67a0a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:47:19.052 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:19 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:19.024+0000 7fca67a0a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:47:19.408 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:19 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:19.147+0000 7fca67a0a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:47:19.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.414+0000 7f71ef000700 1 -- 172.21.15.112:0/587234170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e8108a80 msgr2=0x7f71e8108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:19.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.414+0000 7f71ef000700 1 --2- 172.21.15.112:0/587234170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e8108a80 0x7f71e8108e60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009990 tx=0x7f71d8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.414+0000 7f71ef000700 1 -- 172.21.15.112:0/587234170 shutdown_connections 2024-04-25T18:47:19.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.414+0000 7f71ef000700 1 --2- 172.21.15.112:0/587234170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71e8103450 0x7f71e810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.414+0000 7f71ef000700 1 --2- 172.21.15.112:0/587234170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e8108a80 0x7f71e8108e60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.414+0000 7f71ef000700 1 --2- 172.21.15.112:0/587234170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8102a90 0x7f71e8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.414+0000 7f71ef000700 1 -- 172.21.15.112:0/587234170 >> 172.21.15.112:0/587234170 conn(0x7f71e8079bf0 msgr2=0x7f71e807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:19.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.414+0000 7f71ef000700 1 -- 172.21.15.112:0/587234170 shutdown_connections 2024-04-25T18:47:19.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.414+0000 7f71ef000700 1 -- 172.21.15.112:0/587234170 wait complete. 2024-04-25T18:47:19.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.415+0000 7f71ef000700 1 Processor -- start 2024-04-25T18:47:19.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.415+0000 7f71ef000700 1 -- start start 2024-04-25T18:47:19.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.415+0000 7f71ef000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e8102a90 0x7f71e8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:19.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71ef000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8103450 0x7f71e8075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:19.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71ef000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71e8108a80 0x7f71e80757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:19.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71ef000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8112c10 con 0x7f71e8102a90 2024-04-25T18:47:19.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71ef000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8112a90 con 0x7f71e8103450 2024-04-25T18:47:19.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71ef000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8112d90 con 0x7f71e8108a80 2024-04-25T18:47:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71e7400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8103450 0x7f71e8075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71e7400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8103450 0x7f71e8075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:46544/0 (socket says 172.21.15.112:46544) 2024-04-25T18:47:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71e7e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e8102a90 0x7f71e8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71e7400700 1 -- 172.21.15.112:0/3063031127 learned_addr learned my addr 172.21.15.112:0/3063031127 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71ed000700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71e8108a80 0x7f71e80757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71e7400700 1 -- 172.21.15.112:0/3063031127 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71e8108a80 msgr2=0x7f71e80757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71e7400700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71e8108a80 0x7f71e80757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71e7400700 1 -- 172.21.15.112:0/3063031127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e8102a90 msgr2=0x7f71e8074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71e7400700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e8102a90 0x7f71e8074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.416+0000 7f71e7400700 1 -- 172.21.15.112:0/3063031127 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e81a7670 con 0x7f71e8103450 2024-04-25T18:47:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.417+0000 7f71e7400700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8103450 0x7f71e8075280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f71d400e9a0 tx=0x7f71d400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.417+0000 7f71e4c00700 1 -- 172.21.15.112:0/3063031127 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d400ca10 con 0x7f71e8103450 2024-04-25T18:47:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.417+0000 7f71e4c00700 1 -- 172.21.15.112:0/3063031127 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d400cbb0 con 0x7f71e8103450 2024-04-25T18:47:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.417+0000 7f71ef000700 1 -- 172.21.15.112:0/3063031127 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e81a7960 con 0x7f71e8103450 2024-04-25T18:47:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.417+0000 7f71e4c00700 1 -- 172.21.15.112:0/3063031127 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4018680 con 0x7f71e8103450 2024-04-25T18:47:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.417+0000 7f71ef000700 1 -- 172.21.15.112:0/3063031127 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e81a7ea0 con 0x7f71e8103450 2024-04-25T18:47:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.418+0000 7f71ef000700 1 -- 172.21.15.112:0/3063031127 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e8069730 con 0x7f71e8103450 2024-04-25T18:47:19.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.420+0000 7f71e4c00700 1 -- 172.21.15.112:0/3063031127 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f71d4010430 con 0x7f71e8103450 2024-04-25T18:47:19.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.421+0000 7f71e4c00700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f71cc0a3c90 0x7f71cc0a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:19.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.421+0000 7f71e7e00700 1 -- 172.21.15.112:0/3063031127 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f71cc0a3c90 msgr2=0x7f71cc0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/605196912 2024-04-25T18:47:19.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.421+0000 7f71e4c00700 1 -- 172.21.15.112:0/3063031127 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f71d40d0660 con 0x7f71e8103450 2024-04-25T18:47:19.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.421+0000 7f71e7e00700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f71cc0a3c90 0x7f71cc0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:47:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.425+0000 7f71e4c00700 1 -- 172.21.15.112:0/3063031127 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71d409cfc0 con 0x7f71e8103450 2024-04-25T18:47:19.432 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:19.110+0000 7fb81647e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:47:19.470 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:19.045+0000 7f460fd4c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:47:19.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.622+0000 7f71e7e00700 1 -- 172.21.15.112:0/3063031127 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f71cc0a3c90 msgr2=0x7f71cc0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/605196912 2024-04-25T18:47:19.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.622+0000 7f71e7e00700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f71cc0a3c90 0x7f71cc0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:47:19.667 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:19 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:19.406+0000 7fca67a0a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:47:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.694+0000 7f71ef000700 1 -- 172.21.15.112:0/3063031127 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f71e802e140 con 0x7f71e8103450 2024-04-25T18:47:19.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.694+0000 7f71e4c00700 1 -- 172.21.15.112:0/3063031127 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f71d409c8c0 con 0x7f71e8103450 2024-04-25T18:47:19.696 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T18:47:19.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 -- 172.21.15.112:0/3063031127 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f71cc0a3c90 msgr2=0x7f71cc0a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:47:19.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f71cc0a3c90 0x7f71cc0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 -- 172.21.15.112:0/3063031127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8103450 msgr2=0x7f71e8075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:19.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8103450 0x7f71e8075280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f71d400e9a0 tx=0x7f71d400ecb0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 -- 172.21.15.112:0/3063031127 shutdown_connections 2024-04-25T18:47:19.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.112:6800/605196912,v1:172.21.15.112:6801/605196912] conn(0x7f71cc0a3c90 0x7f71cc0a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71e8108a80 0x7f71e80757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71e8102a90 0x7f71e8074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 --2- 172.21.15.112:0/3063031127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8103450 0x7f71e8075280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:19.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 -- 172.21.15.112:0/3063031127 >> 172.21.15.112:0/3063031127 conn(0x7f71e8079bf0 msgr2=0x7f71e8101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:19.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 -- 172.21.15.112:0/3063031127 shutdown_connections 2024-04-25T18:47:19.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:19.697+0000 7f71ef000700 1 -- 172.21.15.112:0/3063031127 wait complete. 2024-04-25T18:47:19.721 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:19.468+0000 7f460fd4c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:47:19.721 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:19.564+0000 7f460fd4c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:47:19.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:19 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3063031127' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T18:47:19.831 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:19.533+0000 7fb81647e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:47:19.832 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:19.629+0000 7fb81647e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:47:19.832 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:19.830+0000 7fb81647e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:47:19.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:19 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3063031127' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T18:47:19.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:19 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:19.665+0000 7fca67a0a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:47:20.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:19 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3063031127' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T18:47:20.221 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:19.768+0000 7f460fd4c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:47:20.503 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:20.304+0000 7f460fd4c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:47:20.503 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:20.400+0000 7f460fd4c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:47:20.682 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:20.356+0000 7fb81647e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:47:20.682 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:20.448+0000 7fb81647e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:47:20.695 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T18:47:20.695 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T18:47:20.695 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-04-25T18:47:20.695 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T18:47:20.695 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T18:47:20.695 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T18:47:20.695 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T18:47:20.696 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T18:47:20.696 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T18:47:20.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:47:20.696 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-04-25T18:47:20.723 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:20 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:20.463+0000 7fca67a0a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:47:20.747 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T18:47:20.747 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T18:47:20.747 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T18:47:20.747 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T18:47:20.747 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-04-25T18:47:20.747 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 41891sec preferred_lft 41891sec 2024-04-25T18:47:20.748 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2024-04-25T18:47:20.749 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T18:47:20.749 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T18:47:20.749 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T18:47:20.749 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T18:47:20.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-04-25T18:47:20.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-04-25T18:47:20.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-04-25T18:47:20.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T18:47:20.750 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T18:47:20.799 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:20.657+0000 7f460fd4c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:47:20.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:47:21.034 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:20.690+0000 7fb81647e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:47:21.034 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:20.823+0000 7fb81647e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:47:21.034 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:20.917+0000 7fb81647e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:47:21.034 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:21.032+0000 7fb81647e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:47:21.159 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:21 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:21.033+0000 7fca67a0a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:47:21.221 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:20.797+0000 7f460fd4c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:47:21.221 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:20.896+0000 7f460fd4c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:47:21.221 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:21.015+0000 7f460fd4c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:47:21.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:47:21.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:47:21.233 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T18:47:21.296 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-04-25T18:47:21.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:47:21.299 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-04-25T18:47:21.350 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-04-25T18:47:21.350 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T18:47:21.351 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T18:47:21.351 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T18:47:21.351 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T18:47:21.351 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T18:47:21.351 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T18:47:21.351 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T18:47:21.351 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T18:47:21.351 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T18:47:21.351 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T18:47:21.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:47:21.458 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:21 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:21.168+0000 7fca67a0a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:47:21.682 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:21.341+0000 7fb81647e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:47:21.721 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:21.330+0000 7f460fd4c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:47:21.723 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:21 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:21.455+0000 7fca67a0a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:47:22.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:47:22.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:47:22.108 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T18:47:22.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:47:22.154 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-04-25T18:47:22.221 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:21.938+0000 7f460fd4c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:47:22.221 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y[33995]: 2024-04-25T18:47:22.152+0000 7f460fd4c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:47:22.230 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:21.945+0000 7fb81647e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:47:22.230 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-25T18:47:22.159+0000 7fb81647e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:47:22.246 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T18:47:22.246 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T18:47:22.247 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T18:47:22.247 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T18:47:22.247 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T18:47:22.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:47:22.354 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:22.148+0000 7fca67a0a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:47:22.354 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:22.286+0000 7fca67a0a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:47:22.354 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: Standby manager daemon y restarted 2024-04-25T18:47:22.354 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: Standby manager daemon y started 2024-04-25T18:47:22.354 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:47:22.354 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:47:22.354 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:47:22.355 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:47:22.355 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: Standby manager daemon z restarted 2024-04-25T18:47:22.355 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: Standby manager daemon z started 2024-04-25T18:47:22.355 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:47:22.355 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:47:22.355 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:47:22.355 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-mon[27864]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:47:22.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:47:22.404 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T18:47:22.404 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T18:47:22.404 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T18:47:22.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T18:47:22.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T18:47:22.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T18:47:22.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T18:47:22.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T18:47:22.404 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T18:47:22.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:47:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: Standby manager daemon y restarted 2024-04-25T18:47:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: Standby manager daemon y started 2024-04-25T18:47:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:47:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:47:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:47:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:47:22.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: Standby manager daemon z restarted 2024-04-25T18:47:22.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: Standby manager daemon z started 2024-04-25T18:47:22.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:47:22.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:47:22.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:47:22.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:22 smithi171 ceph-mon[33266]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:47:22.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:47:22.703 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T18:47:22.703 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T18:47:22.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-04-25T18:47:22.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T18:47:22.703 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T18:47:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: Standby manager daemon y restarted 2024-04-25T18:47:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: Standby manager daemon y started 2024-04-25T18:47:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:47:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:47:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:47:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.161:0/2826824093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:47:22.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: Standby manager daemon z restarted 2024-04-25T18:47:22.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: Standby manager daemon z started 2024-04-25T18:47:22.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:47:22.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:47:22.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:47:22.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:22 smithi161 ceph-mon[32609]: from='mgr.? 172.21.15.171:0/290948978' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:47:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:47:22.872 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:22.643+0000 7fca67a0a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:47:22.872 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:22.836+0000 7fca67a0a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:47:22.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:47:22.896 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T18:47:22.896 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T18:47:22.934 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T18:47:22.934 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T18:47:22.934 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T18:47:23.019 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T18:47:23.019 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T18:47:23.020 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/588a97fc-5973-4692-b8e8-3364c5c8fa39 2024-04-25T18:47:23.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:47:23.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/588a97fc-5973-4692-b8e8-3364c5c8fa39 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T18:47:23.132 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:22 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:22.967+0000 7fca67a0a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:47:23.132 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:23 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:23.130+0000 7fca67a0a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:47:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T18:47:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T18:47:23.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:23 smithi112 ceph-mon[27864]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-25T18:47:23.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:23 smithi171 ceph-mon[33266]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-25T18:47:23.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:23 smithi161 ceph-mon[32609]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-25T18:47:23.973 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:23 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:23.513+0000 7fca67a0a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:47:24.051 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T18:47:24.051 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T18:47:24.051 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.161:3300,172.21.15.171:3300" 2024-04-25T18:47:24.052 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with new device syntax: 0@2fe18a52-0332-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/588a97fc-5973-4692-b8e8-3364c5c8fa39 2024-04-25T18:47:24.052 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.112:3300/172.21.15.161:3300/172.21.15.171:3300" will pass to kernel 2024-04-25T18:47:24.054 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T18:47:24.054 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-04-25T18:47:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T18:47:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T18:47:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T18:47:24.059 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T18:47:24.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:47:24.059 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T18:47:24.125 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T18:47:24.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T18:47:24.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:47:24.238 INFO:teuthology.orchestra.run.smithi112.stdout:client25058 2024-04-25T18:47:24.239 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:47:24.239 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T18:47:24.288 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T18:47:24.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T18:47:24.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:47:24.392 INFO:teuthology.orchestra.run.smithi112.stdout:client25058 2024-04-25T18:47:24.392 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T18:47:24.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T18:47:24.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:47:24.506 INFO:teuthology.orchestra.run.smithi112.stdout:client25058 2024-04-25T18:47:24.506 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2fe18a52-0332-11ef-bc93-c7b262605968.client25058/status') 2024-04-25T18:47:24.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:47:24.617 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.313+0000 7fca67a0a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:47:24.627 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.25058 (3)172.21.15.112:0/1837588920 2024-04-25T18:47:24.627 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-04-25T18:47:24.627 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-04-25T18:47:24.627 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-04-25T18:47:24.627 INFO:teuthology.orchestra.run.smithi112.stderr:72 bytes copied, 6.3388e-05 s, 1.1 MB/s 2024-04-25T18:47:24.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:47:24.642 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T18:47:24.652 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T18:47:24.652 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T18:47:24.652 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T18:47:24.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.842+0000 7f15d4c00640 1 -- 172.21.15.112:0/3456340132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15d0103190 msgr2=0x7f15d0105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:24.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.842+0000 7f15d4c00640 1 --2- 172.21.15.112:0/3456340132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15d0103190 0x7f15d0105600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009860 tx=0x7f15bc02a340 comp rx=0 tx=0).stop 2024-04-25T18:47:24.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.843+0000 7f15d4c00640 1 -- 172.21.15.112:0/3456340132 shutdown_connections 2024-04-25T18:47:24.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.843+0000 7f15d4c00640 1 --2- 172.21.15.112:0/3456340132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d0105b40 0x7f15d0105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:24.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.843+0000 7f15d4c00640 1 --2- 172.21.15.112:0/3456340132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15d0103190 0x7f15d0105600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:24.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.843+0000 7f15d4c00640 1 --2- 172.21.15.112:0/3456340132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d0100820 0x7f15d0102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:24.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.843+0000 7f15d4c00640 1 -- 172.21.15.112:0/3456340132 >> 172.21.15.112:0/3456340132 conn(0x7f15d00fc420 msgr2=0x7f15d00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:24.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.843+0000 7f15d4c00640 1 -- 172.21.15.112:0/3456340132 shutdown_connections 2024-04-25T18:47:24.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.843+0000 7f15d4c00640 1 -- 172.21.15.112:0/3456340132 wait complete. 2024-04-25T18:47:24.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.844+0000 7f15d4c00640 1 Processor -- start 2024-04-25T18:47:24.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.844+0000 7f15d4c00640 1 -- start start 2024-04-25T18:47:24.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.844+0000 7f15d4c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d0100820 0x7f15d019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:24.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.844+0000 7f15d4c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d0103190 0x7f15d019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:24.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.844+0000 7f15d4c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15d0105b40 0x7f15d01a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:24.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.844+0000 7f15d4c00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0114130 con 0x7f15d0100820 2024-04-25T18:47:24.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.844+0000 7f15d4c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0113fb0 con 0x7f15d0105b40 2024-04-25T18:47:24.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.844+0000 7f15d4c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f15d01142b0 con 0x7f15d0103190 2024-04-25T18:47:24.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.845+0000 7f15ce800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15d0105b40 0x7f15d01a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.845+0000 7f15ce800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15d0105b40 0x7f15d01a5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:57506/0 (socket says 172.21.15.112:57506) 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.845+0000 7f15ce800640 1 -- 172.21.15.112:0/1072518730 learned_addr learned my addr 172.21.15.112:0/1072518730 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.845+0000 7f15cd400640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d0103190 0x7f15d019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.845+0000 7f15cde00640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d0100820 0x7f15d019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.845+0000 7f15ce800640 1 -- 172.21.15.112:0/1072518730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d0103190 msgr2=0x7f15d019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.845+0000 7f15ce800640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d0103190 0x7f15d019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.845+0000 7f15ce800640 1 -- 172.21.15.112:0/1072518730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d0100820 msgr2=0x7f15d019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.845+0000 7f15ce800640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d0100820 0x7f15d019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.845+0000 7f15ce800640 1 -- 172.21.15.112:0/1072518730 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d01a63c0 con 0x7f15d0105b40 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.846+0000 7f15ce800640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15d0105b40 0x7f15d01a5d50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f15c800b970 tx=0x7f15c800be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.846+0000 7f15c6a00640 1 -- 172.21.15.112:0/1072518730 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8019070 con 0x7f15d0105b40 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.846+0000 7f15c6a00640 1 -- 172.21.15.112:0/1072518730 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15c8014440 con 0x7f15d0105b40 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.846+0000 7f15c6a00640 1 -- 172.21.15.112:0/1072518730 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c80075d0 con 0x7f15d0105b40 2024-04-25T18:47:24.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.846+0000 7f15d4c00640 1 -- 172.21.15.112:0/1072518730 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d01a66b0 con 0x7f15d0105b40 2024-04-25T18:47:24.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.846+0000 7f15d4c00640 1 -- 172.21.15.112:0/1072518730 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d01a6b80 con 0x7f15d0105b40 2024-04-25T18:47:24.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.848+0000 7f15d4c00640 1 -- 172.21.15.112:0/1072518730 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d00fdf10 con 0x7f15d0105b40 2024-04-25T18:47:24.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.850+0000 7f15c6a00640 1 -- 172.21.15.112:0/1072518730 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f15c801c070 con 0x7f15d0105b40 2024-04-25T18:47:24.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.852+0000 7f15c6a00640 1 -- 172.21.15.112:0/1072518730 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f15c80cf4b0 con 0x7f15d0105b40 2024-04-25T18:47:24.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:24.856+0000 7f15c6a00640 1 -- 172.21.15.112:0/1072518730 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15c809bb00 con 0x7f15d0105b40 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: Active manager daemon x restarted 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: Activating manager daemon x 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: mgrmap e21: x(active, starting, since 0.0134899s), standbys: y, z 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:47:24.929 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:47:24.930 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:47:24.931 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:47:24.931 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:47:24.931 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:47:24.931 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:47:24.931 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:47:24.931 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:47:24.931 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:47:24.931 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:47:24.931 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:24 smithi161 ceph-mon[32609]: Manager daemon x is now available 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: Active manager daemon x restarted 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: Activating manager daemon x 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: mgrmap e21: x(active, starting, since 0.0134899s), standbys: y, z 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:47:24.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:47:24.934 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:47:24.935 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:47:24.935 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:47:24.935 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:47:24.935 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:47:24.935 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:47:24.935 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:47:24.935 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:24 smithi171 ceph-mon[33266]: Manager daemon x is now available 2024-04-25T18:47:24.939 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.615+0000 7fca67a0a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:47:24.940 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: Active manager daemon x restarted 2024-04-25T18:47:24.940 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: Activating manager daemon x 2024-04-25T18:47:24.940 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T18:47:24.940 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: mgrmap e21: x(active, starting, since 0.0134899s), standbys: y, z 2024-04-25T18:47:24.940 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:47:24.940 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:47:24.940 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:47:24.940 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:47:24.940 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:47:24.940 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:47:24.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:47:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-mon[27864]: Manager daemon x is now available 2024-04-25T18:47:25.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.171+0000 7f15d4c00640 1 -- 172.21.15.112:0/1072518730 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f15d0062b50 con 0x7f15d0105b40 2024-04-25T18:47:25.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.966+0000 7fc9bde00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:25.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.966+0000 7fc9bde00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:25.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.966+0000 7fc9bde00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:25.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.966+0000 7fc9bde00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:25.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.966+0000 7fc9bde00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:25.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.969+0000 7fc9ba200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:25.223 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.969+0000 7fc9ba200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:25.224 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.969+0000 7fc9ba200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:25.224 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.969+0000 7fc9ba200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:25.224 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:24.969+0000 7fc9ba200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:25.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.651+0000 7f15c6a00640 1 -- 172.21.15.112:0/1072518730 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f15c8097850 con 0x7f15d0105b40 2024-04-25T18:47:25.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.652+0000 7f15c6a00640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f15b40aaad0 0x7f15b40acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:25.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.658+0000 7f15cde00640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f15b40aaad0 0x7f15b40acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:25.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.672+0000 7f15cde00640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f15b40aaad0 0x7f15b40acec0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f15c0005fd0 tx=0x7f15c0009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:25.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.685+0000 7f15c6a00640 1 -- 172.21.15.112:0/1072518730 <== mon.2 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f15c8091a50 con 0x7f15d0105b40 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 -- 172.21.15.112:0/1072518730 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f15b40aaad0 msgr2=0x7f15b40acec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f15b40aaad0 0x7f15b40acec0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f15c0005fd0 tx=0x7f15c0009260 comp rx=0 tx=0).stop 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 -- 172.21.15.112:0/1072518730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15d0105b40 msgr2=0x7f15d01a5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15d0105b40 0x7f15d01a5d50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f15c800b970 tx=0x7f15c800be30 comp rx=0 tx=0).stop 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 -- 172.21.15.112:0/1072518730 shutdown_connections 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f15b40aaad0 0x7f15b40acec0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15d0105b40 0x7f15d01a5d50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15d0103190 0x7f15d019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 --2- 172.21.15.112:0/1072518730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15d0100820 0x7f15d019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 -- 172.21.15.112:0/1072518730 >> 172.21.15.112:0/1072518730 conn(0x7f15d00fc420 msgr2=0x7f15d0101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 -- 172.21.15.112:0/1072518730 shutdown_connections 2024-04-25T18:47:25.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.689+0000 7f15d4c00640 1 -- 172.21.15.112:0/1072518730 wait complete. 2024-04-25T18:47:25.707 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T18:47:25.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.879+0000 7f6e0da00640 1 -- 172.21.15.112:0/507889459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e081031c0 msgr2=0x7f6e08103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.879+0000 7f6e0da00640 1 --2- 172.21.15.112:0/507889459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e081031c0 0x7f6e08103640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009900 tx=0x7f6df402a410 comp rx=0 tx=0).stop 2024-04-25T18:47:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.879+0000 7f6e0da00640 1 -- 172.21.15.112:0/507889459 shutdown_connections 2024-04-25T18:47:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.879+0000 7f6e0da00640 1 --2- 172.21.15.112:0/507889459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e08103b80 0x7f6e0810a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.879+0000 7f6e0da00640 1 --2- 172.21.15.112:0/507889459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e081031c0 0x7f6e08103640 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.879+0000 7f6e0da00640 1 --2- 172.21.15.112:0/507889459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e08101fd0 0x7f6e081023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.879+0000 7f6e0da00640 1 -- 172.21.15.112:0/507889459 >> 172.21.15.112:0/507889459 conn(0x7f6e080fd540 msgr2=0x7f6e080ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.879+0000 7f6e0da00640 1 -- 172.21.15.112:0/507889459 shutdown_connections 2024-04-25T18:47:25.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.879+0000 7f6e0da00640 1 -- 172.21.15.112:0/507889459 wait complete. 2024-04-25T18:47:25.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e0da00640 1 Processor -- start 2024-04-25T18:47:25.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e0da00640 1 -- start start 2024-04-25T18:47:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e0da00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e08101fd0 0x7f6e0819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e0da00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e081031c0 0x7f6e0819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e0da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e08103b80 0x7f6e081a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e0da00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0810db10 con 0x7f6e081031c0 2024-04-25T18:47:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e0da00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0810d990 con 0x7f6e08103b80 2024-04-25T18:47:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e0da00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0810dc90 con 0x7f6e08101fd0 2024-04-25T18:47:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e06a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e08101fd0 0x7f6e0819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e07400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e08103b80 0x7f6e081a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e06000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e081031c0 0x7f6e0819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e07400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e08103b80 0x7f6e081a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:57588/0 (socket says 172.21.15.112:57588) 2024-04-25T18:47:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.881+0000 7f6e07400640 1 -- 172.21.15.112:0/3283780627 learned_addr learned my addr 172.21.15.112:0/3283780627 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.882+0000 7f6e07400640 1 -- 172.21.15.112:0/3283780627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e08101fd0 msgr2=0x7f6e0819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.882+0000 7f6e07400640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e08101fd0 0x7f6e0819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.882+0000 7f6e07400640 1 -- 172.21.15.112:0/3283780627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e081031c0 msgr2=0x7f6e0819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.882+0000 7f6e07400640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e081031c0 0x7f6e0819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.882+0000 7f6e07400640 1 -- 172.21.15.112:0/3283780627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e081a1f70 con 0x7f6e08103b80 2024-04-25T18:47:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.882+0000 7f6e07400640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e08103b80 0x7f6e081a1900 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000eb10 tx=0x7f6e0000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.882+0000 7f6dff400640 1 -- 172.21.15.112:0/3283780627 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00004b20 con 0x7f6e08103b80 2024-04-25T18:47:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.882+0000 7f6dff400640 1 -- 172.21.15.112:0/3283780627 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e00004cc0 con 0x7f6e08103b80 2024-04-25T18:47:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.882+0000 7f6e0da00640 1 -- 172.21.15.112:0/3283780627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e081a2260 con 0x7f6e08103b80 2024-04-25T18:47:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.882+0000 7f6dff400640 1 -- 172.21.15.112:0/3283780627 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00010670 con 0x7f6e08103b80 2024-04-25T18:47:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.882+0000 7f6e0da00640 1 -- 172.21.15.112:0/3283780627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e081a2780 con 0x7f6e08103b80 2024-04-25T18:47:25.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.884+0000 7f6e0da00640 1 -- 172.21.15.112:0/3283780627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e081a2a40 con 0x7f6e08103b80 2024-04-25T18:47:25.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.885+0000 7f6dff400640 1 -- 172.21.15.112:0/3283780627 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f6e00005020 con 0x7f6e08103b80 2024-04-25T18:47:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.887+0000 7f6dff400640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6dec0a3ae0 0x7f6dec0a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.887+0000 7f6dff400640 1 -- 172.21.15.112:0/3283780627 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f6e000d0690 con 0x7f6e08103b80 2024-04-25T18:47:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.887+0000 7f6e06a00640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6dec0a3ae0 0x7f6dec0a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:25.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.888+0000 7f6e06a00640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6dec0a3ae0 0x7f6dec0a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e08103020 tx=0x7f6df8005f90 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:25.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:25.891+0000 7f6dff400640 1 -- 172.21.15.112:0/3283780627 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e0009d200 con 0x7f6e08103b80 2024-04-25T18:47:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:47:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:47:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:47:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:47:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: [25/Apr/2024:18:47:24] ENGINE Bus STARTING 2024-04-25T18:47:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: [25/Apr/2024:18:47:25] ENGINE Serving on http://172.21.15.112:8765 2024-04-25T18:47:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1072518730' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T18:47:25.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T18:47:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: [25/Apr/2024:18:47:25] ENGINE Serving on https://172.21.15.112:7150 2024-04-25T18:47:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: [25/Apr/2024:18:47:25] ENGINE Bus STARTED 2024-04-25T18:47:25.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:25 smithi171 ceph-mon[33266]: mgrmap e22: x(active, since 1.03026s), standbys: y, z 2024-04-25T18:47:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:47:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:47:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:47:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:47:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: [25/Apr/2024:18:47:24] ENGINE Bus STARTING 2024-04-25T18:47:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: [25/Apr/2024:18:47:25] ENGINE Serving on http://172.21.15.112:8765 2024-04-25T18:47:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1072518730' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T18:47:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T18:47:25.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: [25/Apr/2024:18:47:25] ENGINE Serving on https://172.21.15.112:7150 2024-04-25T18:47:25.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: [25/Apr/2024:18:47:25] ENGINE Bus STARTED 2024-04-25T18:47:25.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:25 smithi161 ceph-mon[32609]: mgrmap e22: x(active, since 1.03026s), standbys: y, z 2024-04-25T18:47:25.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:25.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:47:25.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:47:25.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:47:25.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:47:25.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: [25/Apr/2024:18:47:24] ENGINE Bus STARTING 2024-04-25T18:47:25.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: [25/Apr/2024:18:47:25] ENGINE Serving on http://172.21.15.112:8765 2024-04-25T18:47:25.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1072518730' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T18:47:25.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T18:47:25.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: [25/Apr/2024:18:47:25] ENGINE Serving on https://172.21.15.112:7150 2024-04-25T18:47:25.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: [25/Apr/2024:18:47:25] ENGINE Bus STARTED 2024-04-25T18:47:25.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:25 smithi112 ceph-mon[27864]: mgrmap e22: x(active, since 1.03026s), standbys: y, z 2024-04-25T18:47:26.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.124+0000 7f6e0da00640 1 -- 172.21.15.112:0/3283780627 --> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f6e080758a0 con 0x7f6dec0a3ae0 2024-04-25T18:47:26.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.143+0000 7f6dff400640 1 -- 172.21.15.112:0/3283780627 <== mgr.14922 v2:172.21.15.112:6800/2236031982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f6e080758a0 con 0x7f6dec0a3ae0 2024-04-25T18:47:26.145 INFO:teuthology.orchestra.run.smithi112.stdout:{} 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.145+0000 7f6e0da00640 1 -- 172.21.15.112:0/3283780627 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6dec0a3ae0 msgr2=0x7f6dec0a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.145+0000 7f6e0da00640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6dec0a3ae0 0x7f6dec0a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e08103020 tx=0x7f6df8005f90 comp rx=0 tx=0).stop 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.145+0000 7f6e0da00640 1 -- 172.21.15.112:0/3283780627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e08103b80 msgr2=0x7f6e081a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.145+0000 7f6e0da00640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e08103b80 0x7f6e081a1900 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000eb10 tx=0x7f6e0000c2d0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.146+0000 7f6e0da00640 1 -- 172.21.15.112:0/3283780627 shutdown_connections 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.146+0000 7f6e0da00640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6dec0a3ae0 0x7f6dec0a5fa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.146+0000 7f6e0da00640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e08103b80 0x7f6e081a1900 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.146+0000 7f6e0da00640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e081031c0 0x7f6e0819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.146+0000 7f6e0da00640 1 --2- 172.21.15.112:0/3283780627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e08101fd0 0x7f6e0819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.146+0000 7f6e0da00640 1 -- 172.21.15.112:0/3283780627 >> 172.21.15.112:0/3283780627 conn(0x7f6e080fd540 msgr2=0x7f6e08074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.146+0000 7f6e0da00640 1 -- 172.21.15.112:0/3283780627 shutdown_connections 2024-04-25T18:47:26.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.146+0000 7f6e0da00640 1 -- 172.21.15.112:0/3283780627 wait complete. 2024-04-25T18:47:26.165 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T18:47:26.179 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T18:47:26.187 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T18:47:26.187 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T18:47:26.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T18:47:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.357+0000 7f5d5f000640 1 -- 172.21.15.112:0/1596746400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d58101df0 msgr2=0x7f5d58102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.357+0000 7f5d5f000640 1 --2- 172.21.15.112:0/1596746400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d58101df0 0x7f5d58102210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009860 tx=0x7f5d4c02a340 comp rx=0 tx=0).stop 2024-04-25T18:47:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.357+0000 7f5d5f000640 1 -- 172.21.15.112:0/1596746400 shutdown_connections 2024-04-25T18:47:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.357+0000 7f5d5f000640 1 --2- 172.21.15.112:0/1596746400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d581039a0 0x7f5d5810a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.357+0000 7f5d5f000640 1 --2- 172.21.15.112:0/1596746400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d58102fe0 0x7f5d58103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.357+0000 7f5d5f000640 1 --2- 172.21.15.112:0/1596746400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d58101df0 0x7f5d58102210 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.357+0000 7f5d5f000640 1 -- 172.21.15.112:0/1596746400 >> 172.21.15.112:0/1596746400 conn(0x7f5d580fd360 msgr2=0x7f5d580ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.357+0000 7f5d5f000640 1 -- 172.21.15.112:0/1596746400 shutdown_connections 2024-04-25T18:47:26.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.357+0000 7f5d5f000640 1 -- 172.21.15.112:0/1596746400 wait complete. 2024-04-25T18:47:26.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5f000640 1 Processor -- start 2024-04-25T18:47:26.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5f000640 1 -- start start 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5f000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d58101df0 0x7f5d5819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5f000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d58102fe0 0x7f5d5819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5f000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d581039a0 0x7f5d581a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5f000640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5810d8f0 con 0x7f5d58101df0 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5f000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5810d770 con 0x7f5d581039a0 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5f000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5810da70 con 0x7f5d58102fe0 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5dc00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d58101df0 0x7f5d5819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5dc00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d58101df0 0x7f5d5819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43318/0 (socket says 172.21.15.112:43318) 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5dc00640 1 -- 172.21.15.112:0/1126459889 learned_addr learned my addr 172.21.15.112:0/1126459889 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5d200640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d58102fe0 0x7f5d5819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.358+0000 7f5d5e600640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d581039a0 0x7f5d581a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.359+0000 7f5d5dc00640 1 -- 172.21.15.112:0/1126459889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d58102fe0 msgr2=0x7f5d5819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:26.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.359+0000 7f5d5dc00640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d58102fe0 0x7f5d5819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.359+0000 7f5d5dc00640 1 -- 172.21.15.112:0/1126459889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d581039a0 msgr2=0x7f5d581a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.359+0000 7f5d5dc00640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d581039a0 0x7f5d581a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.359+0000 7f5d5dc00640 1 -- 172.21.15.112:0/1126459889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d581a1e70 con 0x7f5d58101df0 2024-04-25T18:47:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.359+0000 7f5d5dc00640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d58101df0 0x7f5d5819a250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0057c0 tx=0x7f5d4c032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.359+0000 7f5d56000640 1 -- 172.21.15.112:0/1126459889 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c037040 con 0x7f5d58101df0 2024-04-25T18:47:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.359+0000 7f5d56000640 1 -- 172.21.15.112:0/1126459889 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d4c030bd0 con 0x7f5d58101df0 2024-04-25T18:47:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.359+0000 7f5d5f000640 1 -- 172.21.15.112:0/1126459889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d581a2100 con 0x7f5d58101df0 2024-04-25T18:47:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.359+0000 7f5d56000640 1 -- 172.21.15.112:0/1126459889 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c005280 con 0x7f5d58101df0 2024-04-25T18:47:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.359+0000 7f5d5f000640 1 -- 172.21.15.112:0/1126459889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d581a2590 con 0x7f5d58101df0 2024-04-25T18:47:26.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.361+0000 7f5d5f000640 1 -- 172.21.15.112:0/1126459889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d581a27f0 con 0x7f5d58101df0 2024-04-25T18:47:26.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.363+0000 7f5d56000640 1 -- 172.21.15.112:0/1126459889 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f5d4c043020 con 0x7f5d58101df0 2024-04-25T18:47:26.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.364+0000 7f5d56000640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f5d400a3990 0x7f5d400a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:26.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.364+0000 7f5d56000640 1 -- 172.21.15.112:0/1126459889 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f5d4c0ef700 con 0x7f5d58101df0 2024-04-25T18:47:26.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.364+0000 7f5d5d200640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f5d400a3990 0x7f5d400a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:26.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.365+0000 7f5d5d200640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f5d400a3990 0x7f5d400a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5d5819b760 tx=0x7f5d44009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:26.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.369+0000 7f5d56000640 1 -- 172.21.15.112:0/1126459889 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d4c0bc270 con 0x7f5d58101df0 2024-04-25T18:47:26.417 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:26.131+0000 7fc9b6600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:26.418 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:26.131+0000 7fc9b6600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:26.418 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:26.131+0000 7fc9b6600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:26.418 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:26.131+0000 7fc9b6600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:26.418 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:47:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:47:26.131+0000 7fc9b6600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:47:26.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.599+0000 7f5d5f000640 1 -- 172.21.15.112:0/1126459889 --> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f5d58109ed0 con 0x7f5d400a3990 2024-04-25T18:47:26.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.694+0000 7f5d56000640 1 -- 172.21.15.112:0/1126459889 <== mgr.14922 v2:172.21.15.112:6800/2236031982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5d58109ed0 con 0x7f5d400a3990 2024-04-25T18:47:26.697 INFO:teuthology.orchestra.run.smithi112.stdout:Schedule set for path / 2024-04-25T18:47:26.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.697+0000 7f5d5f000640 1 -- 172.21.15.112:0/1126459889 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f5d400a3990 msgr2=0x7f5d400a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:26.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.697+0000 7f5d5f000640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f5d400a3990 0x7f5d400a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5d5819b760 tx=0x7f5d44009260 comp rx=0 tx=0).stop 2024-04-25T18:47:26.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.698+0000 7f5d5f000640 1 -- 172.21.15.112:0/1126459889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d58101df0 msgr2=0x7f5d5819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:26.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.698+0000 7f5d5f000640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d58101df0 0x7f5d5819a250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0057c0 tx=0x7f5d4c032da0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.698+0000 7f5d5f000640 1 -- 172.21.15.112:0/1126459889 shutdown_connections 2024-04-25T18:47:26.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.698+0000 7f5d5f000640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f5d400a3990 0x7f5d400a5e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.698+0000 7f5d5f000640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d581039a0 0x7f5d581a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.698+0000 7f5d5f000640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d58102fe0 0x7f5d5819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.698+0000 7f5d5f000640 1 --2- 172.21.15.112:0/1126459889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d58101df0 0x7f5d5819a250 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.698+0000 7f5d5f000640 1 -- 172.21.15.112:0/1126459889 >> 172.21.15.112:0/1126459889 conn(0x7f5d580fd360 msgr2=0x7f5d581087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:26.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.698+0000 7f5d5f000640 1 -- 172.21.15.112:0/1126459889 shutdown_connections 2024-04-25T18:47:26.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.698+0000 7f5d5f000640 1 -- 172.21.15.112:0/1126459889 wait complete. 2024-04-25T18:47:26.720 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T18:47:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.897+0000 7f8477e00640 1 -- 172.21.15.112:0/1842045921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f847810e3b0 msgr2=0x7f84781107a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.897+0000 7f8477e00640 1 --2- 172.21.15.112:0/1842045921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f847810e3b0 0x7f84781107a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8470009cd0 tx=0x7f847002a8f0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.897+0000 7f8477e00640 1 -- 172.21.15.112:0/1842045921 shutdown_connections 2024-04-25T18:47:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.897+0000 7f8477e00640 1 --2- 172.21.15.112:0/1842045921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f847810e3b0 0x7f84781107a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.897+0000 7f8477e00640 1 --2- 172.21.15.112:0/1842045921 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84780749b0 0x7f8478074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.897+0000 7f8477e00640 1 --2- 172.21.15.112:0/1842045921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8478075f90 0x7f84780743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.897+0000 7f8477e00640 1 -- 172.21.15.112:0/1842045921 >> 172.21.15.112:0/1842045921 conn(0x7f84780fd350 msgr2=0x7f84780ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.897+0000 7f8477e00640 1 -- 172.21.15.112:0/1842045921 shutdown_connections 2024-04-25T18:47:26.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.897+0000 7f8477e00640 1 -- 172.21.15.112:0/1842045921 wait complete. 2024-04-25T18:47:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.898+0000 7f8477e00640 1 Processor -- start 2024-04-25T18:47:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.898+0000 7f8477e00640 1 -- start start 2024-04-25T18:47:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.898+0000 7f8477e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84780749b0 0x7f847819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.898+0000 7f8477e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8478075f90 0x7f847819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.898+0000 7f8477e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f847810e3b0 0x7f84781a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.898+0000 7f8477e00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8478113f70 con 0x7f847810e3b0 2024-04-25T18:47:26.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.898+0000 7f8477e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8478113df0 con 0x7f84780749b0 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.898+0000 7f8477e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f84781140f0 con 0x7f8478075f90 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8476a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84780749b0 0x7f847819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8476a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84780749b0 0x7f847819e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:57684/0 (socket says 172.21.15.112:57684) 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8476a00640 1 -- 172.21.15.112:0/1840876915 learned_addr learned my addr 172.21.15.112:0/1840876915 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8477400640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f847810e3b0 0x7f84781a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8476000640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8478075f90 0x7f847819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8476a00640 1 -- 172.21.15.112:0/1840876915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8478075f90 msgr2=0x7f847819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8476a00640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8478075f90 0x7f847819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8476a00640 1 -- 172.21.15.112:0/1840876915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f847810e3b0 msgr2=0x7f84781a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8476a00640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f847810e3b0 0x7f84781a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8476a00640 1 -- 172.21.15.112:0/1840876915 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84781a61e0 con 0x7f84780749b0 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8476a00640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84780749b0 0x7f847819e5c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f846800d950 tx=0x7f846800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:26.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f847ca00640 1 -- 172.21.15.112:0/1840876915 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468017030 con 0x7f84780749b0 2024-04-25T18:47:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f8477e00640 1 -- 172.21.15.112:0/1840876915 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84781a64d0 con 0x7f84780749b0 2024-04-25T18:47:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.899+0000 7f847ca00640 1 -- 172.21.15.112:0/1840876915 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8468004020 con 0x7f84780749b0 2024-04-25T18:47:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.900+0000 7f847ca00640 1 -- 172.21.15.112:0/1840876915 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680047a0 con 0x7f84780749b0 2024-04-25T18:47:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.900+0000 7f8477e00640 1 -- 172.21.15.112:0/1840876915 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84781a6a70 con 0x7f84780749b0 2024-04-25T18:47:26.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.901+0000 7f8477e00640 1 -- 172.21.15.112:0/1840876915 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8478104470 con 0x7f84780749b0 2024-04-25T18:47:26.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.903+0000 7f847ca00640 1 -- 172.21.15.112:0/1840876915 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f8468009a30 con 0x7f84780749b0 2024-04-25T18:47:26.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.904+0000 7f847ca00640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f845c0a3da0 0x7f845c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:26.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.904+0000 7f847ca00640 1 -- 172.21.15.112:0/1840876915 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f84680d0490 con 0x7f84780749b0 2024-04-25T18:47:26.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.904+0000 7f8476000640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f845c0a3da0 0x7f845c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:26.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.905+0000 7f8476000640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f845c0a3da0 0x7f845c0a6260 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8464005fd0 tx=0x7f8464005850 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:26.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:26.908+0000 7f847ca00640 1 -- 172.21.15.112:0/1840876915 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f846809c9c0 con 0x7f84780749b0 2024-04-25T18:47:26.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:26 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T18:47:26.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:26 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:47:26.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:26 smithi161 ceph-mon[32609]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:26.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:26 smithi161 ceph-mon[32609]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:26.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:26 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T18:47:26.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:26 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:47:26.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:26 smithi112 ceph-mon[27864]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:26.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:26 smithi112 ceph-mon[27864]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.145+0000 7f8477e00640 1 -- 172.21.15.112:0/1840876915 --> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f8478062430 con 0x7f845c0a3da0 2024-04-25T18:47:27.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:26 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T18:47:27.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:26 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:47:27.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:26 smithi171 ceph-mon[33266]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:27.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:26 smithi171 ceph-mon[33266]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:27.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.216+0000 7f847ca00640 1 -- 172.21.15.112:0/1840876915 <== mgr.14922 v2:172.21.15.112:6800/2236031982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8478062430 con 0x7f845c0a3da0 2024-04-25T18:47:27.218 INFO:teuthology.orchestra.run.smithi112.stdout:Retention added to path / 2024-04-25T18:47:27.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 -- 172.21.15.112:0/1840876915 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f845c0a3da0 msgr2=0x7f845c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f845c0a3da0 0x7f845c0a6260 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8464005fd0 tx=0x7f8464005850 comp rx=0 tx=0).stop 2024-04-25T18:47:27.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 -- 172.21.15.112:0/1840876915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84780749b0 msgr2=0x7f847819e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84780749b0 0x7f847819e5c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f846800d950 tx=0x7f846800de10 comp rx=0 tx=0).stop 2024-04-25T18:47:27.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 -- 172.21.15.112:0/1840876915 shutdown_connections 2024-04-25T18:47:27.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f845c0a3da0 0x7f845c0a6260 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f847810e3b0 0x7f84781a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8478075f90 0x7f847819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 --2- 172.21.15.112:0/1840876915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84780749b0 0x7f847819e5c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 -- 172.21.15.112:0/1840876915 >> 172.21.15.112:0/1840876915 conn(0x7f84780fd350 msgr2=0x7f8478101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:27.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 -- 172.21.15.112:0/1840876915 shutdown_connections 2024-04-25T18:47:27.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.218+0000 7f8477e00640 1 -- 172.21.15.112:0/1840876915 wait complete. 2024-04-25T18:47:27.235 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.424+0000 7fb9ebc00640 1 -- 172.21.15.112:0/3122375941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9e4107c90 msgr2=0x7fb9e410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.424+0000 7fb9ebc00640 1 --2- 172.21.15.112:0/3122375941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9e4107c90 0x7fb9e410a0e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009ca0 tx=0x7fb9dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.424+0000 7fb9ebc00640 1 -- 172.21.15.112:0/3122375941 shutdown_connections 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.424+0000 7fb9ebc00640 1 --2- 172.21.15.112:0/3122375941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9e4107c90 0x7fb9e410a0e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.424+0000 7fb9ebc00640 1 --2- 172.21.15.112:0/3122375941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9e4105360 0x7fb9e4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.424+0000 7fb9ebc00640 1 --2- 172.21.15.112:0/3122375941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e4102a30 0x7fb9e4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.424+0000 7fb9ebc00640 1 -- 172.21.15.112:0/3122375941 >> 172.21.15.112:0/3122375941 conn(0x7fb9e40fc440 msgr2=0x7fb9e40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.425+0000 7fb9ebc00640 1 -- 172.21.15.112:0/3122375941 shutdown_connections 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.425+0000 7fb9ebc00640 1 -- 172.21.15.112:0/3122375941 wait complete. 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.426+0000 7fb9ebc00640 1 Processor -- start 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.426+0000 7fb9ebc00640 1 -- start start 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.426+0000 7fb9ebc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9e4102a30 0x7fb9e41980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.426+0000 7fb9ebc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9e4105360 0x7fb9e41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.426+0000 7fb9ebc00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e4107c90 0x7fb9e419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9ebc00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e410db40 con 0x7fb9e4107c90 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9ebc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e410d9c0 con 0x7fb9e4105360 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9ebc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e410dcc0 con 0x7fb9e4102a30 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9e9200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9e4102a30 0x7fb9e41980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:27.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9e9200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9e4102a30 0x7fb9e41980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:55838/0 (socket says 172.21.15.112:55838) 2024-04-25T18:47:27.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9e9200640 1 -- 172.21.15.112:0/1978089935 learned_addr learned my addr 172.21.15.112:0/1978089935 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:27.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9e9c00640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e4107c90 0x7fb9e419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:27.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9e3e00640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9e4105360 0x7fb9e41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:27.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9e9200640 1 -- 172.21.15.112:0/1978089935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9e4105360 msgr2=0x7fb9e41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9e9200640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9e4105360 0x7fb9e41985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9e9200640 1 -- 172.21.15.112:0/1978089935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e4107c90 msgr2=0x7fb9e419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9e9200640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e4107c90 0x7fb9e419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9e9200640 1 -- 172.21.15.112:0/1978089935 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e419fcd0 con 0x7fb9e4102a30 2024-04-25T18:47:27.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.427+0000 7fb9e9200640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9e4102a30 0x7fb9e41980b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800e9f0 tx=0x7fb9d800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:27.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.428+0000 7fb9e1600640 1 -- 172.21.15.112:0/1978089935 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800cd80 con 0x7fb9e4102a30 2024-04-25T18:47:27.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.428+0000 7fb9e1600640 1 -- 172.21.15.112:0/1978089935 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9d80108e0 con 0x7fb9e4102a30 2024-04-25T18:47:27.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.428+0000 7fb9ebc00640 1 -- 172.21.15.112:0/1978089935 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e419ffc0 con 0x7fb9e4102a30 2024-04-25T18:47:27.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.428+0000 7fb9e1600640 1 -- 172.21.15.112:0/1978089935 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8018980 con 0x7fb9e4102a30 2024-04-25T18:47:27.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.428+0000 7fb9ebc00640 1 -- 172.21.15.112:0/1978089935 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e41a0480 con 0x7fb9e4102a30 2024-04-25T18:47:27.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.429+0000 7fb9ebc00640 1 -- 172.21.15.112:0/1978089935 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e40fdf30 con 0x7fb9e4102a30 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.431+0000 7fb9e1600640 1 -- 172.21.15.112:0/1978089935 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7fb9d8010430 con 0x7fb9e4102a30 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.432+0000 7fb9e1600640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb9cc0a3ee0 0x7fb9cc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.432+0000 7fb9e1600640 1 -- 172.21.15.112:0/1978089935 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fb9d80d0cd0 con 0x7fb9e4102a30 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.433+0000 7fb9e3e00640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb9cc0a3ee0 0x7fb9cc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:27.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.433+0000 7fb9e3e00640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb9cc0a3ee0 0x7fb9cc0a63a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb9e41995c0 tx=0x7fb9d0005f90 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:27.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.437+0000 7fb9e1600640 1 -- 172.21.15.112:0/1978089935 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9d809cb50 con 0x7fb9e4102a30 2024-04-25T18:47:27.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.700+0000 7fb9ebc00640 1 -- 172.21.15.112:0/1978089935 --> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb9e4062390 con 0x7fb9cc0a3ee0 2024-04-25T18:47:27.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.709+0000 7fb9e1600640 1 -- 172.21.15.112:0/1978089935 <== mgr.14922 v2:172.21.15.112:6800/2236031982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fb9e4062390 con 0x7fb9cc0a3ee0 2024-04-25T18:47:27.711 INFO:teuthology.orchestra.run.smithi112.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T18:47:26", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T18:47:27.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.711+0000 7fb9ebc00640 1 -- 172.21.15.112:0/1978089935 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb9cc0a3ee0 msgr2=0x7fb9cc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.711+0000 7fb9ebc00640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb9cc0a3ee0 0x7fb9cc0a63a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb9e41995c0 tx=0x7fb9d0005f90 comp rx=0 tx=0).stop 2024-04-25T18:47:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.711+0000 7fb9ebc00640 1 -- 172.21.15.112:0/1978089935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9e4102a30 msgr2=0x7fb9e41980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.711+0000 7fb9ebc00640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9e4102a30 0x7fb9e41980b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800e9f0 tx=0x7fb9d800eeb0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.712+0000 7fb9ebc00640 1 -- 172.21.15.112:0/1978089935 shutdown_connections 2024-04-25T18:47:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.712+0000 7fb9ebc00640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb9cc0a3ee0 0x7fb9cc0a63a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.712+0000 7fb9ebc00640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9e4107c90 0x7fb9e419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.712+0000 7fb9ebc00640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9e4105360 0x7fb9e41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.712+0000 7fb9ebc00640 1 --2- 172.21.15.112:0/1978089935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9e4102a30 0x7fb9e41980b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.712+0000 7fb9ebc00640 1 -- 172.21.15.112:0/1978089935 >> 172.21.15.112:0/1978089935 conn(0x7fb9e40fc440 msgr2=0x7fb9e4103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.712+0000 7fb9ebc00640 1 -- 172.21.15.112:0/1978089935 shutdown_connections 2024-04-25T18:47:27.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.712+0000 7fb9ebc00640 1 -- 172.21.15.112:0/1978089935 wait complete. 2024-04-25T18:47:27.732 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T18:47:27.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.909+0000 7f648b400640 1 -- 172.21.15.112:0/2345715265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64841031a0 msgr2=0x7f6484103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.909+0000 7f648b400640 1 --2- 172.21.15.112:0/2345715265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64841031a0 0x7f6484103620 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6474009900 tx=0x7f647402a3f0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.909+0000 7f648b400640 1 -- 172.21.15.112:0/2345715265 shutdown_connections 2024-04-25T18:47:27.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.909+0000 7f648b400640 1 --2- 172.21.15.112:0/2345715265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6484103b60 0x7f648410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.909+0000 7f648b400640 1 --2- 172.21.15.112:0/2345715265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64841031a0 0x7f6484103620 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.909+0000 7f648b400640 1 --2- 172.21.15.112:0/2345715265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6484101fb0 0x7f64841023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.909+0000 7f648b400640 1 -- 172.21.15.112:0/2345715265 >> 172.21.15.112:0/2345715265 conn(0x7f64840fd540 msgr2=0x7f64840ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:27.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.909+0000 7f648b400640 1 -- 172.21.15.112:0/2345715265 shutdown_connections 2024-04-25T18:47:27.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.909+0000 7f648b400640 1 -- 172.21.15.112:0/2345715265 wait complete. 2024-04-25T18:47:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.910+0000 7f648b400640 1 Processor -- start 2024-04-25T18:47:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.910+0000 7f648b400640 1 -- start start 2024-04-25T18:47:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f648b400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484101fb0 0x7f6484198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f648b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64841031a0 0x7f6484198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f648b400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6484103b60 0x7f648419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f648b400640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f648410db40 con 0x7f6484101fb0 2024-04-25T18:47:27.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f648b400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f648410d9c0 con 0x7f64841031a0 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f648b400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f648410dcc0 con 0x7f6484103b60 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f6488a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484101fb0 0x7f6484198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f6483e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64841031a0 0x7f6484198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f6488a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484101fb0 0x7f6484198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43388/0 (socket says 172.21.15.112:43388) 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f6488a00640 1 -- 172.21.15.112:0/2312672622 learned_addr learned my addr 172.21.15.112:0/2312672622 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f6489400640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6484103b60 0x7f648419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f6483e00640 1 -- 172.21.15.112:0/2312672622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6484103b60 msgr2=0x7f648419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f6483e00640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6484103b60 0x7f648419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f6483e00640 1 -- 172.21.15.112:0/2312672622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484101fb0 msgr2=0x7f6484198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f6483e00640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484101fb0 0x7f6484198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f6483e00640 1 -- 172.21.15.112:0/2312672622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648419fd40 con 0x7f64841031a0 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.911+0000 7f6488a00640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484101fb0 0x7f6484198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:27.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.912+0000 7f6483e00640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64841031a0 0x7f6484198660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f647402a9e0 tx=0x7f6474003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:27.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.913+0000 7f6481600640 1 -- 172.21.15.112:0/2312672622 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647403f070 con 0x7f64841031a0 2024-04-25T18:47:27.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.913+0000 7f6481600640 1 -- 172.21.15.112:0/2312672622 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6474031c50 con 0x7f64841031a0 2024-04-25T18:47:27.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.913+0000 7f6481600640 1 -- 172.21.15.112:0/2312672622 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474035030 con 0x7f64841031a0 2024-04-25T18:47:27.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.913+0000 7f648b400640 1 -- 172.21.15.112:0/2312672622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648419ffd0 con 0x7f64841031a0 2024-04-25T18:47:27.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.913+0000 7f648b400640 1 -- 172.21.15.112:0/2312672622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64841a0440 con 0x7f64841031a0 2024-04-25T18:47:27.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.914+0000 7f648b400640 1 -- 172.21.15.112:0/2312672622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64841092d0 con 0x7f64841031a0 2024-04-25T18:47:27.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.915+0000 7f6481600640 1 -- 172.21.15.112:0/2312672622 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f6474042070 con 0x7f64841031a0 2024-04-25T18:47:27.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.917+0000 7f6481600640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f646c0a3ee0 0x7f646c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.917+0000 7f6481600640 1 -- 172.21.15.112:0/2312672622 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f64740f0fd0 con 0x7f64841031a0 2024-04-25T18:47:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.917+0000 7f6488a00640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f646c0a3ee0 0x7f646c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:27.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.918+0000 7f6488a00640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f646c0a3ee0 0x7f646c0a63a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f64780058c0 tx=0x7f6478005f90 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:27.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:27.925+0000 7f6481600640 1 -- 172.21.15.112:0/2312672622 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f64740bce50 con 0x7f64841031a0 2024-04-25T18:47:27.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:27 smithi161 ceph-mon[32609]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:27.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:27 smithi161 ceph-mon[32609]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:27.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:27 smithi161 ceph-mon[32609]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T18:47:27.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:27 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:27.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:27 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:27.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:27 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:27.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:27 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:27.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:27 smithi161 ceph-mon[32609]: from='client.25127 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:27.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:27 smithi112 ceph-mon[27864]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:27.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:27 smithi112 ceph-mon[27864]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:27.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:27 smithi112 ceph-mon[27864]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T18:47:27.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:27 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:27.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:27 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:27.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:27 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:27.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:27 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:27.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:27 smithi112 ceph-mon[27864]: from='client.25127 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:28.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.158+0000 7f648b400640 1 -- 172.21.15.112:0/2312672622 --> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f6484062390 con 0x7f646c0a3ee0 2024-04-25T18:47:28.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.168+0000 7f6481600640 1 -- 172.21.15.112:0/2312672622 <== mgr.14922 v2:172.21.15.112:6800/2236031982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f6484062390 con 0x7f646c0a3ee0 2024-04-25T18:47:28.170 INFO:teuthology.orchestra.run.smithi112.stdout:/ 1m 6m3h 2024-04-25T18:47:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 -- 172.21.15.112:0/2312672622 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f646c0a3ee0 msgr2=0x7f646c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f646c0a3ee0 0x7f646c0a63a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f64780058c0 tx=0x7f6478005f90 comp rx=0 tx=0).stop 2024-04-25T18:47:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 -- 172.21.15.112:0/2312672622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64841031a0 msgr2=0x7f6484198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64841031a0 0x7f6484198660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f647402a9e0 tx=0x7f6474003c10 comp rx=0 tx=0).stop 2024-04-25T18:47:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 -- 172.21.15.112:0/2312672622 shutdown_connections 2024-04-25T18:47:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f646c0a3ee0 0x7f646c0a63a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6484103b60 0x7f648419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64841031a0 0x7f6484198660 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 --2- 172.21.15.112:0/2312672622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484101fb0 0x7f6484198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 -- 172.21.15.112:0/2312672622 >> 172.21.15.112:0/2312672622 conn(0x7f64840fd540 msgr2=0x7f64841064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:28.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 -- 172.21.15.112:0/2312672622 shutdown_connections 2024-04-25T18:47:28.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:28.171+0000 7f648b400640 1 -- 172.21.15.112:0/2312672622 wait complete. 2024-04-25T18:47:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:27 smithi171 ceph-mon[33266]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:27 smithi171 ceph-mon[33266]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:27 smithi171 ceph-mon[33266]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T18:47:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:27 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:27 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:27 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:27 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:27 smithi171 ceph-mon[33266]: from='client.25127 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:28.190 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T18:47:28.262 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T18:47:28.276 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T18:47:28.318 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T18:47:28.318 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T18:47:28.326 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T18:47:28.338 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T18:47:28.338 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:47:28.757 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:29.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='client.25138 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T18:47:29.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T18:47:29.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:28 smithi171 ceph-mon[33266]: from='client.25139 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:29.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='client.25138 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:29.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.223 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-25T18:47:29.224 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T18:47:29.224 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:28 smithi161 ceph-mon[32609]: from='client.25139 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:29.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='client.25138 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:29.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:29.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-25T18:47:29.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T18:47:29.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:28 smithi112 ceph-mon[27864]: from='client.25139 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T18:47:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:47:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:30 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:47:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T18:47:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:47:30.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:30 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:47:30.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:30.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T18:47:30.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:30.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:30.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:30.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:47:30.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:30 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:47:31.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:31 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi112 to 176.4M 2024-04-25T18:47:31.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:31 smithi161 ceph-mon[32609]: Unable to set osd_memory_target on smithi112 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-04-25T18:47:31.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:31 smithi161 ceph-mon[32609]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T18:47:31.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:31 smithi161 ceph-mon[32609]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-25T18:47:31.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:31 smithi161 ceph-mon[32609]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T18:47:31.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:31 smithi161 ceph-mon[32609]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:47:31.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:31 smithi161 ceph-mon[32609]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:47:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.322+0000 7f6576c00700 1 -- 172.21.15.112:0/3408341732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657010f4f0 msgr2=0x7f65701118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.322+0000 7f6576c00700 1 --2- 172.21.15.112:0/3408341732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657010f4f0 0x7f65701118e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f656800b0a0 tx=0x7f656800b3b0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.325 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:31 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi112 to 176.4M 2024-04-25T18:47:31.325 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:31 smithi112 ceph-mon[27864]: Unable to set osd_memory_target on smithi112 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-04-25T18:47:31.326 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:31 smithi112 ceph-mon[27864]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T18:47:31.326 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:31 smithi112 ceph-mon[27864]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-25T18:47:31.326 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:31 smithi112 ceph-mon[27864]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T18:47:31.326 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:31 smithi112 ceph-mon[27864]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:47:31.326 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:31 smithi112 ceph-mon[27864]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.323+0000 7f6576c00700 1 -- 172.21.15.112:0/3408341732 shutdown_connections 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.323+0000 7f6576c00700 1 --2- 172.21.15.112:0/3408341732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657010f4f0 0x7f65701118e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.323+0000 7f6576c00700 1 --2- 172.21.15.112:0/3408341732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f657006a750 0x7f657006ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.323+0000 7f6576c00700 1 --2- 172.21.15.112:0/3408341732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f657006b070 0x7f657010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.323+0000 7f6576c00700 1 -- 172.21.15.112:0/3408341732 >> 172.21.15.112:0/3408341732 conn(0x7f65700788f0 msgr2=0x7f6570078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.323+0000 7f6576c00700 1 -- 172.21.15.112:0/3408341732 shutdown_connections 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.323+0000 7f6576c00700 1 -- 172.21.15.112:0/3408341732 wait complete. 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.324+0000 7f6576c00700 1 Processor -- start 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.324+0000 7f6576c00700 1 -- start start 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f6576c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657006a750 0x7f657019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f6576c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f657006b070 0x7f657019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f6576c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f657010f4f0 0x7f657019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:31.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f6576c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6570114e10 con 0x7f657006a750 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f6576c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6570114c90 con 0x7f657006b070 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f6576c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6570114f90 con 0x7f657010f4f0 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f656fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657006a750 0x7f657019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f656f400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f657006b070 0x7f657019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f656f400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f657006b070 0x7f657019b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:57432/0 (socket says 172.21.15.112:57432) 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f656f400700 1 -- 172.21.15.112:0/1393101773 learned_addr learned my addr 172.21.15.112:0/1393101773 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f656fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657006a750 0x7f657019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34656/0 (socket says 172.21.15.112:34656) 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f6574c00700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f657010f4f0 0x7f657019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f656f400700 1 -- 172.21.15.112:0/1393101773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f657010f4f0 msgr2=0x7f657019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f656f400700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f657010f4f0 0x7f657019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.325+0000 7f656f400700 1 -- 172.21.15.112:0/1393101773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657006a750 msgr2=0x7f657019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.326+0000 7f656f400700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657006a750 0x7f657019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.326+0000 7f656f400700 1 -- 172.21.15.112:0/1393101773 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65701a02e0 con 0x7f657006b070 2024-04-25T18:47:31.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.326+0000 7f656fe00700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657006a750 0x7f657019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:31.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.326+0000 7f656f400700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f657006b070 0x7f657019b860 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f655c00eb30 tx=0x7f655c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:31.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.326+0000 7f656cc00700 1 -- 172.21.15.112:0/1393101773 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c00cc90 con 0x7f657006b070 2024-04-25T18:47:31.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.326+0000 7f656cc00700 1 -- 172.21.15.112:0/1393101773 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655c0044d0 con 0x7f657006b070 2024-04-25T18:47:31.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.326+0000 7f6576c00700 1 -- 172.21.15.112:0/1393101773 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65701a05d0 con 0x7f657006b070 2024-04-25T18:47:31.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.327+0000 7f6576c00700 1 -- 172.21.15.112:0/1393101773 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65701a7e50 con 0x7f657006b070 2024-04-25T18:47:31.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.327+0000 7f656cc00700 1 -- 172.21.15.112:0/1393101773 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c010430 con 0x7f657006b070 2024-04-25T18:47:31.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.328+0000 7f6576c00700 1 -- 172.21.15.112:0/1393101773 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6570069730 con 0x7f657006b070 2024-04-25T18:47:31.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.329+0000 7f656cc00700 1 -- 172.21.15.112:0/1393101773 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f655c003ea0 con 0x7f657006b070 2024-04-25T18:47:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.331+0000 7f656cc00700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f65540a3f50 0x7f65540a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.332+0000 7f656cc00700 1 -- 172.21.15.112:0/1393101773 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f655c0d0c80 con 0x7f657006b070 2024-04-25T18:47:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.332+0000 7f656fe00700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f65540a3f50 0x7f65540a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.332+0000 7f656fe00700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f65540a3f50 0x7f65540a6410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6560011490 tx=0x7f6560009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.335+0000 7f656cc00700 1 -- 172.21.15.112:0/1393101773 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f655c09d070 con 0x7f657006b070 2024-04-25T18:47:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:31 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi112 to 176.4M 2024-04-25T18:47:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:31 smithi171 ceph-mon[33266]: Unable to set osd_memory_target on smithi112 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-04-25T18:47:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:31 smithi171 ceph-mon[33266]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T18:47:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:31 smithi171 ceph-mon[33266]: Updating smithi161:/etc/ceph/ceph.conf 2024-04-25T18:47:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:31 smithi171 ceph-mon[33266]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T18:47:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:31 smithi171 ceph-mon[33266]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:47:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:31 smithi171 ceph-mon[33266]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:47:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.558+0000 7f6576c00700 1 -- 172.21.15.112:0/1393101773 --> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f657019c630 con 0x7f65540a3f50 2024-04-25T18:47:31.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.568+0000 7f656cc00700 1 -- 172.21.15.112:0/1393101773 <== mgr.14922 v2:172.21.15.112:6800/2236031982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257785 (secure 0 0 0) 0x7f657019c630 con 0x7f65540a3f50 2024-04-25T18:47:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:47:31.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 -- 172.21.15.112:0/1393101773 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f65540a3f50 msgr2=0x7f65540a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:31.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f65540a3f50 0x7f65540a6410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6560011490 tx=0x7f6560009260 comp rx=0 tx=0).stop 2024-04-25T18:47:31.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 -- 172.21.15.112:0/1393101773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f657006b070 msgr2=0x7f657019b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:31.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f657006b070 0x7f657019b860 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f655c00eb30 tx=0x7f655c00ee40 comp rx=0 tx=0).stop 2024-04-25T18:47:31.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 -- 172.21.15.112:0/1393101773 shutdown_connections 2024-04-25T18:47:31.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f65540a3f50 0x7f65540a6410 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f657010f4f0 0x7f657019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f657006a750 0x7f657019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 --2- 172.21.15.112:0/1393101773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f657006b070 0x7f657019b860 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 -- 172.21.15.112:0/1393101773 >> 172.21.15.112:0/1393101773 conn(0x7f65700788f0 msgr2=0x7f65701109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:31.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 -- 172.21.15.112:0/1393101773 shutdown_connections 2024-04-25T18:47:31.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:31.573+0000 7f6576c00700 1 -- 172.21.15.112:0/1393101773 wait complete. 2024-04-25T18:47:31.577 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T18:47:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:32 smithi161 ceph-mon[32609]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:32 smithi161 ceph-mon[32609]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:32.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:32 smithi161 ceph-mon[32609]: Updating smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:47:32.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:32 smithi161 ceph-mon[32609]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:32.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:32 smithi161 ceph-mon[32609]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:47:32.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:32 smithi161 ceph-mon[32609]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:47:32.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:32 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:32 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:32 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:32 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:32 smithi161 ceph-mon[32609]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:32 smithi112 ceph-mon[27864]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:32 smithi112 ceph-mon[27864]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:32 smithi112 ceph-mon[27864]: Updating smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:47:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:32 smithi112 ceph-mon[27864]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:32 smithi112 ceph-mon[27864]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:47:32.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:32 smithi112 ceph-mon[27864]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:47:32.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:32 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:32 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:32 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:32 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:32 smithi112 ceph-mon[27864]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:32.563 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-25T18:47:30.651262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191389,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1605,"num_read_kb":960,"num_write":477,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181318,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":868,"ondisk_log_size":868,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381384,"kb_used_data":15508,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560440,"statfs":{"total":1151940427776,"available":1151549890560,"internally_reserved":0,"allocated":15880192,"data_stored":9233370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.963310"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689752+0000","last_change":"2024-04-25T18:43:16.262618+0000","last_active":"2024-04-25T18:47:24.689752+0000","last_peered":"2024-04-25T18:47:24.689752+0000","last_clean":"2024-04-25T18:47:24.689752+0000","last_became_active":"2024-04-25T18:43:16.262215+0000","last_became_peered":"2024-04-25T18:43:16.262215+0000","last_unstale":"2024-04-25T18:47:24.689752+0000","last_undegraded":"2024-04-25T18:47:24.689752+0000","last_fullsized":"2024-04-25T18:47:24.689752+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:40.319102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981786+0000","last_change":"2024-04-25T18:43:16.254085+0000","last_active":"2024-04-25T18:47:24.981786+0000","last_peered":"2024-04-25T18:47:24.981786+0000","last_clean":"2024-04-25T18:47:24.981786+0000","last_became_active":"2024-04-25T18:43:16.248943+0000","last_became_peered":"2024-04-25T18:43:16.248943+0000","last_unstale":"2024-04-25T18:47:24.981786+0000","last_undegraded":"2024-04-25T18:47:24.981786+0000","last_fullsized":"2024-04-25T18:47:24.981786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:53.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981717+0000","last_change":"2024-04-25T18:43:16.252684+0000","last_active":"2024-04-25T18:47:24.981717+0000","last_peered":"2024-04-25T18:47:24.981717+0000","last_clean":"2024-04-25T18:47:24.981717+0000","last_became_active":"2024-04-25T18:43:16.252318+0000","last_became_peered":"2024-04-25T18:43:16.252318+0000","last_unstale":"2024-04-25T18:47:24.981717+0000","last_undegraded":"2024-04-25T18:47:24.981717+0000","last_fullsized":"2024-04-25T18:47:24.981717+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:58.904865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631697+0000","last_change":"2024-04-25T18:43:16.247059+0000","last_active":"2024-04-25T18:47:24.631697+0000","last_peered":"2024-04-25T18:47:24.631697+0000","last_clean":"2024-04-25T18:47:24.631697+0000","last_became_active":"2024-04-25T18:43:16.246655+0000","last_became_peered":"2024-04-25T18:43:16.246655+0000","last_unstale":"2024-04-25T18:47:24.631697+0000","last_undegraded":"2024-04-25T18:47:24.631697+0000","last_fullsized":"2024-04-25T18:47:24.631697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:31.026579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628963+0000","last_change":"2024-04-25T18:43:16.250033+0000","last_active":"2024-04-25T18:47:24.628963+0000","last_peered":"2024-04-25T18:47:24.628963+0000","last_clean":"2024-04-25T18:47:24.628963+0000","last_became_active":"2024-04-25T18:43:16.249017+0000","last_became_peered":"2024-04-25T18:43:16.249017+0000","last_unstale":"2024-04-25T18:47:24.628963+0000","last_undegraded":"2024-04-25T18:47:24.628963+0000","last_fullsized":"2024-04-25T18:47:24.628963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:34.304263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641422+0000","last_change":"2024-04-25T18:43:56.642410+0000","last_active":"2024-04-25T18:47:24.641422+0000","last_peered":"2024-04-25T18:47:24.641422+0000","last_clean":"2024-04-25T18:47:24.641422+0000","last_became_active":"2024-04-25T18:43:56.642009+0000","last_became_peered":"2024-04-25T18:43:56.642009+0000","last_unstale":"2024-04-25T18:47:24.641422+0000","last_undegraded":"2024-04-25T18:47:24.641422+0000","last_fullsized":"2024-04-25T18:47:24.641422+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:19.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804293+0000","last_change":"2024-04-25T18:43:56.647158+0000","last_active":"2024-04-25T18:47:24.804293+0000","last_peered":"2024-04-25T18:47:24.804293+0000","last_clean":"2024-04-25T18:47:24.804293+0000","last_became_active":"2024-04-25T18:43:56.646779+0000","last_became_peered":"2024-04-25T18:43:56.646779+0000","last_unstale":"2024-04-25T18:47:24.804293+0000","last_undegraded":"2024-04-25T18:47:24.804293+0000","last_fullsized":"2024-04-25T18:47:24.804293+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:04:14.055762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.766601+0000","last_change":"2024-04-25T18:43:16.247607+0000","last_active":"2024-04-25T18:47:24.766601+0000","last_peered":"2024-04-25T18:47:24.766601+0000","last_clean":"2024-04-25T18:47:24.766601+0000","last_became_active":"2024-04-25T18:43:16.247234+0000","last_became_peered":"2024-04-25T18:43:16.247234+0000","last_unstale":"2024-04-25T18:47:24.766601+0000","last_undegraded":"2024-04-25T18:47:24.766601+0000","last_fullsized":"2024-04-25T18:47:24.766601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:57.117900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973993+0000","last_change":"2024-04-25T18:43:16.247909+0000","last_active":"2024-04-25T18:47:24.973993+0000","last_peered":"2024-04-25T18:47:24.973993+0000","last_clean":"2024-04-25T18:47:24.973993+0000","last_became_active":"2024-04-25T18:43:16.247533+0000","last_became_peered":"2024-04-25T18:43:16.247533+0000","last_unstale":"2024-04-25T18:47:24.973993+0000","last_undegraded":"2024-04-25T18:47:24.973993+0000","last_fullsized":"2024-04-25T18:47:24.973993+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:02.257832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641756+0000","last_change":"2024-04-25T18:43:16.252088+0000","last_active":"2024-04-25T18:47:24.641756+0000","last_peered":"2024-04-25T18:47:24.641756+0000","last_clean":"2024-04-25T18:47:24.641756+0000","last_became_active":"2024-04-25T18:43:16.251580+0000","last_became_peered":"2024-04-25T18:43:16.251580+0000","last_unstale":"2024-04-25T18:47:24.641756+0000","last_undegraded":"2024-04-25T18:47:24.641756+0000","last_fullsized":"2024-04-25T18:47:24.641756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:22.992087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804685+0000","last_change":"2024-04-25T18:43:16.239696+0000","last_active":"2024-04-25T18:47:24.804685+0000","last_peered":"2024-04-25T18:47:24.804685+0000","last_clean":"2024-04-25T18:47:24.804685+0000","last_became_active":"2024-04-25T18:43:16.239354+0000","last_became_peered":"2024-04-25T18:43:16.239354+0000","last_unstale":"2024-04-25T18:47:24.804685+0000","last_undegraded":"2024-04-25T18:47:24.804685+0000","last_fullsized":"2024-04-25T18:47:24.804685+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:39.682138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631515+0000","last_change":"2024-04-25T18:43:16.242487+0000","last_active":"2024-04-25T18:47:24.631515+0000","last_peered":"2024-04-25T18:47:24.631515+0000","last_clean":"2024-04-25T18:47:24.631515+0000","last_became_active":"2024-04-25T18:43:16.242158+0000","last_became_peered":"2024-04-25T18:43:16.242158+0000","last_unstale":"2024-04-25T18:47:24.631515+0000","last_undegraded":"2024-04-25T18:47:24.631515+0000","last_fullsized":"2024-04-25T18:47:24.631515+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:36.477440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.630730+0000","last_change":"2024-04-25T18:43:16.244154+0000","last_active":"2024-04-25T18:47:24.630730+0000","last_peered":"2024-04-25T18:47:24.630730+0000","last_clean":"2024-04-25T18:47:24.630730+0000","last_became_active":"2024-04-25T18:43:16.243829+0000","last_became_peered":"2024-04-25T18:43:16.243829+0000","last_unstale":"2024-04-25T18:47:24.630730+0000","last_undegraded":"2024-04-25T18:47:24.630730+0000","last_fullsized":"2024-04-25T18:47:24.630730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:43.262751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981216+0000","last_change":"2024-04-25T18:43:16.261227+0000","last_active":"2024-04-25T18:47:24.981216+0000","last_peered":"2024-04-25T18:47:24.981216+0000","last_clean":"2024-04-25T18:47:24.981216+0000","last_became_active":"2024-04-25T18:43:16.250468+0000","last_became_peered":"2024-04-25T18:43:16.250468+0000","last_unstale":"2024-04-25T18:47:24.981216+0000","last_undegraded":"2024-04-25T18:47:24.981216+0000","last_fullsized":"2024-04-25T18:47:24.981216+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:44.805643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981094+0000","last_change":"2024-04-25T18:43:16.257956+0000","last_active":"2024-04-25T18:47:24.981094+0000","last_peered":"2024-04-25T18:47:24.981094+0000","last_clean":"2024-04-25T18:47:24.981094+0000","last_became_active":"2024-04-25T18:43:16.255126+0000","last_became_peered":"2024-04-25T18:43:16.255126+0000","last_unstale":"2024-04-25T18:47:24.981094+0000","last_undegraded":"2024-04-25T18:47:24.981094+0000","last_fullsized":"2024-04-25T18:47:24.981094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:30.047165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973501+0000","last_change":"2024-04-25T18:43:16.256208+0000","last_active":"2024-04-25T18:47:24.973501+0000","last_peered":"2024-04-25T18:47:24.973501+0000","last_clean":"2024-04-25T18:47:24.973501+0000","last_became_active":"2024-04-25T18:43:16.255614+0000","last_became_peered":"2024-04-25T18:43:16.255614+0000","last_unstale":"2024-04-25T18:47:24.973501+0000","last_undegraded":"2024-04-25T18:47:24.973501+0000","last_fullsized":"2024-04-25T18:47:24.973501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:46.700209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689815+0000","last_change":"2024-04-25T18:43:16.262730+0000","last_active":"2024-04-25T18:47:24.689815+0000","last_peered":"2024-04-25T18:47:24.689815+0000","last_clean":"2024-04-25T18:47:24.689815+0000","last_became_active":"2024-04-25T18:43:16.262192+0000","last_became_peered":"2024-04-25T18:43:16.262192+0000","last_unstale":"2024-04-25T18:47:24.689815+0000","last_undegraded":"2024-04-25T18:47:24.689815+0000","last_fullsized":"2024-04-25T18:47:24.689815+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:29.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.763135+0000","last_change":"2024-04-25T18:43:16.261161+0000","last_active":"2024-04-25T18:47:24.763135+0000","last_peered":"2024-04-25T18:47:24.763135+0000","last_clean":"2024-04-25T18:47:24.763135+0000","last_became_active":"2024-04-25T18:43:16.258217+0000","last_became_peered":"2024-04-25T18:43:16.258217+0000","last_unstale":"2024-04-25T18:47:24.763135+0000","last_undegraded":"2024-04-25T18:47:24.763135+0000","last_fullsized":"2024-04-25T18:47:24.763135+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.564609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973806+0000","last_change":"2024-04-25T18:43:16.266660+0000","last_active":"2024-04-25T18:47:24.973806+0000","last_peered":"2024-04-25T18:47:24.973806+0000","last_clean":"2024-04-25T18:47:24.973806+0000","last_became_active":"2024-04-25T18:43:16.266295+0000","last_became_peered":"2024-04-25T18:43:16.266295+0000","last_unstale":"2024-04-25T18:47:24.973806+0000","last_undegraded":"2024-04-25T18:47:24.973806+0000","last_fullsized":"2024-04-25T18:47:24.973806+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:42.406585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.763304+0000","last_change":"2024-04-25T18:43:12.200172+0000","last_active":"2024-04-25T18:47:24.763304+0000","last_peered":"2024-04-25T18:47:24.763304+0000","last_clean":"2024-04-25T18:47:24.763304+0000","last_became_active":"2024-04-25T18:43:12.199760+0000","last_became_peered":"2024-04-25T18:43:12.199760+0000","last_unstale":"2024-04-25T18:47:24.763304+0000","last_undegraded":"2024-04-25T18:47:24.763304+0000","last_fullsized":"2024-04-25T18:47:24.763304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:57.460274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768028+0000","last_change":"2024-04-25T18:43:16.255989+0000","last_active":"2024-04-25T18:47:24.768028+0000","last_peered":"2024-04-25T18:47:24.768028+0000","last_clean":"2024-04-25T18:47:24.768028+0000","last_became_active":"2024-04-25T18:43:16.255114+0000","last_became_peered":"2024-04-25T18:43:16.255114+0000","last_unstale":"2024-04-25T18:47:24.768028+0000","last_undegraded":"2024-04-25T18:47:24.768028+0000","last_fullsized":"2024-04-25T18:47:24.768028+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:52.888876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768305+0000","last_change":"2024-04-25T18:43:12.208194+0000","last_active":"2024-04-25T18:47:24.768305+0000","last_peered":"2024-04-25T18:47:24.768305+0000","last_clean":"2024-04-25T18:47:24.768305+0000","last_became_active":"2024-04-25T18:43:12.207801+0000","last_became_peered":"2024-04-25T18:43:12.207801+0000","last_unstale":"2024-04-25T18:47:24.768305+0000","last_undegraded":"2024-04-25T18:47:24.768305+0000","last_fullsized":"2024-04-25T18:47:24.768305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:24.742486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981924+0000","last_change":"2024-04-25T18:43:16.268960+0000","last_active":"2024-04-25T18:47:24.981924+0000","last_peered":"2024-04-25T18:47:24.981924+0000","last_clean":"2024-04-25T18:47:24.981924+0000","last_became_active":"2024-04-25T18:43:16.263626+0000","last_became_peered":"2024-04-25T18:43:16.263626+0000","last_unstale":"2024-04-25T18:47:24.981924+0000","last_undegraded":"2024-04-25T18:47:24.981924+0000","last_fullsized":"2024-04-25T18:47:24.981924+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:57.721918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629378+0000","last_change":"2024-04-25T18:43:57.110208+0000","last_active":"2024-04-25T18:47:24.629378+0000","last_peered":"2024-04-25T18:47:24.629378+0000","last_clean":"2024-04-25T18:47:24.629378+0000","last_became_active":"2024-04-25T18:43:57.106027+0000","last_became_peered":"2024-04-25T18:43:57.106027+0000","last_unstale":"2024-04-25T18:47:24.629378+0000","last_undegraded":"2024-04-25T18:47:24.629378+0000","last_fullsized":"2024-04-25T18:47:24.629378+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:20.819961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973596+0000","last_change":"2024-04-25T18:43:16.250414+0000","last_active":"2024-04-25T18:47:24.973596+0000","last_peered":"2024-04-25T18:47:24.973596+0000","last_clean":"2024-04-25T18:47:24.973596+0000","last_became_active":"2024-04-25T18:43:16.249832+0000","last_became_peered":"2024-04-25T18:43:16.249832+0000","last_unstale":"2024-04-25T18:47:24.973596+0000","last_undegraded":"2024-04-25T18:47:24.973596+0000","last_fullsized":"2024-04-25T18:47:24.973596+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.577854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629313+0000","last_change":"2024-04-25T18:43:12.195099+0000","last_active":"2024-04-25T18:47:24.629313+0000","last_peered":"2024-04-25T18:47:24.629313+0000","last_clean":"2024-04-25T18:47:24.629313+0000","last_became_active":"2024-04-25T18:43:12.194726+0000","last_became_peered":"2024-04-25T18:43:12.194726+0000","last_unstale":"2024-04-25T18:47:24.629313+0000","last_undegraded":"2024-04-25T18:47:24.629313+0000","last_fullsized":"2024-04-25T18:47:24.629313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.866905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644755+0000","last_change":"2024-04-25T18:43:16.245208+0000","last_active":"2024-04-25T18:47:24.644755+0000","last_peered":"2024-04-25T18:47:24.644755+0000","last_clean":"2024-04-25T18:47:24.644755+0000","last_became_active":"2024-04-25T18:43:16.244836+0000","last_became_peered":"2024-04-25T18:43:16.244836+0000","last_unstale":"2024-04-25T18:47:24.644755+0000","last_undegraded":"2024-04-25T18:47:24.644755+0000","last_fullsized":"2024-04-25T18:47:24.644755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:35.917060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":267,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:25.646331+0000","last_change":"2024-04-25T18:43:12.198252+0000","last_active":"2024-04-25T18:47:25.646331+0000","last_peered":"2024-04-25T18:47:25.646331+0000","last_clean":"2024-04-25T18:47:25.646331+0000","last_became_active":"2024-04-25T18:43:12.197691+0000","last_became_peered":"2024-04-25T18:43:12.197691+0000","last_unstale":"2024-04-25T18:47:25.646331+0000","last_undegraded":"2024-04-25T18:47:25.646331+0000","last_fullsized":"2024-04-25T18:47:25.646331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.900012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":230,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628891+0000","last_change":"2024-04-25T18:43:16.255576+0000","last_active":"2024-04-25T18:47:24.628891+0000","last_peered":"2024-04-25T18:47:24.628891+0000","last_clean":"2024-04-25T18:47:24.628891+0000","last_became_active":"2024-04-25T18:43:16.255195+0000","last_became_peered":"2024-04-25T18:43:16.255195+0000","last_unstale":"2024-04-25T18:47:24.628891+0000","last_undegraded":"2024-04-25T18:47:24.628891+0000","last_fullsized":"2024-04-25T18:47:24.628891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:41.510872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641395+0000","last_change":"2024-04-25T18:43:12.200786+0000","last_active":"2024-04-25T18:47:24.641395+0000","last_peered":"2024-04-25T18:47:24.641395+0000","last_clean":"2024-04-25T18:47:24.641395+0000","last_became_active":"2024-04-25T18:43:12.200409+0000","last_became_peered":"2024-04-25T18:43:12.200409+0000","last_unstale":"2024-04-25T18:47:24.641395+0000","last_undegraded":"2024-04-25T18:47:24.641395+0000","last_fullsized":"2024-04-25T18:47:24.641395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:52.177787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644567+0000","last_change":"2024-04-25T18:43:16.255563+0000","last_active":"2024-04-25T18:47:24.644567+0000","last_peered":"2024-04-25T18:47:24.644567+0000","last_clean":"2024-04-25T18:47:24.644567+0000","last_became_active":"2024-04-25T18:43:16.255174+0000","last_became_peered":"2024-04-25T18:43:16.255174+0000","last_unstale":"2024-04-25T18:47:24.644567+0000","last_undegraded":"2024-04-25T18:47:24.644567+0000","last_fullsized":"2024-04-25T18:47:24.644567+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:32:15.735051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631536+0000","last_change":"2024-04-25T18:43:12.210242+0000","last_active":"2024-04-25T18:47:24.631536+0000","last_peered":"2024-04-25T18:47:24.631536+0000","last_clean":"2024-04-25T18:47:24.631536+0000","last_became_active":"2024-04-25T18:43:12.209600+0000","last_became_peered":"2024-04-25T18:43:12.209600+0000","last_unstale":"2024-04-25T18:47:24.631536+0000","last_undegraded":"2024-04-25T18:47:24.631536+0000","last_fullsized":"2024-04-25T18:47:24.631536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.886712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.762109+0000","last_change":"2024-04-25T18:43:56.649102+0000","last_active":"2024-04-25T18:47:24.762109+0000","last_peered":"2024-04-25T18:47:24.762109+0000","last_clean":"2024-04-25T18:47:24.762109+0000","last_became_active":"2024-04-25T18:43:56.647127+0000","last_became_peered":"2024-04-25T18:43:56.647127+0000","last_unstale":"2024-04-25T18:47:24.762109+0000","last_undegraded":"2024-04-25T18:47:24.762109+0000","last_fullsized":"2024-04-25T18:47:24.762109+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:43.111539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644363+0000","last_change":"2024-04-25T18:43:12.198957+0000","last_active":"2024-04-25T18:47:24.644363+0000","last_peered":"2024-04-25T18:47:24.644363+0000","last_clean":"2024-04-25T18:47:24.644363+0000","last_became_active":"2024-04-25T18:43:12.198605+0000","last_became_peered":"2024-04-25T18:43:12.198605+0000","last_unstale":"2024-04-25T18:47:24.644363+0000","last_undegraded":"2024-04-25T18:47:24.644363+0000","last_fullsized":"2024-04-25T18:47:24.644363+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:11.759430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.762246+0000","last_change":"2024-04-25T18:43:16.266829+0000","last_active":"2024-04-25T18:47:24.762246+0000","last_peered":"2024-04-25T18:47:24.762246+0000","last_clean":"2024-04-25T18:47:24.762246+0000","last_became_active":"2024-04-25T18:43:16.266256+0000","last_became_peered":"2024-04-25T18:43:16.266256+0000","last_unstale":"2024-04-25T18:47:24.762246+0000","last_undegraded":"2024-04-25T18:47:24.762246+0000","last_fullsized":"2024-04-25T18:47:24.762246+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:43.343613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":259,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.976747+0000","last_change":"2024-04-25T18:43:12.201641+0000","last_active":"2024-04-25T18:47:24.976747+0000","last_peered":"2024-04-25T18:47:24.976747+0000","last_clean":"2024-04-25T18:47:24.976747+0000","last_became_active":"2024-04-25T18:43:12.201274+0000","last_became_peered":"2024-04-25T18:43:12.201274+0000","last_unstale":"2024-04-25T18:47:24.976747+0000","last_undegraded":"2024-04-25T18:47:24.976747+0000","last_fullsized":"2024-04-25T18:47:24.976747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:47.683850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":225,"num_read_kb":225,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.974010+0000","last_change":"2024-04-25T18:43:16.259551+0000","last_active":"2024-04-25T18:47:24.974010+0000","last_peered":"2024-04-25T18:47:24.974010+0000","last_clean":"2024-04-25T18:47:24.974010+0000","last_became_active":"2024-04-25T18:43:16.259165+0000","last_became_peered":"2024-04-25T18:43:16.259165+0000","last_unstale":"2024-04-25T18:47:24.974010+0000","last_undegraded":"2024-04-25T18:47:24.974010+0000","last_fullsized":"2024-04-25T18:47:24.974010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:23.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804701+0000","last_change":"2024-04-25T18:43:12.197534+0000","last_active":"2024-04-25T18:47:24.804701+0000","last_peered":"2024-04-25T18:47:24.804701+0000","last_clean":"2024-04-25T18:47:24.804701+0000","last_became_active":"2024-04-25T18:43:12.197106+0000","last_became_peered":"2024-04-25T18:43:12.197106+0000","last_unstale":"2024-04-25T18:47:24.804701+0000","last_undegraded":"2024-04-25T18:47:24.804701+0000","last_fullsized":"2024-04-25T18:47:24.804701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:35.399044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981722+0000","last_change":"2024-04-25T18:43:16.257668+0000","last_active":"2024-04-25T18:47:24.981722+0000","last_peered":"2024-04-25T18:47:24.981722+0000","last_clean":"2024-04-25T18:47:24.981722+0000","last_became_active":"2024-04-25T18:43:16.253609+0000","last_became_peered":"2024-04-25T18:43:16.253609+0000","last_unstale":"2024-04-25T18:47:24.981722+0000","last_undegraded":"2024-04-25T18:47:24.981722+0000","last_fullsized":"2024-04-25T18:47:24.981722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:36.048386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644803+0000","last_change":"2024-04-25T18:43:12.207322+0000","last_active":"2024-04-25T18:47:24.644803+0000","last_peered":"2024-04-25T18:47:24.644803+0000","last_clean":"2024-04-25T18:47:24.644803+0000","last_became_active":"2024-04-25T18:43:12.206947+0000","last_became_peered":"2024-04-25T18:43:12.206947+0000","last_unstale":"2024-04-25T18:47:24.644803+0000","last_undegraded":"2024-04-25T18:47:24.644803+0000","last_fullsized":"2024-04-25T18:47:24.644803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:20.536598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973667+0000","last_change":"2024-04-25T18:43:16.239575+0000","last_active":"2024-04-25T18:47:24.973667+0000","last_peered":"2024-04-25T18:47:24.973667+0000","last_clean":"2024-04-25T18:47:24.973667+0000","last_became_active":"2024-04-25T18:43:16.239208+0000","last_became_peered":"2024-04-25T18:43:16.239208+0000","last_unstale":"2024-04-25T18:47:24.973667+0000","last_undegraded":"2024-04-25T18:47:24.973667+0000","last_fullsized":"2024-04-25T18:47:24.973667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:58.195855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"95'39","reported_seq":527,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973323+0000","last_change":"2024-04-25T18:43:12.197199+0000","last_active":"2024-04-25T18:47:24.973323+0000","last_peered":"2024-04-25T18:47:24.973323+0000","last_clean":"2024-04-25T18:47:24.973323+0000","last_became_active":"2024-04-25T18:43:12.196776+0000","last_became_peered":"2024-04-25T18:43:12.196776+0000","last_unstale":"2024-04-25T18:47:24.973323+0000","last_undegraded":"2024-04-25T18:47:24.973323+0000","last_fullsized":"2024-04-25T18:47:24.973323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:54.149782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71438,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":459,"num_read_kb":293,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629197+0000","last_change":"2024-04-25T18:43:16.250993+0000","last_active":"2024-04-25T18:47:24.629197+0000","last_peered":"2024-04-25T18:47:24.629197+0000","last_clean":"2024-04-25T18:47:24.629197+0000","last_became_active":"2024-04-25T18:43:16.250216+0000","last_became_peered":"2024-04-25T18:43:16.250216+0000","last_unstale":"2024-04-25T18:47:24.629197+0000","last_undegraded":"2024-04-25T18:47:24.629197+0000","last_fullsized":"2024-04-25T18:47:24.629197+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:37.996463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973654+0000","last_change":"2024-04-25T18:43:12.204606+0000","last_active":"2024-04-25T18:47:24.973654+0000","last_peered":"2024-04-25T18:47:24.973654+0000","last_clean":"2024-04-25T18:47:24.973654+0000","last_became_active":"2024-04-25T18:43:12.204306+0000","last_became_peered":"2024-04-25T18:43:12.204306+0000","last_unstale":"2024-04-25T18:47:24.973654+0000","last_undegraded":"2024-04-25T18:47:24.973654+0000","last_fullsized":"2024-04-25T18:47:24.973654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:29.903775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.632833+0000","last_change":"2024-04-25T18:43:16.250093+0000","last_active":"2024-04-25T18:47:24.632833+0000","last_peered":"2024-04-25T18:47:24.632833+0000","last_clean":"2024-04-25T18:47:24.632833+0000","last_became_active":"2024-04-25T18:43:16.249567+0000","last_became_peered":"2024-04-25T18:43:16.249567+0000","last_unstale":"2024-04-25T18:47:24.632833+0000","last_undegraded":"2024-04-25T18:47:24.632833+0000","last_fullsized":"2024-04-25T18:47:24.632833+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:35.498641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689370+0000","last_change":"2024-04-25T18:43:56.652488+0000","last_active":"2024-04-25T18:47:24.689370+0000","last_peered":"2024-04-25T18:47:24.689370+0000","last_clean":"2024-04-25T18:47:24.689370+0000","last_became_active":"2024-04-25T18:43:56.647745+0000","last_became_peered":"2024-04-25T18:43:56.647745+0000","last_unstale":"2024-04-25T18:47:24.689370+0000","last_undegraded":"2024-04-25T18:47:24.689370+0000","last_fullsized":"2024-04-25T18:47:24.689370+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:33.474470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804304+0000","last_change":"2024-04-25T18:43:56.642673+0000","last_active":"2024-04-25T18:47:24.804304+0000","last_peered":"2024-04-25T18:47:24.804304+0000","last_clean":"2024-04-25T18:47:24.804304+0000","last_became_active":"2024-04-25T18:43:56.642313+0000","last_became_peered":"2024-04-25T18:43:56.642313+0000","last_unstale":"2024-04-25T18:47:24.804304+0000","last_undegraded":"2024-04-25T18:47:24.804304+0000","last_fullsized":"2024-04-25T18:47:24.804304+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:45.805738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":258,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631413+0000","last_change":"2024-04-25T18:43:12.197341+0000","last_active":"2024-04-25T18:47:24.631413+0000","last_peered":"2024-04-25T18:47:24.631413+0000","last_clean":"2024-04-25T18:47:24.631413+0000","last_became_active":"2024-04-25T18:43:12.196952+0000","last_became_peered":"2024-04-25T18:43:12.196952+0000","last_unstale":"2024-04-25T18:47:24.631413+0000","last_undegraded":"2024-04-25T18:47:24.631413+0000","last_fullsized":"2024-04-25T18:47:24.631413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:52.902579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":227,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804357+0000","last_change":"2024-04-25T18:43:16.245436+0000","last_active":"2024-04-25T18:47:24.804357+0000","last_peered":"2024-04-25T18:47:24.804357+0000","last_clean":"2024-04-25T18:47:24.804357+0000","last_became_active":"2024-04-25T18:43:16.244943+0000","last_became_peered":"2024-04-25T18:43:16.244943+0000","last_unstale":"2024-04-25T18:47:24.804357+0000","last_undegraded":"2024-04-25T18:47:24.804357+0000","last_fullsized":"2024-04-25T18:47:24.804357+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:03.402999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628839+0000","last_change":"2024-04-25T18:43:12.197642+0000","last_active":"2024-04-25T18:47:24.628839+0000","last_peered":"2024-04-25T18:47:24.628839+0000","last_clean":"2024-04-25T18:47:24.628839+0000","last_became_active":"2024-04-25T18:43:12.196395+0000","last_became_peered":"2024-04-25T18:43:12.196395+0000","last_unstale":"2024-04-25T18:47:24.628839+0000","last_undegraded":"2024-04-25T18:47:24.628839+0000","last_fullsized":"2024-04-25T18:47:24.628839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:11.783422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644371+0000","last_change":"2024-04-25T18:43:16.245344+0000","last_active":"2024-04-25T18:47:24.644371+0000","last_peered":"2024-04-25T18:47:24.644371+0000","last_clean":"2024-04-25T18:47:24.644371+0000","last_became_active":"2024-04-25T18:43:16.244984+0000","last_became_peered":"2024-04-25T18:43:16.244984+0000","last_unstale":"2024-04-25T18:47:24.644371+0000","last_undegraded":"2024-04-25T18:47:24.644371+0000","last_fullsized":"2024-04-25T18:47:24.644371+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:04.822629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629491+0000","last_change":"2024-04-25T18:43:12.199685+0000","last_active":"2024-04-25T18:47:24.629491+0000","last_peered":"2024-04-25T18:47:24.629491+0000","last_clean":"2024-04-25T18:47:24.629491+0000","last_became_active":"2024-04-25T18:43:12.199261+0000","last_became_peered":"2024-04-25T18:43:12.199261+0000","last_unstale":"2024-04-25T18:47:24.629491+0000","last_undegraded":"2024-04-25T18:47:24.629491+0000","last_fullsized":"2024-04-25T18:47:24.629491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:09.784616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689634+0000","last_change":"2024-04-25T18:43:16.247745+0000","last_active":"2024-04-25T18:47:24.689634+0000","last_peered":"2024-04-25T18:47:24.689634+0000","last_clean":"2024-04-25T18:47:24.689634+0000","last_became_active":"2024-04-25T18:43:16.247311+0000","last_became_peered":"2024-04-25T18:43:16.247311+0000","last_unstale":"2024-04-25T18:47:24.689634+0000","last_undegraded":"2024-04-25T18:47:24.689634+0000","last_fullsized":"2024-04-25T18:47:24.689634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:29.925464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.763129+0000","last_change":"2024-04-25T18:43:12.210209+0000","last_active":"2024-04-25T18:47:24.763129+0000","last_peered":"2024-04-25T18:47:24.763129+0000","last_clean":"2024-04-25T18:47:24.763129+0000","last_became_active":"2024-04-25T18:43:12.209708+0000","last_became_peered":"2024-04-25T18:43:12.209708+0000","last_unstale":"2024-04-25T18:47:24.763129+0000","last_undegraded":"2024-04-25T18:47:24.763129+0000","last_fullsized":"2024-04-25T18:47:24.763129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:07:18.331518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641610+0000","last_change":"2024-04-25T18:43:16.263616+0000","last_active":"2024-04-25T18:47:24.641610+0000","last_peered":"2024-04-25T18:47:24.641610+0000","last_clean":"2024-04-25T18:47:24.641610+0000","last_became_active":"2024-04-25T18:43:16.263287+0000","last_became_peered":"2024-04-25T18:43:16.263287+0000","last_unstale":"2024-04-25T18:47:24.641610+0000","last_undegraded":"2024-04-25T18:47:24.641610+0000","last_fullsized":"2024-04-25T18:47:24.641610+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:46.366457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804594+0000","last_change":"2024-04-25T18:43:12.203705+0000","last_active":"2024-04-25T18:47:24.804594+0000","last_peered":"2024-04-25T18:47:24.804594+0000","last_clean":"2024-04-25T18:47:24.804594+0000","last_became_active":"2024-04-25T18:43:12.203371+0000","last_became_peered":"2024-04-25T18:43:12.203371+0000","last_unstale":"2024-04-25T18:47:24.804594+0000","last_undegraded":"2024-04-25T18:47:24.804594+0000","last_fullsized":"2024-04-25T18:47:24.804594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641542+0000","last_change":"2024-04-25T18:43:56.647597+0000","last_active":"2024-04-25T18:47:24.641542+0000","last_peered":"2024-04-25T18:47:24.641542+0000","last_clean":"2024-04-25T18:47:24.641542+0000","last_became_active":"2024-04-25T18:43:56.647121+0000","last_became_peered":"2024-04-25T18:43:56.647121+0000","last_unstale":"2024-04-25T18:47:24.641542+0000","last_undegraded":"2024-04-25T18:47:24.641542+0000","last_fullsized":"2024-04-25T18:47:24.641542+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:07.143806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644686+0000","last_change":"2024-04-25T18:43:16.247133+0000","last_active":"2024-04-25T18:47:24.644686+0000","last_peered":"2024-04-25T18:47:24.644686+0000","last_clean":"2024-04-25T18:47:24.644686+0000","last_became_active":"2024-04-25T18:43:16.246232+0000","last_became_peered":"2024-04-25T18:43:16.246232+0000","last_unstale":"2024-04-25T18:47:24.644686+0000","last_undegraded":"2024-04-25T18:47:24.644686+0000","last_fullsized":"2024-04-25T18:47:24.644686+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.288404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":258,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.684213+0000","last_change":"2024-04-25T18:43:12.209908+0000","last_active":"2024-04-25T18:47:24.684213+0000","last_peered":"2024-04-25T18:47:24.684213+0000","last_clean":"2024-04-25T18:47:24.684213+0000","last_became_active":"2024-04-25T18:43:12.209042+0000","last_became_peered":"2024-04-25T18:43:12.209042+0000","last_unstale":"2024-04-25T18:47:24.684213+0000","last_undegraded":"2024-04-25T18:47:24.684213+0000","last_fullsized":"2024-04-25T18:47:24.684213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:36.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":224,"num_read_kb":224,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.767262+0000","last_change":"2024-04-25T18:43:16.242364+0000","last_active":"2024-04-25T18:47:24.767262+0000","last_peered":"2024-04-25T18:47:24.767262+0000","last_clean":"2024-04-25T18:47:24.767262+0000","last_became_active":"2024-04-25T18:43:16.242019+0000","last_became_peered":"2024-04-25T18:43:16.242019+0000","last_unstale":"2024-04-25T18:47:24.767262+0000","last_undegraded":"2024-04-25T18:47:24.767262+0000","last_fullsized":"2024-04-25T18:47:24.767262+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:13.238129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631512+0000","last_change":"2024-04-25T18:43:12.206606+0000","last_active":"2024-04-25T18:47:24.631512+0000","last_peered":"2024-04-25T18:47:24.631512+0000","last_clean":"2024-04-25T18:47:24.631512+0000","last_became_active":"2024-04-25T18:43:12.206238+0000","last_became_peered":"2024-04-25T18:43:12.206238+0000","last_unstale":"2024-04-25T18:47:24.631512+0000","last_undegraded":"2024-04-25T18:47:24.631512+0000","last_fullsized":"2024-04-25T18:47:24.631512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:07.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804586+0000","last_change":"2024-04-25T18:41:40.527363+0000","last_active":"2024-04-25T18:47:24.804586+0000","last_peered":"2024-04-25T18:47:24.804586+0000","last_clean":"2024-04-25T18:47:24.804586+0000","last_became_active":"2024-04-25T18:41:40.526907+0000","last_became_peered":"2024-04-25T18:41:40.526907+0000","last_unstale":"2024-04-25T18:47:24.804586+0000","last_undegraded":"2024-04-25T18:47:24.804586+0000","last_fullsized":"2024-04-25T18:47:24.804586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_clean_scrub_stamp":"2024-04-25T18:37:45.963565+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:16.689693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.630889+0000","last_change":"2024-04-25T18:43:16.247552+0000","last_active":"2024-04-25T18:47:24.630889+0000","last_peered":"2024-04-25T18:47:24.630889+0000","last_clean":"2024-04-25T18:47:24.630889+0000","last_became_active":"2024-04-25T18:43:16.247178+0000","last_became_peered":"2024-04-25T18:43:16.247178+0000","last_unstale":"2024-04-25T18:47:24.630889+0000","last_undegraded":"2024-04-25T18:47:24.630889+0000","last_fullsized":"2024-04-25T18:47:24.630889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:50.072986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804356+0000","last_change":"2024-04-25T18:43:56.636197+0000","last_active":"2024-04-25T18:47:24.804356+0000","last_peered":"2024-04-25T18:47:24.804356+0000","last_clean":"2024-04-25T18:47:24.804356+0000","last_became_active":"2024-04-25T18:43:56.633431+0000","last_became_peered":"2024-04-25T18:43:56.633431+0000","last_unstale":"2024-04-25T18:47:24.804356+0000","last_undegraded":"2024-04-25T18:47:24.804356+0000","last_fullsized":"2024-04-25T18:47:24.804356+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:12.463489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629270+0000","last_change":"2024-04-25T18:43:12.202101+0000","last_active":"2024-04-25T18:47:24.629270+0000","last_peered":"2024-04-25T18:47:24.629270+0000","last_clean":"2024-04-25T18:47:24.629270+0000","last_became_active":"2024-04-25T18:43:12.201727+0000","last_became_peered":"2024-04-25T18:43:12.201727+0000","last_unstale":"2024-04-25T18:47:24.629270+0000","last_undegraded":"2024-04-25T18:47:24.629270+0000","last_fullsized":"2024-04-25T18:47:24.629270+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:33.132072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631038+0000","last_change":"2024-04-25T18:43:16.255433+0000","last_active":"2024-04-25T18:47:24.631038+0000","last_peered":"2024-04-25T18:47:24.631038+0000","last_clean":"2024-04-25T18:47:24.631038+0000","last_became_active":"2024-04-25T18:43:16.254866+0000","last_became_peered":"2024-04-25T18:43:16.254866+0000","last_unstale":"2024-04-25T18:47:24.631038+0000","last_undegraded":"2024-04-25T18:47:24.631038+0000","last_fullsized":"2024-04-25T18:47:24.631038+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:07.876973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"95'48","reported_seq":112,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.915547+0000","last_change":"2024-04-25T18:43:56.656998+0000","last_active":"2024-04-25T18:47:24.915547+0000","last_peered":"2024-04-25T18:47:24.915547+0000","last_clean":"2024-04-25T18:47:24.915547+0000","last_became_active":"2024-04-25T18:43:56.647456+0000","last_became_peered":"2024-04-25T18:43:56.647456+0000","last_unstale":"2024-04-25T18:47:24.915547+0000","last_undegraded":"2024-04-25T18:47:24.915547+0000","last_fullsized":"2024-04-25T18:47:24.915547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:00.719426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":87,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768548+0000","last_change":"2024-04-25T18:43:16.252826+0000","last_active":"2024-04-25T18:47:24.768548+0000","last_peered":"2024-04-25T18:47:24.768548+0000","last_clean":"2024-04-25T18:47:24.768548+0000","last_became_active":"2024-04-25T18:43:16.252435+0000","last_became_peered":"2024-04-25T18:43:16.252435+0000","last_unstale":"2024-04-25T18:47:24.768548+0000","last_undegraded":"2024-04-25T18:47:24.768548+0000","last_fullsized":"2024-04-25T18:47:24.768548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:13.401006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631883+0000","last_change":"2024-04-25T18:43:12.376121+0000","last_active":"2024-04-25T18:47:24.631883+0000","last_peered":"2024-04-25T18:47:24.631883+0000","last_clean":"2024-04-25T18:47:24.631883+0000","last_became_active":"2024-04-25T18:43:12.375757+0000","last_became_peered":"2024-04-25T18:43:12.375757+0000","last_unstale":"2024-04-25T18:47:24.631883+0000","last_undegraded":"2024-04-25T18:47:24.631883+0000","last_fullsized":"2024-04-25T18:47:24.631883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:15.610442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981855+0000","last_change":"2024-04-25T18:43:16.266504+0000","last_active":"2024-04-25T18:47:24.981855+0000","last_peered":"2024-04-25T18:47:24.981855+0000","last_clean":"2024-04-25T18:47:24.981855+0000","last_became_active":"2024-04-25T18:43:16.265048+0000","last_became_peered":"2024-04-25T18:43:16.265048+0000","last_unstale":"2024-04-25T18:47:24.981855+0000","last_undegraded":"2024-04-25T18:47:24.981855+0000","last_fullsized":"2024-04-25T18:47:24.981855+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:36.924872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973501+0000","last_change":"2024-04-25T18:43:12.207529+0000","last_active":"2024-04-25T18:47:24.973501+0000","last_peered":"2024-04-25T18:47:24.973501+0000","last_clean":"2024-04-25T18:47:24.973501+0000","last_became_active":"2024-04-25T18:43:12.207122+0000","last_became_peered":"2024-04-25T18:43:12.207122+0000","last_unstale":"2024-04-25T18:47:24.973501+0000","last_undegraded":"2024-04-25T18:47:24.973501+0000","last_fullsized":"2024-04-25T18:47:24.973501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:44.518497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628853+0000","last_change":"2024-04-25T18:43:16.249616+0000","last_active":"2024-04-25T18:47:24.628853+0000","last_peered":"2024-04-25T18:47:24.628853+0000","last_clean":"2024-04-25T18:47:24.628853+0000","last_became_active":"2024-04-25T18:43:16.249252+0000","last_became_peered":"2024-04-25T18:43:16.249252+0000","last_unstale":"2024-04-25T18:47:24.628853+0000","last_undegraded":"2024-04-25T18:47:24.628853+0000","last_fullsized":"2024-04-25T18:47:24.628853+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:34.721513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.766887+0000","last_change":"2024-04-25T18:43:12.209304+0000","last_active":"2024-04-25T18:47:24.766887+0000","last_peered":"2024-04-25T18:47:24.766887+0000","last_clean":"2024-04-25T18:47:24.766887+0000","last_became_active":"2024-04-25T18:43:12.208887+0000","last_became_peered":"2024-04-25T18:43:12.208887+0000","last_unstale":"2024-04-25T18:47:24.766887+0000","last_undegraded":"2024-04-25T18:47:24.766887+0000","last_fullsized":"2024-04-25T18:47:24.766887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:22.049809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689315+0000","last_change":"2024-04-25T18:43:16.262482+0000","last_active":"2024-04-25T18:47:24.689315+0000","last_peered":"2024-04-25T18:47:24.689315+0000","last_clean":"2024-04-25T18:47:24.689315+0000","last_became_active":"2024-04-25T18:43:16.262108+0000","last_became_peered":"2024-04-25T18:43:16.262108+0000","last_unstale":"2024-04-25T18:47:24.689315+0000","last_undegraded":"2024-04-25T18:47:24.689315+0000","last_fullsized":"2024-04-25T18:47:24.689315+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:14.777939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973787+0000","last_change":"2024-04-25T18:43:12.205742+0000","last_active":"2024-04-25T18:47:24.973787+0000","last_peered":"2024-04-25T18:47:24.973787+0000","last_clean":"2024-04-25T18:47:24.973787+0000","last_became_active":"2024-04-25T18:43:12.205314+0000","last_became_peered":"2024-04-25T18:43:12.205314+0000","last_unstale":"2024-04-25T18:47:24.973787+0000","last_undegraded":"2024-04-25T18:47:24.973787+0000","last_fullsized":"2024-04-25T18:47:24.973787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:49.641568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.767568+0000","last_change":"2024-04-25T18:43:16.266100+0000","last_active":"2024-04-25T18:47:24.767568+0000","last_peered":"2024-04-25T18:47:24.767568+0000","last_clean":"2024-04-25T18:47:24.767568+0000","last_became_active":"2024-04-25T18:43:16.263060+0000","last_became_peered":"2024-04-25T18:43:16.263060+0000","last_unstale":"2024-04-25T18:47:24.767568+0000","last_undegraded":"2024-04-25T18:47:24.767568+0000","last_fullsized":"2024-04-25T18:47:24.767568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:38.269244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628994+0000","last_change":"2024-04-25T18:43:12.196054+0000","last_active":"2024-04-25T18:47:24.628994+0000","last_peered":"2024-04-25T18:47:24.628994+0000","last_clean":"2024-04-25T18:47:24.628994+0000","last_became_active":"2024-04-25T18:43:12.195503+0000","last_became_peered":"2024-04-25T18:43:12.195503+0000","last_unstale":"2024-04-25T18:47:24.628994+0000","last_undegraded":"2024-04-25T18:47:24.628994+0000","last_fullsized":"2024-04-25T18:47:24.628994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:55.187176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804544+0000","last_change":"2024-04-25T18:43:16.242536+0000","last_active":"2024-04-25T18:47:24.804544+0000","last_peered":"2024-04-25T18:47:24.804544+0000","last_clean":"2024-04-25T18:47:24.804544+0000","last_became_active":"2024-04-25T18:43:16.241060+0000","last_became_peered":"2024-04-25T18:43:16.241060+0000","last_unstale":"2024-04-25T18:47:24.804544+0000","last_undegraded":"2024-04-25T18:47:24.804544+0000","last_fullsized":"2024-04-25T18:47:24.804544+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:38.306190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631518+0000","last_change":"2024-04-25T18:43:56.653377+0000","last_active":"2024-04-25T18:47:24.631518+0000","last_peered":"2024-04-25T18:47:24.631518+0000","last_clean":"2024-04-25T18:47:24.631518+0000","last_became_active":"2024-04-25T18:43:56.645516+0000","last_became_peered":"2024-04-25T18:43:56.645516+0000","last_unstale":"2024-04-25T18:47:24.631518+0000","last_undegraded":"2024-04-25T18:47:24.631518+0000","last_fullsized":"2024-04-25T18:47:24.631518+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:11.189494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804771+0000","last_change":"2024-04-25T18:43:16.247610+0000","last_active":"2024-04-25T18:47:24.804771+0000","last_peered":"2024-04-25T18:47:24.804771+0000","last_clean":"2024-04-25T18:47:24.804771+0000","last_became_active":"2024-04-25T18:43:16.247273+0000","last_became_peered":"2024-04-25T18:43:16.247273+0000","last_unstale":"2024-04-25T18:47:24.804771+0000","last_undegraded":"2024-04-25T18:47:24.804771+0000","last_fullsized":"2024-04-25T18:47:24.804771+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:26.151480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'39","reported_seq":76,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.907938+0000","last_change":"2024-04-25T18:43:12.202011+0000","last_active":"2024-04-25T18:47:24.907938+0000","last_peered":"2024-04-25T18:47:24.907938+0000","last_clean":"2024-04-25T18:47:24.907938+0000","last_became_active":"2024-04-25T18:43:12.201447+0000","last_became_peered":"2024-04-25T18:43:12.201447+0000","last_unstale":"2024-04-25T18:47:24.907938+0000","last_undegraded":"2024-04-25T18:47:24.907938+0000","last_fullsized":"2024-04-25T18:47:24.907938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:48.094757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804583+0000","last_change":"2024-04-25T18:43:16.249033+0000","last_active":"2024-04-25T18:47:24.804583+0000","last_peered":"2024-04-25T18:47:24.804583+0000","last_clean":"2024-04-25T18:47:24.804583+0000","last_became_active":"2024-04-25T18:43:16.248691+0000","last_became_peered":"2024-04-25T18:43:16.248691+0000","last_unstale":"2024-04-25T18:47:24.804583+0000","last_undegraded":"2024-04-25T18:47:24.804583+0000","last_fullsized":"2024-04-25T18:47:24.804583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:37.335824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631177+0000","last_change":"2024-04-25T18:43:12.205134+0000","last_active":"2024-04-25T18:47:24.631177+0000","last_peered":"2024-04-25T18:47:24.631177+0000","last_clean":"2024-04-25T18:47:24.631177+0000","last_became_active":"2024-04-25T18:43:12.204568+0000","last_became_peered":"2024-04-25T18:43:12.204568+0000","last_unstale":"2024-04-25T18:47:24.631177+0000","last_undegraded":"2024-04-25T18:47:24.631177+0000","last_fullsized":"2024-04-25T18:47:24.631177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:32.806580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804824+0000","last_change":"2024-04-25T18:43:16.247337+0000","last_active":"2024-04-25T18:47:24.804824+0000","last_peered":"2024-04-25T18:47:24.804824+0000","last_clean":"2024-04-25T18:47:24.804824+0000","last_became_active":"2024-04-25T18:43:16.246971+0000","last_became_peered":"2024-04-25T18:43:16.246971+0000","last_unstale":"2024-04-25T18:47:24.804824+0000","last_undegraded":"2024-04-25T18:47:24.804824+0000","last_fullsized":"2024-04-25T18:47:24.804824+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.882941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689499+0000","last_change":"2024-04-25T18:43:16.252196+0000","last_active":"2024-04-25T18:47:24.689499+0000","last_peered":"2024-04-25T18:47:24.689499+0000","last_clean":"2024-04-25T18:47:24.689499+0000","last_became_active":"2024-04-25T18:43:16.251715+0000","last_became_peered":"2024-04-25T18:43:16.251715+0000","last_unstale":"2024-04-25T18:47:24.689499+0000","last_undegraded":"2024-04-25T18:47:24.689499+0000","last_fullsized":"2024-04-25T18:47:24.689499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:17.218925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973591+0000","last_change":"2024-04-25T18:43:16.247632+0000","last_active":"2024-04-25T18:47:24.973591+0000","last_peered":"2024-04-25T18:47:24.973591+0000","last_clean":"2024-04-25T18:47:24.973591+0000","last_became_active":"2024-04-25T18:43:16.247305+0000","last_became_peered":"2024-04-25T18:43:16.247305+0000","last_unstale":"2024-04-25T18:47:24.973591+0000","last_undegraded":"2024-04-25T18:47:24.973591+0000","last_fullsized":"2024-04-25T18:47:24.973591+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:52.576146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973560+0000","last_change":"2024-04-25T18:43:16.266539+0000","last_active":"2024-04-25T18:47:24.973560+0000","last_peered":"2024-04-25T18:47:24.973560+0000","last_clean":"2024-04-25T18:47:24.973560+0000","last_became_active":"2024-04-25T18:43:16.266083+0000","last_became_peered":"2024-04-25T18:43:16.266083+0000","last_unstale":"2024-04-25T18:47:24.973560+0000","last_undegraded":"2024-04-25T18:47:24.973560+0000","last_fullsized":"2024-04-25T18:47:24.973560+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:49.589019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981371+0000","last_change":"2024-04-25T18:43:16.264002+0000","last_active":"2024-04-25T18:47:24.981371+0000","last_peered":"2024-04-25T18:47:24.981371+0000","last_clean":"2024-04-25T18:47:24.981371+0000","last_became_active":"2024-04-25T18:43:16.261555+0000","last_became_peered":"2024-04-25T18:43:16.261555+0000","last_unstale":"2024-04-25T18:47:24.981371+0000","last_undegraded":"2024-04-25T18:47:24.981371+0000","last_fullsized":"2024-04-25T18:47:24.981371+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:12.637040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.630913+0000","last_change":"2024-04-25T18:43:16.270914+0000","last_active":"2024-04-25T18:47:24.630913+0000","last_peered":"2024-04-25T18:47:24.630913+0000","last_clean":"2024-04-25T18:47:24.630913+0000","last_became_active":"2024-04-25T18:43:16.270117+0000","last_became_peered":"2024-04-25T18:43:16.270117+0000","last_unstale":"2024-04-25T18:47:24.630913+0000","last_undegraded":"2024-04-25T18:47:24.630913+0000","last_fullsized":"2024-04-25T18:47:24.630913+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:19.874006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689429+0000","last_change":"2024-04-25T18:43:16.247032+0000","last_active":"2024-04-25T18:47:24.689429+0000","last_peered":"2024-04-25T18:47:24.689429+0000","last_clean":"2024-04-25T18:47:24.689429+0000","last_became_active":"2024-04-25T18:43:16.246734+0000","last_became_peered":"2024-04-25T18:43:16.246734+0000","last_unstale":"2024-04-25T18:47:24.689429+0000","last_undegraded":"2024-04-25T18:47:24.689429+0000","last_fullsized":"2024-04-25T18:47:24.689429+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:39.159143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973860+0000","last_change":"2024-04-25T18:43:16.245263+0000","last_active":"2024-04-25T18:47:24.973860+0000","last_peered":"2024-04-25T18:47:24.973860+0000","last_clean":"2024-04-25T18:47:24.973860+0000","last_became_active":"2024-04-25T18:43:16.244856+0000","last_became_peered":"2024-04-25T18:43:16.244856+0000","last_unstale":"2024-04-25T18:47:24.973860+0000","last_undegraded":"2024-04-25T18:47:24.973860+0000","last_fullsized":"2024-04-25T18:47:24.973860+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:36.157927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768461+0000","last_change":"2024-04-25T18:43:16.253523+0000","last_active":"2024-04-25T18:47:24.768461+0000","last_peered":"2024-04-25T18:47:24.768461+0000","last_clean":"2024-04-25T18:47:24.768461+0000","last_became_active":"2024-04-25T18:43:16.252689+0000","last_became_peered":"2024-04-25T18:43:16.252689+0000","last_unstale":"2024-04-25T18:47:24.768461+0000","last_undegraded":"2024-04-25T18:47:24.768461+0000","last_fullsized":"2024-04-25T18:47:24.768461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.035788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644415+0000","last_change":"2024-04-25T18:43:16.255818+0000","last_active":"2024-04-25T18:47:24.644415+0000","last_peered":"2024-04-25T18:47:24.644415+0000","last_clean":"2024-04-25T18:47:24.644415+0000","last_became_active":"2024-04-25T18:43:16.255521+0000","last_became_peered":"2024-04-25T18:43:16.255521+0000","last_unstale":"2024-04-25T18:47:24.644415+0000","last_undegraded":"2024-04-25T18:47:24.644415+0000","last_fullsized":"2024-04-25T18:47:24.644415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:15.457585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.766729+0000","last_change":"2024-04-25T18:43:16.259680+0000","last_active":"2024-04-25T18:47:24.766729+0000","last_peered":"2024-04-25T18:47:24.766729+0000","last_clean":"2024-04-25T18:47:24.766729+0000","last_became_active":"2024-04-25T18:43:16.250065+0000","last_became_peered":"2024-04-25T18:43:16.250065+0000","last_unstale":"2024-04-25T18:47:24.766729+0000","last_undegraded":"2024-04-25T18:47:24.766729+0000","last_fullsized":"2024-04-25T18:47:24.766729+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:44.974919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973908+0000","last_change":"2024-04-25T18:43:16.239801+0000","last_active":"2024-04-25T18:47:24.973908+0000","last_peered":"2024-04-25T18:47:24.973908+0000","last_clean":"2024-04-25T18:47:24.973908+0000","last_became_active":"2024-04-25T18:43:16.239475+0000","last_became_peered":"2024-04-25T18:43:16.239475+0000","last_unstale":"2024-04-25T18:47:24.973908+0000","last_undegraded":"2024-04-25T18:47:24.973908+0000","last_fullsized":"2024-04-25T18:47:24.973908+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:57.407476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.630762+0000","last_change":"2024-04-25T18:43:16.270419+0000","last_active":"2024-04-25T18:47:24.630762+0000","last_peered":"2024-04-25T18:47:24.630762+0000","last_clean":"2024-04-25T18:47:24.630762+0000","last_became_active":"2024-04-25T18:43:16.269995+0000","last_became_peered":"2024-04-25T18:43:16.269995+0000","last_unstale":"2024-04-25T18:47:24.630762+0000","last_undegraded":"2024-04-25T18:47:24.630762+0000","last_fullsized":"2024-04-25T18:47:24.630762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:27.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689424+0000","last_change":"2024-04-25T18:43:16.246948+0000","last_active":"2024-04-25T18:47:24.689424+0000","last_peered":"2024-04-25T18:47:24.689424+0000","last_clean":"2024-04-25T18:47:24.689424+0000","last_became_active":"2024-04-25T18:43:16.246616+0000","last_became_peered":"2024-04-25T18:43:16.246616+0000","last_unstale":"2024-04-25T18:47:24.689424+0000","last_undegraded":"2024-04-25T18:47:24.689424+0000","last_fullsized":"2024-04-25T18:47:24.689424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:51.817448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600787,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1469,"num_read_kb":841,"num_write":294,"num_write_kb":1442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":582,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2039808,"data_stored":1802361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252931,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":498757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514568,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37240,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707912,"statfs":{"total":95995035648,"available":95956901888,"internally_reserved":0,"allocated":1888256,"data_stored":1302261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710796,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1454080,"data_stored":901194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":507898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234073,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1090984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430302,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":507367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":563458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":508638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1142784,"data_stored":580289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117104,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1593344,"data_stored":1032494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":508386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691940,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28776,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716376,"statfs":{"total":95995035648,"available":95965569024,"internally_reserved":0,"allocated":1802240,"data_stored":1231644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":204616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:47:32.567 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:47:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:32 smithi171 ceph-mon[33266]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:32 smithi171 ceph-mon[33266]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:32 smithi171 ceph-mon[33266]: Updating smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:47:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:32 smithi171 ceph-mon[33266]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:47:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:32 smithi171 ceph-mon[33266]: Updating smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:47:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:32 smithi171 ceph-mon[33266]: Updating smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:47:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:32 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:32 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:32 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:32 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:32 smithi171 ceph-mon[33266]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:33.026 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:33.433 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:33 smithi112 ceph-mon[27864]: from='client.25145 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:47:33.433 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:33 smithi112 ceph-mon[27864]: Updating smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:47:33.433 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:33 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:33.433 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:33 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:33.434 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:33 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:33.434 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:33 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:33.434 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:33 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:47:33.434 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:33 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:47:33.434 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:33 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:33 smithi161 ceph-mon[32609]: from='client.25145 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:47:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:33 smithi161 ceph-mon[32609]: Updating smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:47:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:33 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:33 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:33 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:33 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:33 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:47:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:33 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:47:33.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:33 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:33.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:33 smithi171 ceph-mon[33266]: from='client.25145 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:47:33.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:33 smithi171 ceph-mon[33266]: Updating smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:47:33.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:33 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:33.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:33 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:33.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:33 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:33.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:33 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:33.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:33 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:47:33.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:33 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:47:33.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:33 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:47:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:34 smithi161 ceph-mon[32609]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-25T18:47:34.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:34 smithi112 ceph-mon[27864]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-25T18:47:34.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:34 smithi171 ceph-mon[33266]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-25T18:47:35.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.458+0000 7fea8ec00700 1 -- 172.21.15.112:0/1324477895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea88105830 msgr2=0x7fea88109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:35.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.458+0000 7fea8ec00700 1 --2- 172.21.15.112:0/1324477895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea88105830 0x7fea88109880 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fea740098f0 tx=0x7fea74009c00 comp rx=0 tx=0).stop 2024-04-25T18:47:35.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.458+0000 7fea8ec00700 1 -- 172.21.15.112:0/1324477895 shutdown_connections 2024-04-25T18:47:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.458+0000 7fea8ec00700 1 --2- 172.21.15.112:0/1324477895 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea88109f60 0x7fea88111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.458+0000 7fea8ec00700 1 --2- 172.21.15.112:0/1324477895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88104e80 0x7fea88105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.458+0000 7fea8ec00700 1 --2- 172.21.15.112:0/1324477895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea88105830 0x7fea88109880 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.458+0000 7fea8ec00700 1 -- 172.21.15.112:0/1324477895 >> 172.21.15.112:0/1324477895 conn(0x7fea881006f0 msgr2=0x7fea88102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.458+0000 7fea8ec00700 1 -- 172.21.15.112:0/1324477895 shutdown_connections 2024-04-25T18:47:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.458+0000 7fea8ec00700 1 -- 172.21.15.112:0/1324477895 wait complete. 2024-04-25T18:47:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.459+0000 7fea8ec00700 1 Processor -- start 2024-04-25T18:47:35.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea8ec00700 1 -- start start 2024-04-25T18:47:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea8ec00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea88104e80 0x7fea8819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea8ec00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88105830 0x7fea8819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea8ec00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea88109f60 0x7fea8819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea8ec00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea88114e20 con 0x7fea88105830 2024-04-25T18:47:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea8ec00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fea88114ca0 con 0x7fea88109f60 2024-04-25T18:47:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea8ec00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fea88114fa0 con 0x7fea88104e80 2024-04-25T18:47:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea87400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88105830 0x7fea8819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea87400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88105830 0x7fea8819b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34662/0 (socket says 172.21.15.112:34662) 2024-04-25T18:47:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea87400700 1 -- 172.21.15.112:0/1447175954 learned_addr learned my addr 172.21.15.112:0/1447175954 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:35.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea8cc00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea88109f60 0x7fea8819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.460+0000 7fea87e00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea88104e80 0x7fea8819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.461+0000 7fea87400700 1 -- 172.21.15.112:0/1447175954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea88104e80 msgr2=0x7fea8819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.461+0000 7fea87400700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea88104e80 0x7fea8819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.461+0000 7fea87400700 1 -- 172.21.15.112:0/1447175954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea88109f60 msgr2=0x7fea8819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.461+0000 7fea87400700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea88109f60 0x7fea8819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.461+0000 7fea87400700 1 -- 172.21.15.112:0/1447175954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea881a03d0 con 0x7fea88105830 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.461+0000 7fea8cc00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea88109f60 0x7fea8819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.461+0000 7fea87400700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88105830 0x7fea8819b950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea7400c010 tx=0x7fea7400f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.462+0000 7fea84c00700 1 -- 172.21.15.112:0/1447175954 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74027070 con 0x7fea88105830 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.462+0000 7fea8ec00700 1 -- 172.21.15.112:0/1447175954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea881a0660 con 0x7fea88105830 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.462+0000 7fea84c00700 1 -- 172.21.15.112:0/1447175954 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea740054f0 con 0x7fea88105830 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.462+0000 7fea84c00700 1 -- 172.21.15.112:0/1447175954 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74004930 con 0x7fea88105830 2024-04-25T18:47:35.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.462+0000 7fea8ec00700 1 -- 172.21.15.112:0/1447175954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881a7f40 con 0x7fea88105830 2024-04-25T18:47:35.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.463+0000 7fea8ec00700 1 -- 172.21.15.112:0/1447175954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea88069730 con 0x7fea88105830 2024-04-25T18:47:35.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.465+0000 7fea84c00700 1 -- 172.21.15.112:0/1447175954 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fea7402a070 con 0x7fea88105830 2024-04-25T18:47:35.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.466+0000 7fea84c00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fea6c0a3ed0 0x7fea6c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:35.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.466+0000 7fea84c00700 1 -- 172.21.15.112:0/1447175954 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fea740d17e0 con 0x7fea88105830 2024-04-25T18:47:35.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.467+0000 7fea87e00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fea6c0a3ed0 0x7fea6c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:35.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.467+0000 7fea87e00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fea6c0a3ed0 0x7fea6c0a6390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fea78009c60 tx=0x7fea78009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:35.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.470+0000 7fea84c00700 1 -- 172.21.15.112:0/1447175954 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea74031020 con 0x7fea88105830 2024-04-25T18:47:35.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.693+0000 7fea8ec00700 1 -- 172.21.15.112:0/1447175954 --> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea8819c720 con 0x7fea6c0a3ed0 2024-04-25T18:47:35.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.715+0000 7fea84c00700 1 -- 172.21.15.112:0/1447175954 <== mgr.14922 v2:172.21.15.112:6800/2236031982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257773 (secure 0 0 0) 0x7fea8819c720 con 0x7fea6c0a3ed0 2024-04-25T18:47:35.722 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 -- 172.21.15.112:0/1447175954 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fea6c0a3ed0 msgr2=0x7fea6c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fea6c0a3ed0 0x7fea6c0a6390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fea78009c60 tx=0x7fea78009260 comp rx=0 tx=0).stop 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 -- 172.21.15.112:0/1447175954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88105830 msgr2=0x7fea8819b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88105830 0x7fea8819b950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea7400c010 tx=0x7fea7400f9b0 comp rx=0 tx=0).stop 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 -- 172.21.15.112:0/1447175954 shutdown_connections 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fea6c0a3ed0 0x7fea6c0a6390 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea88109f60 0x7fea8819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea88104e80 0x7fea8819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 --2- 172.21.15.112:0/1447175954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea88105830 0x7fea8819b950 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 -- 172.21.15.112:0/1447175954 >> 172.21.15.112:0/1447175954 conn(0x7fea881006f0 msgr2=0x7fea88101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 -- 172.21.15.112:0/1447175954 shutdown_connections 2024-04-25T18:47:35.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:35.721+0000 7fea8ec00700 1 -- 172.21.15.112:0/1447175954 wait complete. 2024-04-25T18:47:35.725 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T18:47:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:36 smithi161 ceph-mon[32609]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-25T18:47:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:36 smithi112 ceph-mon[27864]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-25T18:47:36.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:36 smithi171 ceph-mon[33266]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-25T18:47:36.773 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-25T18:47:34.652457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391323,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1992,"num_read_kb":1427,"num_write":631,"num_write_kb":5304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181318,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":964,"ondisk_log_size":964,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382568,"kb_used_data":16524,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559256,"statfs":{"total":1151940427776,"available":1151548678144,"internally_reserved":0,"allocated":16920576,"data_stored":10259889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199934,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":387,"num_read_kb":467,"num_write":154,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.964505"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689752+0000","last_change":"2024-04-25T18:43:16.262618+0000","last_active":"2024-04-25T18:47:24.689752+0000","last_peered":"2024-04-25T18:47:24.689752+0000","last_clean":"2024-04-25T18:47:24.689752+0000","last_became_active":"2024-04-25T18:43:16.262215+0000","last_became_peered":"2024-04-25T18:43:16.262215+0000","last_unstale":"2024-04-25T18:47:24.689752+0000","last_undegraded":"2024-04-25T18:47:24.689752+0000","last_fullsized":"2024-04-25T18:47:24.689752+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:40.319102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981786+0000","last_change":"2024-04-25T18:43:16.254085+0000","last_active":"2024-04-25T18:47:24.981786+0000","last_peered":"2024-04-25T18:47:24.981786+0000","last_clean":"2024-04-25T18:47:24.981786+0000","last_became_active":"2024-04-25T18:43:16.248943+0000","last_became_peered":"2024-04-25T18:43:16.248943+0000","last_unstale":"2024-04-25T18:47:24.981786+0000","last_undegraded":"2024-04-25T18:47:24.981786+0000","last_fullsized":"2024-04-25T18:47:24.981786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:53.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981717+0000","last_change":"2024-04-25T18:43:16.252684+0000","last_active":"2024-04-25T18:47:24.981717+0000","last_peered":"2024-04-25T18:47:24.981717+0000","last_clean":"2024-04-25T18:47:24.981717+0000","last_became_active":"2024-04-25T18:43:16.252318+0000","last_became_peered":"2024-04-25T18:43:16.252318+0000","last_unstale":"2024-04-25T18:47:24.981717+0000","last_undegraded":"2024-04-25T18:47:24.981717+0000","last_fullsized":"2024-04-25T18:47:24.981717+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:58.904865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631697+0000","last_change":"2024-04-25T18:43:16.247059+0000","last_active":"2024-04-25T18:47:24.631697+0000","last_peered":"2024-04-25T18:47:24.631697+0000","last_clean":"2024-04-25T18:47:24.631697+0000","last_became_active":"2024-04-25T18:43:16.246655+0000","last_became_peered":"2024-04-25T18:43:16.246655+0000","last_unstale":"2024-04-25T18:47:24.631697+0000","last_undegraded":"2024-04-25T18:47:24.631697+0000","last_fullsized":"2024-04-25T18:47:24.631697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:31.026579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628963+0000","last_change":"2024-04-25T18:43:16.250033+0000","last_active":"2024-04-25T18:47:24.628963+0000","last_peered":"2024-04-25T18:47:24.628963+0000","last_clean":"2024-04-25T18:47:24.628963+0000","last_became_active":"2024-04-25T18:43:16.249017+0000","last_became_peered":"2024-04-25T18:43:16.249017+0000","last_unstale":"2024-04-25T18:47:24.628963+0000","last_undegraded":"2024-04-25T18:47:24.628963+0000","last_fullsized":"2024-04-25T18:47:24.628963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:34.304263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641422+0000","last_change":"2024-04-25T18:43:56.642410+0000","last_active":"2024-04-25T18:47:24.641422+0000","last_peered":"2024-04-25T18:47:24.641422+0000","last_clean":"2024-04-25T18:47:24.641422+0000","last_became_active":"2024-04-25T18:43:56.642009+0000","last_became_peered":"2024-04-25T18:43:56.642009+0000","last_unstale":"2024-04-25T18:47:24.641422+0000","last_undegraded":"2024-04-25T18:47:24.641422+0000","last_fullsized":"2024-04-25T18:47:24.641422+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:19.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804293+0000","last_change":"2024-04-25T18:43:56.647158+0000","last_active":"2024-04-25T18:47:24.804293+0000","last_peered":"2024-04-25T18:47:24.804293+0000","last_clean":"2024-04-25T18:47:24.804293+0000","last_became_active":"2024-04-25T18:43:56.646779+0000","last_became_peered":"2024-04-25T18:43:56.646779+0000","last_unstale":"2024-04-25T18:47:24.804293+0000","last_undegraded":"2024-04-25T18:47:24.804293+0000","last_fullsized":"2024-04-25T18:47:24.804293+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:04:14.055762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.766601+0000","last_change":"2024-04-25T18:43:16.247607+0000","last_active":"2024-04-25T18:47:24.766601+0000","last_peered":"2024-04-25T18:47:24.766601+0000","last_clean":"2024-04-25T18:47:24.766601+0000","last_became_active":"2024-04-25T18:43:16.247234+0000","last_became_peered":"2024-04-25T18:43:16.247234+0000","last_unstale":"2024-04-25T18:47:24.766601+0000","last_undegraded":"2024-04-25T18:47:24.766601+0000","last_fullsized":"2024-04-25T18:47:24.766601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:57.117900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973993+0000","last_change":"2024-04-25T18:43:16.247909+0000","last_active":"2024-04-25T18:47:24.973993+0000","last_peered":"2024-04-25T18:47:24.973993+0000","last_clean":"2024-04-25T18:47:24.973993+0000","last_became_active":"2024-04-25T18:43:16.247533+0000","last_became_peered":"2024-04-25T18:43:16.247533+0000","last_unstale":"2024-04-25T18:47:24.973993+0000","last_undegraded":"2024-04-25T18:47:24.973993+0000","last_fullsized":"2024-04-25T18:47:24.973993+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:02.257832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641756+0000","last_change":"2024-04-25T18:43:16.252088+0000","last_active":"2024-04-25T18:47:24.641756+0000","last_peered":"2024-04-25T18:47:24.641756+0000","last_clean":"2024-04-25T18:47:24.641756+0000","last_became_active":"2024-04-25T18:43:16.251580+0000","last_became_peered":"2024-04-25T18:43:16.251580+0000","last_unstale":"2024-04-25T18:47:24.641756+0000","last_undegraded":"2024-04-25T18:47:24.641756+0000","last_fullsized":"2024-04-25T18:47:24.641756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:22.992087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804685+0000","last_change":"2024-04-25T18:43:16.239696+0000","last_active":"2024-04-25T18:47:24.804685+0000","last_peered":"2024-04-25T18:47:24.804685+0000","last_clean":"2024-04-25T18:47:24.804685+0000","last_became_active":"2024-04-25T18:43:16.239354+0000","last_became_peered":"2024-04-25T18:43:16.239354+0000","last_unstale":"2024-04-25T18:47:24.804685+0000","last_undegraded":"2024-04-25T18:47:24.804685+0000","last_fullsized":"2024-04-25T18:47:24.804685+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:39.682138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631515+0000","last_change":"2024-04-25T18:43:16.242487+0000","last_active":"2024-04-25T18:47:24.631515+0000","last_peered":"2024-04-25T18:47:24.631515+0000","last_clean":"2024-04-25T18:47:24.631515+0000","last_became_active":"2024-04-25T18:43:16.242158+0000","last_became_peered":"2024-04-25T18:43:16.242158+0000","last_unstale":"2024-04-25T18:47:24.631515+0000","last_undegraded":"2024-04-25T18:47:24.631515+0000","last_fullsized":"2024-04-25T18:47:24.631515+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:36.477440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.630730+0000","last_change":"2024-04-25T18:43:16.244154+0000","last_active":"2024-04-25T18:47:24.630730+0000","last_peered":"2024-04-25T18:47:24.630730+0000","last_clean":"2024-04-25T18:47:24.630730+0000","last_became_active":"2024-04-25T18:43:16.243829+0000","last_became_peered":"2024-04-25T18:43:16.243829+0000","last_unstale":"2024-04-25T18:47:24.630730+0000","last_undegraded":"2024-04-25T18:47:24.630730+0000","last_fullsized":"2024-04-25T18:47:24.630730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:43.262751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981216+0000","last_change":"2024-04-25T18:43:16.261227+0000","last_active":"2024-04-25T18:47:24.981216+0000","last_peered":"2024-04-25T18:47:24.981216+0000","last_clean":"2024-04-25T18:47:24.981216+0000","last_became_active":"2024-04-25T18:43:16.250468+0000","last_became_peered":"2024-04-25T18:43:16.250468+0000","last_unstale":"2024-04-25T18:47:24.981216+0000","last_undegraded":"2024-04-25T18:47:24.981216+0000","last_fullsized":"2024-04-25T18:47:24.981216+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:44.805643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981094+0000","last_change":"2024-04-25T18:43:16.257956+0000","last_active":"2024-04-25T18:47:24.981094+0000","last_peered":"2024-04-25T18:47:24.981094+0000","last_clean":"2024-04-25T18:47:24.981094+0000","last_became_active":"2024-04-25T18:43:16.255126+0000","last_became_peered":"2024-04-25T18:43:16.255126+0000","last_unstale":"2024-04-25T18:47:24.981094+0000","last_undegraded":"2024-04-25T18:47:24.981094+0000","last_fullsized":"2024-04-25T18:47:24.981094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:30.047165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973501+0000","last_change":"2024-04-25T18:43:16.256208+0000","last_active":"2024-04-25T18:47:24.973501+0000","last_peered":"2024-04-25T18:47:24.973501+0000","last_clean":"2024-04-25T18:47:24.973501+0000","last_became_active":"2024-04-25T18:43:16.255614+0000","last_became_peered":"2024-04-25T18:43:16.255614+0000","last_unstale":"2024-04-25T18:47:24.973501+0000","last_undegraded":"2024-04-25T18:47:24.973501+0000","last_fullsized":"2024-04-25T18:47:24.973501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:46.700209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689815+0000","last_change":"2024-04-25T18:43:16.262730+0000","last_active":"2024-04-25T18:47:24.689815+0000","last_peered":"2024-04-25T18:47:24.689815+0000","last_clean":"2024-04-25T18:47:24.689815+0000","last_became_active":"2024-04-25T18:43:16.262192+0000","last_became_peered":"2024-04-25T18:43:16.262192+0000","last_unstale":"2024-04-25T18:47:24.689815+0000","last_undegraded":"2024-04-25T18:47:24.689815+0000","last_fullsized":"2024-04-25T18:47:24.689815+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:29.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.763135+0000","last_change":"2024-04-25T18:43:16.261161+0000","last_active":"2024-04-25T18:47:24.763135+0000","last_peered":"2024-04-25T18:47:24.763135+0000","last_clean":"2024-04-25T18:47:24.763135+0000","last_became_active":"2024-04-25T18:43:16.258217+0000","last_became_peered":"2024-04-25T18:43:16.258217+0000","last_unstale":"2024-04-25T18:47:24.763135+0000","last_undegraded":"2024-04-25T18:47:24.763135+0000","last_fullsized":"2024-04-25T18:47:24.763135+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.564609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973806+0000","last_change":"2024-04-25T18:43:16.266660+0000","last_active":"2024-04-25T18:47:24.973806+0000","last_peered":"2024-04-25T18:47:24.973806+0000","last_clean":"2024-04-25T18:47:24.973806+0000","last_became_active":"2024-04-25T18:43:16.266295+0000","last_became_peered":"2024-04-25T18:43:16.266295+0000","last_unstale":"2024-04-25T18:47:24.973806+0000","last_undegraded":"2024-04-25T18:47:24.973806+0000","last_fullsized":"2024-04-25T18:47:24.973806+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:42.406585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.763304+0000","last_change":"2024-04-25T18:43:12.200172+0000","last_active":"2024-04-25T18:47:24.763304+0000","last_peered":"2024-04-25T18:47:24.763304+0000","last_clean":"2024-04-25T18:47:24.763304+0000","last_became_active":"2024-04-25T18:43:12.199760+0000","last_became_peered":"2024-04-25T18:43:12.199760+0000","last_unstale":"2024-04-25T18:47:24.763304+0000","last_undegraded":"2024-04-25T18:47:24.763304+0000","last_fullsized":"2024-04-25T18:47:24.763304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:57.460274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768028+0000","last_change":"2024-04-25T18:43:16.255989+0000","last_active":"2024-04-25T18:47:24.768028+0000","last_peered":"2024-04-25T18:47:24.768028+0000","last_clean":"2024-04-25T18:47:24.768028+0000","last_became_active":"2024-04-25T18:43:16.255114+0000","last_became_peered":"2024-04-25T18:43:16.255114+0000","last_unstale":"2024-04-25T18:47:24.768028+0000","last_undegraded":"2024-04-25T18:47:24.768028+0000","last_fullsized":"2024-04-25T18:47:24.768028+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:52.888876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768305+0000","last_change":"2024-04-25T18:43:12.208194+0000","last_active":"2024-04-25T18:47:24.768305+0000","last_peered":"2024-04-25T18:47:24.768305+0000","last_clean":"2024-04-25T18:47:24.768305+0000","last_became_active":"2024-04-25T18:43:12.207801+0000","last_became_peered":"2024-04-25T18:43:12.207801+0000","last_unstale":"2024-04-25T18:47:24.768305+0000","last_undegraded":"2024-04-25T18:47:24.768305+0000","last_fullsized":"2024-04-25T18:47:24.768305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:24.742486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981924+0000","last_change":"2024-04-25T18:43:16.268960+0000","last_active":"2024-04-25T18:47:24.981924+0000","last_peered":"2024-04-25T18:47:24.981924+0000","last_clean":"2024-04-25T18:47:24.981924+0000","last_became_active":"2024-04-25T18:43:16.263626+0000","last_became_peered":"2024-04-25T18:43:16.263626+0000","last_unstale":"2024-04-25T18:47:24.981924+0000","last_undegraded":"2024-04-25T18:47:24.981924+0000","last_fullsized":"2024-04-25T18:47:24.981924+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:57.721918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629378+0000","last_change":"2024-04-25T18:43:57.110208+0000","last_active":"2024-04-25T18:47:24.629378+0000","last_peered":"2024-04-25T18:47:24.629378+0000","last_clean":"2024-04-25T18:47:24.629378+0000","last_became_active":"2024-04-25T18:43:57.106027+0000","last_became_peered":"2024-04-25T18:43:57.106027+0000","last_unstale":"2024-04-25T18:47:24.629378+0000","last_undegraded":"2024-04-25T18:47:24.629378+0000","last_fullsized":"2024-04-25T18:47:24.629378+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:20.819961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973596+0000","last_change":"2024-04-25T18:43:16.250414+0000","last_active":"2024-04-25T18:47:24.973596+0000","last_peered":"2024-04-25T18:47:24.973596+0000","last_clean":"2024-04-25T18:47:24.973596+0000","last_became_active":"2024-04-25T18:43:16.249832+0000","last_became_peered":"2024-04-25T18:43:16.249832+0000","last_unstale":"2024-04-25T18:47:24.973596+0000","last_undegraded":"2024-04-25T18:47:24.973596+0000","last_fullsized":"2024-04-25T18:47:24.973596+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.577854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629313+0000","last_change":"2024-04-25T18:43:12.195099+0000","last_active":"2024-04-25T18:47:24.629313+0000","last_peered":"2024-04-25T18:47:24.629313+0000","last_clean":"2024-04-25T18:47:24.629313+0000","last_became_active":"2024-04-25T18:43:12.194726+0000","last_became_peered":"2024-04-25T18:43:12.194726+0000","last_unstale":"2024-04-25T18:47:24.629313+0000","last_undegraded":"2024-04-25T18:47:24.629313+0000","last_fullsized":"2024-04-25T18:47:24.629313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.866905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644755+0000","last_change":"2024-04-25T18:43:16.245208+0000","last_active":"2024-04-25T18:47:24.644755+0000","last_peered":"2024-04-25T18:47:24.644755+0000","last_clean":"2024-04-25T18:47:24.644755+0000","last_became_active":"2024-04-25T18:43:16.244836+0000","last_became_peered":"2024-04-25T18:43:16.244836+0000","last_unstale":"2024-04-25T18:47:24.644755+0000","last_undegraded":"2024-04-25T18:47:24.644755+0000","last_fullsized":"2024-04-25T18:47:24.644755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:35.917060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":272,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:29.652930+0000","last_change":"2024-04-25T18:43:12.198252+0000","last_active":"2024-04-25T18:47:29.652930+0000","last_peered":"2024-04-25T18:47:29.652930+0000","last_clean":"2024-04-25T18:47:29.652930+0000","last_became_active":"2024-04-25T18:43:12.197691+0000","last_became_peered":"2024-04-25T18:43:12.197691+0000","last_unstale":"2024-04-25T18:47:29.652930+0000","last_undegraded":"2024-04-25T18:47:29.652930+0000","last_fullsized":"2024-04-25T18:47:29.652930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.900012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":235,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628891+0000","last_change":"2024-04-25T18:43:16.255576+0000","last_active":"2024-04-25T18:47:24.628891+0000","last_peered":"2024-04-25T18:47:24.628891+0000","last_clean":"2024-04-25T18:47:24.628891+0000","last_became_active":"2024-04-25T18:43:16.255195+0000","last_became_peered":"2024-04-25T18:43:16.255195+0000","last_unstale":"2024-04-25T18:47:24.628891+0000","last_undegraded":"2024-04-25T18:47:24.628891+0000","last_fullsized":"2024-04-25T18:47:24.628891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:41.510872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641395+0000","last_change":"2024-04-25T18:43:12.200786+0000","last_active":"2024-04-25T18:47:24.641395+0000","last_peered":"2024-04-25T18:47:24.641395+0000","last_clean":"2024-04-25T18:47:24.641395+0000","last_became_active":"2024-04-25T18:43:12.200409+0000","last_became_peered":"2024-04-25T18:43:12.200409+0000","last_unstale":"2024-04-25T18:47:24.641395+0000","last_undegraded":"2024-04-25T18:47:24.641395+0000","last_fullsized":"2024-04-25T18:47:24.641395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:52.177787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644567+0000","last_change":"2024-04-25T18:43:16.255563+0000","last_active":"2024-04-25T18:47:24.644567+0000","last_peered":"2024-04-25T18:47:24.644567+0000","last_clean":"2024-04-25T18:47:24.644567+0000","last_became_active":"2024-04-25T18:43:16.255174+0000","last_became_peered":"2024-04-25T18:43:16.255174+0000","last_unstale":"2024-04-25T18:47:24.644567+0000","last_undegraded":"2024-04-25T18:47:24.644567+0000","last_fullsized":"2024-04-25T18:47:24.644567+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:32:15.735051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631536+0000","last_change":"2024-04-25T18:43:12.210242+0000","last_active":"2024-04-25T18:47:24.631536+0000","last_peered":"2024-04-25T18:47:24.631536+0000","last_clean":"2024-04-25T18:47:24.631536+0000","last_became_active":"2024-04-25T18:43:12.209600+0000","last_became_peered":"2024-04-25T18:43:12.209600+0000","last_unstale":"2024-04-25T18:47:24.631536+0000","last_undegraded":"2024-04-25T18:47:24.631536+0000","last_fullsized":"2024-04-25T18:47:24.631536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.886712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.762109+0000","last_change":"2024-04-25T18:43:56.649102+0000","last_active":"2024-04-25T18:47:24.762109+0000","last_peered":"2024-04-25T18:47:24.762109+0000","last_clean":"2024-04-25T18:47:24.762109+0000","last_became_active":"2024-04-25T18:43:56.647127+0000","last_became_peered":"2024-04-25T18:43:56.647127+0000","last_unstale":"2024-04-25T18:47:24.762109+0000","last_undegraded":"2024-04-25T18:47:24.762109+0000","last_fullsized":"2024-04-25T18:47:24.762109+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:43.111539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644363+0000","last_change":"2024-04-25T18:43:12.198957+0000","last_active":"2024-04-25T18:47:24.644363+0000","last_peered":"2024-04-25T18:47:24.644363+0000","last_clean":"2024-04-25T18:47:24.644363+0000","last_became_active":"2024-04-25T18:43:12.198605+0000","last_became_peered":"2024-04-25T18:43:12.198605+0000","last_unstale":"2024-04-25T18:47:24.644363+0000","last_undegraded":"2024-04-25T18:47:24.644363+0000","last_fullsized":"2024-04-25T18:47:24.644363+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:11.759430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.762246+0000","last_change":"2024-04-25T18:43:16.266829+0000","last_active":"2024-04-25T18:47:24.762246+0000","last_peered":"2024-04-25T18:47:24.762246+0000","last_clean":"2024-04-25T18:47:24.762246+0000","last_became_active":"2024-04-25T18:43:16.266256+0000","last_became_peered":"2024-04-25T18:43:16.266256+0000","last_unstale":"2024-04-25T18:47:24.762246+0000","last_undegraded":"2024-04-25T18:47:24.762246+0000","last_fullsized":"2024-04-25T18:47:24.762246+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:43.343613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":264,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:29.652240+0000","last_change":"2024-04-25T18:43:12.201641+0000","last_active":"2024-04-25T18:47:29.652240+0000","last_peered":"2024-04-25T18:47:29.652240+0000","last_clean":"2024-04-25T18:47:29.652240+0000","last_became_active":"2024-04-25T18:43:12.201274+0000","last_became_peered":"2024-04-25T18:43:12.201274+0000","last_unstale":"2024-04-25T18:47:29.652240+0000","last_undegraded":"2024-04-25T18:47:29.652240+0000","last_fullsized":"2024-04-25T18:47:29.652240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:47.683850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.974010+0000","last_change":"2024-04-25T18:43:16.259551+0000","last_active":"2024-04-25T18:47:24.974010+0000","last_peered":"2024-04-25T18:47:24.974010+0000","last_clean":"2024-04-25T18:47:24.974010+0000","last_became_active":"2024-04-25T18:43:16.259165+0000","last_became_peered":"2024-04-25T18:43:16.259165+0000","last_unstale":"2024-04-25T18:47:24.974010+0000","last_undegraded":"2024-04-25T18:47:24.974010+0000","last_fullsized":"2024-04-25T18:47:24.974010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:23.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804701+0000","last_change":"2024-04-25T18:43:12.197534+0000","last_active":"2024-04-25T18:47:24.804701+0000","last_peered":"2024-04-25T18:47:24.804701+0000","last_clean":"2024-04-25T18:47:24.804701+0000","last_became_active":"2024-04-25T18:43:12.197106+0000","last_became_peered":"2024-04-25T18:43:12.197106+0000","last_unstale":"2024-04-25T18:47:24.804701+0000","last_undegraded":"2024-04-25T18:47:24.804701+0000","last_fullsized":"2024-04-25T18:47:24.804701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:35.399044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981722+0000","last_change":"2024-04-25T18:43:16.257668+0000","last_active":"2024-04-25T18:47:24.981722+0000","last_peered":"2024-04-25T18:47:24.981722+0000","last_clean":"2024-04-25T18:47:24.981722+0000","last_became_active":"2024-04-25T18:43:16.253609+0000","last_became_peered":"2024-04-25T18:43:16.253609+0000","last_unstale":"2024-04-25T18:47:24.981722+0000","last_undegraded":"2024-04-25T18:47:24.981722+0000","last_fullsized":"2024-04-25T18:47:24.981722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:36.048386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644803+0000","last_change":"2024-04-25T18:43:12.207322+0000","last_active":"2024-04-25T18:47:24.644803+0000","last_peered":"2024-04-25T18:47:24.644803+0000","last_clean":"2024-04-25T18:47:24.644803+0000","last_became_active":"2024-04-25T18:43:12.206947+0000","last_became_peered":"2024-04-25T18:43:12.206947+0000","last_unstale":"2024-04-25T18:47:24.644803+0000","last_undegraded":"2024-04-25T18:47:24.644803+0000","last_fullsized":"2024-04-25T18:47:24.644803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:20.536598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973667+0000","last_change":"2024-04-25T18:43:16.239575+0000","last_active":"2024-04-25T18:47:24.973667+0000","last_peered":"2024-04-25T18:47:24.973667+0000","last_clean":"2024-04-25T18:47:24.973667+0000","last_became_active":"2024-04-25T18:43:16.239208+0000","last_became_peered":"2024-04-25T18:43:16.239208+0000","last_unstale":"2024-04-25T18:47:24.973667+0000","last_undegraded":"2024-04-25T18:47:24.973667+0000","last_fullsized":"2024-04-25T18:47:24.973667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:58.195855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":541,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:29.748325+0000","last_change":"2024-04-25T18:43:12.197199+0000","last_active":"2024-04-25T18:47:29.748325+0000","last_peered":"2024-04-25T18:47:29.748325+0000","last_clean":"2024-04-25T18:47:29.748325+0000","last_became_active":"2024-04-25T18:43:12.196776+0000","last_became_peered":"2024-04-25T18:43:12.196776+0000","last_unstale":"2024-04-25T18:47:29.748325+0000","last_undegraded":"2024-04-25T18:47:29.748325+0000","last_fullsized":"2024-04-25T18:47:29.748325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:54.149782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":74349,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":471,"num_read_kb":308,"num_write":42,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629197+0000","last_change":"2024-04-25T18:43:16.250993+0000","last_active":"2024-04-25T18:47:24.629197+0000","last_peered":"2024-04-25T18:47:24.629197+0000","last_clean":"2024-04-25T18:47:24.629197+0000","last_became_active":"2024-04-25T18:43:16.250216+0000","last_became_peered":"2024-04-25T18:43:16.250216+0000","last_unstale":"2024-04-25T18:47:24.629197+0000","last_undegraded":"2024-04-25T18:47:24.629197+0000","last_fullsized":"2024-04-25T18:47:24.629197+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:37.996463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973654+0000","last_change":"2024-04-25T18:43:12.204606+0000","last_active":"2024-04-25T18:47:24.973654+0000","last_peered":"2024-04-25T18:47:24.973654+0000","last_clean":"2024-04-25T18:47:24.973654+0000","last_became_active":"2024-04-25T18:43:12.204306+0000","last_became_peered":"2024-04-25T18:43:12.204306+0000","last_unstale":"2024-04-25T18:47:24.973654+0000","last_undegraded":"2024-04-25T18:47:24.973654+0000","last_fullsized":"2024-04-25T18:47:24.973654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:29.903775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.632833+0000","last_change":"2024-04-25T18:43:16.250093+0000","last_active":"2024-04-25T18:47:24.632833+0000","last_peered":"2024-04-25T18:47:24.632833+0000","last_clean":"2024-04-25T18:47:24.632833+0000","last_became_active":"2024-04-25T18:43:16.249567+0000","last_became_peered":"2024-04-25T18:43:16.249567+0000","last_unstale":"2024-04-25T18:47:24.632833+0000","last_undegraded":"2024-04-25T18:47:24.632833+0000","last_fullsized":"2024-04-25T18:47:24.632833+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:35.498641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689370+0000","last_change":"2024-04-25T18:43:56.652488+0000","last_active":"2024-04-25T18:47:24.689370+0000","last_peered":"2024-04-25T18:47:24.689370+0000","last_clean":"2024-04-25T18:47:24.689370+0000","last_became_active":"2024-04-25T18:43:56.647745+0000","last_became_peered":"2024-04-25T18:43:56.647745+0000","last_unstale":"2024-04-25T18:47:24.689370+0000","last_undegraded":"2024-04-25T18:47:24.689370+0000","last_fullsized":"2024-04-25T18:47:24.689370+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:33.474470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804304+0000","last_change":"2024-04-25T18:43:56.642673+0000","last_active":"2024-04-25T18:47:24.804304+0000","last_peered":"2024-04-25T18:47:24.804304+0000","last_clean":"2024-04-25T18:47:24.804304+0000","last_became_active":"2024-04-25T18:43:56.642313+0000","last_became_peered":"2024-04-25T18:43:56.642313+0000","last_unstale":"2024-04-25T18:47:24.804304+0000","last_undegraded":"2024-04-25T18:47:24.804304+0000","last_fullsized":"2024-04-25T18:47:24.804304+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:45.805738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'4","reported_seq":265,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:28.692158+0000","last_change":"2024-04-25T18:43:12.197341+0000","last_active":"2024-04-25T18:47:28.692158+0000","last_peered":"2024-04-25T18:47:28.692158+0000","last_clean":"2024-04-25T18:47:28.692158+0000","last_became_active":"2024-04-25T18:43:12.196952+0000","last_became_peered":"2024-04-25T18:43:12.196952+0000","last_unstale":"2024-04-25T18:47:28.692158+0000","last_undegraded":"2024-04-25T18:47:28.692158+0000","last_fullsized":"2024-04-25T18:47:28.692158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:52.902579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":233,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804357+0000","last_change":"2024-04-25T18:43:16.245436+0000","last_active":"2024-04-25T18:47:24.804357+0000","last_peered":"2024-04-25T18:47:24.804357+0000","last_clean":"2024-04-25T18:47:24.804357+0000","last_became_active":"2024-04-25T18:43:16.244943+0000","last_became_peered":"2024-04-25T18:43:16.244943+0000","last_unstale":"2024-04-25T18:47:24.804357+0000","last_undegraded":"2024-04-25T18:47:24.804357+0000","last_fullsized":"2024-04-25T18:47:24.804357+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:03.402999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628839+0000","last_change":"2024-04-25T18:43:12.197642+0000","last_active":"2024-04-25T18:47:24.628839+0000","last_peered":"2024-04-25T18:47:24.628839+0000","last_clean":"2024-04-25T18:47:24.628839+0000","last_became_active":"2024-04-25T18:43:12.196395+0000","last_became_peered":"2024-04-25T18:43:12.196395+0000","last_unstale":"2024-04-25T18:47:24.628839+0000","last_undegraded":"2024-04-25T18:47:24.628839+0000","last_fullsized":"2024-04-25T18:47:24.628839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:11.783422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644371+0000","last_change":"2024-04-25T18:43:16.245344+0000","last_active":"2024-04-25T18:47:24.644371+0000","last_peered":"2024-04-25T18:47:24.644371+0000","last_clean":"2024-04-25T18:47:24.644371+0000","last_became_active":"2024-04-25T18:43:16.244984+0000","last_became_peered":"2024-04-25T18:43:16.244984+0000","last_unstale":"2024-04-25T18:47:24.644371+0000","last_undegraded":"2024-04-25T18:47:24.644371+0000","last_fullsized":"2024-04-25T18:47:24.644371+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:04.822629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629525+0000","last_change":"2024-04-25T18:43:12.199685+0000","last_active":"2024-04-25T18:47:24.629525+0000","last_peered":"2024-04-25T18:47:24.629525+0000","last_clean":"2024-04-25T18:47:24.629525+0000","last_became_active":"2024-04-25T18:43:12.199261+0000","last_became_peered":"2024-04-25T18:43:12.199261+0000","last_unstale":"2024-04-25T18:47:24.629525+0000","last_undegraded":"2024-04-25T18:47:24.629525+0000","last_fullsized":"2024-04-25T18:47:24.629525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:09.784616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689634+0000","last_change":"2024-04-25T18:43:16.247745+0000","last_active":"2024-04-25T18:47:24.689634+0000","last_peered":"2024-04-25T18:47:24.689634+0000","last_clean":"2024-04-25T18:47:24.689634+0000","last_became_active":"2024-04-25T18:43:16.247311+0000","last_became_peered":"2024-04-25T18:43:16.247311+0000","last_unstale":"2024-04-25T18:47:24.689634+0000","last_undegraded":"2024-04-25T18:47:24.689634+0000","last_fullsized":"2024-04-25T18:47:24.689634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:29.925464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.763129+0000","last_change":"2024-04-25T18:43:12.210209+0000","last_active":"2024-04-25T18:47:24.763129+0000","last_peered":"2024-04-25T18:47:24.763129+0000","last_clean":"2024-04-25T18:47:24.763129+0000","last_became_active":"2024-04-25T18:43:12.209708+0000","last_became_peered":"2024-04-25T18:43:12.209708+0000","last_unstale":"2024-04-25T18:47:24.763129+0000","last_undegraded":"2024-04-25T18:47:24.763129+0000","last_fullsized":"2024-04-25T18:47:24.763129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:07:18.331518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'5","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641664+0000","last_change":"2024-04-25T18:43:16.263616+0000","last_active":"2024-04-25T18:47:24.641664+0000","last_peered":"2024-04-25T18:47:24.641664+0000","last_clean":"2024-04-25T18:47:24.641664+0000","last_became_active":"2024-04-25T18:43:16.263287+0000","last_became_peered":"2024-04-25T18:43:16.263287+0000","last_unstale":"2024-04-25T18:47:24.641664+0000","last_undegraded":"2024-04-25T18:47:24.641664+0000","last_fullsized":"2024-04-25T18:47:24.641664+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:46.366457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804594+0000","last_change":"2024-04-25T18:43:12.203705+0000","last_active":"2024-04-25T18:47:24.804594+0000","last_peered":"2024-04-25T18:47:24.804594+0000","last_clean":"2024-04-25T18:47:24.804594+0000","last_became_active":"2024-04-25T18:43:12.203371+0000","last_became_peered":"2024-04-25T18:43:12.203371+0000","last_unstale":"2024-04-25T18:47:24.804594+0000","last_undegraded":"2024-04-25T18:47:24.804594+0000","last_fullsized":"2024-04-25T18:47:24.804594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641542+0000","last_change":"2024-04-25T18:43:56.647597+0000","last_active":"2024-04-25T18:47:24.641542+0000","last_peered":"2024-04-25T18:47:24.641542+0000","last_clean":"2024-04-25T18:47:24.641542+0000","last_became_active":"2024-04-25T18:43:56.647121+0000","last_became_peered":"2024-04-25T18:43:56.647121+0000","last_unstale":"2024-04-25T18:47:24.641542+0000","last_undegraded":"2024-04-25T18:47:24.641542+0000","last_fullsized":"2024-04-25T18:47:24.641542+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:07.143806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644686+0000","last_change":"2024-04-25T18:43:16.247133+0000","last_active":"2024-04-25T18:47:24.644686+0000","last_peered":"2024-04-25T18:47:24.644686+0000","last_clean":"2024-04-25T18:47:24.644686+0000","last_became_active":"2024-04-25T18:43:16.246232+0000","last_became_peered":"2024-04-25T18:43:16.246232+0000","last_unstale":"2024-04-25T18:47:24.644686+0000","last_undegraded":"2024-04-25T18:47:24.644686+0000","last_fullsized":"2024-04-25T18:47:24.644686+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.288404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":264,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:28.691677+0000","last_change":"2024-04-25T18:43:12.209908+0000","last_active":"2024-04-25T18:47:28.691677+0000","last_peered":"2024-04-25T18:47:28.691677+0000","last_clean":"2024-04-25T18:47:28.691677+0000","last_became_active":"2024-04-25T18:43:12.209042+0000","last_became_peered":"2024-04-25T18:43:12.209042+0000","last_unstale":"2024-04-25T18:47:28.691677+0000","last_undegraded":"2024-04-25T18:47:28.691677+0000","last_fullsized":"2024-04-25T18:47:28.691677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:36.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":229,"num_read_kb":229,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.767262+0000","last_change":"2024-04-25T18:43:16.242364+0000","last_active":"2024-04-25T18:47:24.767262+0000","last_peered":"2024-04-25T18:47:24.767262+0000","last_clean":"2024-04-25T18:47:24.767262+0000","last_became_active":"2024-04-25T18:43:16.242019+0000","last_became_peered":"2024-04-25T18:43:16.242019+0000","last_unstale":"2024-04-25T18:47:24.767262+0000","last_undegraded":"2024-04-25T18:47:24.767262+0000","last_fullsized":"2024-04-25T18:47:24.767262+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:13.238129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631512+0000","last_change":"2024-04-25T18:43:12.206606+0000","last_active":"2024-04-25T18:47:24.631512+0000","last_peered":"2024-04-25T18:47:24.631512+0000","last_clean":"2024-04-25T18:47:24.631512+0000","last_became_active":"2024-04-25T18:43:12.206238+0000","last_became_peered":"2024-04-25T18:43:12.206238+0000","last_unstale":"2024-04-25T18:47:24.631512+0000","last_undegraded":"2024-04-25T18:47:24.631512+0000","last_fullsized":"2024-04-25T18:47:24.631512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:07.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"95'148","reported_seq":410,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:26.771988+0000","last_change":"2024-04-25T18:41:40.527363+0000","last_active":"2024-04-25T18:47:26.771988+0000","last_peered":"2024-04-25T18:47:26.771988+0000","last_clean":"2024-04-25T18:47:26.771988+0000","last_became_active":"2024-04-25T18:41:40.526907+0000","last_became_peered":"2024-04-25T18:41:40.526907+0000","last_unstale":"2024-04-25T18:47:26.771988+0000","last_undegraded":"2024-04-25T18:47:26.771988+0000","last_fullsized":"2024-04-25T18:47:26.771988+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_clean_scrub_stamp":"2024-04-25T18:37:45.963565+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:16.689693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.630889+0000","last_change":"2024-04-25T18:43:16.247552+0000","last_active":"2024-04-25T18:47:24.630889+0000","last_peered":"2024-04-25T18:47:24.630889+0000","last_clean":"2024-04-25T18:47:24.630889+0000","last_became_active":"2024-04-25T18:43:16.247178+0000","last_became_peered":"2024-04-25T18:43:16.247178+0000","last_unstale":"2024-04-25T18:47:24.630889+0000","last_undegraded":"2024-04-25T18:47:24.630889+0000","last_fullsized":"2024-04-25T18:47:24.630889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:50.072986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804356+0000","last_change":"2024-04-25T18:43:56.636197+0000","last_active":"2024-04-25T18:47:24.804356+0000","last_peered":"2024-04-25T18:47:24.804356+0000","last_clean":"2024-04-25T18:47:24.804356+0000","last_became_active":"2024-04-25T18:43:56.633431+0000","last_became_peered":"2024-04-25T18:43:56.633431+0000","last_unstale":"2024-04-25T18:47:24.804356+0000","last_undegraded":"2024-04-25T18:47:24.804356+0000","last_fullsized":"2024-04-25T18:47:24.804356+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:12.463489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629270+0000","last_change":"2024-04-25T18:43:12.202101+0000","last_active":"2024-04-25T18:47:24.629270+0000","last_peered":"2024-04-25T18:47:24.629270+0000","last_clean":"2024-04-25T18:47:24.629270+0000","last_became_active":"2024-04-25T18:43:12.201727+0000","last_became_peered":"2024-04-25T18:43:12.201727+0000","last_unstale":"2024-04-25T18:47:24.629270+0000","last_undegraded":"2024-04-25T18:47:24.629270+0000","last_fullsized":"2024-04-25T18:47:24.629270+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:33.132072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631089+0000","last_change":"2024-04-25T18:43:16.255433+0000","last_active":"2024-04-25T18:47:24.631089+0000","last_peered":"2024-04-25T18:47:24.631089+0000","last_clean":"2024-04-25T18:47:24.631089+0000","last_became_active":"2024-04-25T18:43:16.254866+0000","last_became_peered":"2024-04-25T18:43:16.254866+0000","last_unstale":"2024-04-25T18:47:24.631089+0000","last_undegraded":"2024-04-25T18:47:24.631089+0000","last_fullsized":"2024-04-25T18:47:24.631089+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:07.876973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"95'102","reported_seq":236,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:28.169230+0000","last_change":"2024-04-25T18:43:56.656998+0000","last_active":"2024-04-25T18:47:28.169230+0000","last_peered":"2024-04-25T18:47:28.169230+0000","last_clean":"2024-04-25T18:47:28.169230+0000","last_became_active":"2024-04-25T18:43:56.647456+0000","last_became_peered":"2024-04-25T18:43:56.647456+0000","last_unstale":"2024-04-25T18:47:28.169230+0000","last_undegraded":"2024-04-25T18:47:28.169230+0000","last_fullsized":"2024-04-25T18:47:28.169230+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:00.719426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768548+0000","last_change":"2024-04-25T18:43:16.252826+0000","last_active":"2024-04-25T18:47:24.768548+0000","last_peered":"2024-04-25T18:47:24.768548+0000","last_clean":"2024-04-25T18:47:24.768548+0000","last_became_active":"2024-04-25T18:43:16.252435+0000","last_became_peered":"2024-04-25T18:43:16.252435+0000","last_unstale":"2024-04-25T18:47:24.768548+0000","last_undegraded":"2024-04-25T18:47:24.768548+0000","last_fullsized":"2024-04-25T18:47:24.768548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:13.401006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631883+0000","last_change":"2024-04-25T18:43:12.376121+0000","last_active":"2024-04-25T18:47:24.631883+0000","last_peered":"2024-04-25T18:47:24.631883+0000","last_clean":"2024-04-25T18:47:24.631883+0000","last_became_active":"2024-04-25T18:43:12.375757+0000","last_became_peered":"2024-04-25T18:43:12.375757+0000","last_unstale":"2024-04-25T18:47:24.631883+0000","last_undegraded":"2024-04-25T18:47:24.631883+0000","last_fullsized":"2024-04-25T18:47:24.631883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:15.610442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981855+0000","last_change":"2024-04-25T18:43:16.266504+0000","last_active":"2024-04-25T18:47:24.981855+0000","last_peered":"2024-04-25T18:47:24.981855+0000","last_clean":"2024-04-25T18:47:24.981855+0000","last_became_active":"2024-04-25T18:43:16.265048+0000","last_became_peered":"2024-04-25T18:43:16.265048+0000","last_unstale":"2024-04-25T18:47:24.981855+0000","last_undegraded":"2024-04-25T18:47:24.981855+0000","last_fullsized":"2024-04-25T18:47:24.981855+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:36.924872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973501+0000","last_change":"2024-04-25T18:43:12.207529+0000","last_active":"2024-04-25T18:47:24.973501+0000","last_peered":"2024-04-25T18:47:24.973501+0000","last_clean":"2024-04-25T18:47:24.973501+0000","last_became_active":"2024-04-25T18:43:12.207122+0000","last_became_peered":"2024-04-25T18:43:12.207122+0000","last_unstale":"2024-04-25T18:47:24.973501+0000","last_undegraded":"2024-04-25T18:47:24.973501+0000","last_fullsized":"2024-04-25T18:47:24.973501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:44.518497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628853+0000","last_change":"2024-04-25T18:43:16.249616+0000","last_active":"2024-04-25T18:47:24.628853+0000","last_peered":"2024-04-25T18:47:24.628853+0000","last_clean":"2024-04-25T18:47:24.628853+0000","last_became_active":"2024-04-25T18:43:16.249252+0000","last_became_peered":"2024-04-25T18:43:16.249252+0000","last_unstale":"2024-04-25T18:47:24.628853+0000","last_undegraded":"2024-04-25T18:47:24.628853+0000","last_fullsized":"2024-04-25T18:47:24.628853+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:34.721513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.766887+0000","last_change":"2024-04-25T18:43:12.209304+0000","last_active":"2024-04-25T18:47:24.766887+0000","last_peered":"2024-04-25T18:47:24.766887+0000","last_clean":"2024-04-25T18:47:24.766887+0000","last_became_active":"2024-04-25T18:43:12.208887+0000","last_became_peered":"2024-04-25T18:43:12.208887+0000","last_unstale":"2024-04-25T18:47:24.766887+0000","last_undegraded":"2024-04-25T18:47:24.766887+0000","last_fullsized":"2024-04-25T18:47:24.766887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:22.049809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689315+0000","last_change":"2024-04-25T18:43:16.262482+0000","last_active":"2024-04-25T18:47:24.689315+0000","last_peered":"2024-04-25T18:47:24.689315+0000","last_clean":"2024-04-25T18:47:24.689315+0000","last_became_active":"2024-04-25T18:43:16.262108+0000","last_became_peered":"2024-04-25T18:43:16.262108+0000","last_unstale":"2024-04-25T18:47:24.689315+0000","last_undegraded":"2024-04-25T18:47:24.689315+0000","last_fullsized":"2024-04-25T18:47:24.689315+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:14.777939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973787+0000","last_change":"2024-04-25T18:43:12.205742+0000","last_active":"2024-04-25T18:47:24.973787+0000","last_peered":"2024-04-25T18:47:24.973787+0000","last_clean":"2024-04-25T18:47:24.973787+0000","last_became_active":"2024-04-25T18:43:12.205314+0000","last_became_peered":"2024-04-25T18:43:12.205314+0000","last_unstale":"2024-04-25T18:47:24.973787+0000","last_undegraded":"2024-04-25T18:47:24.973787+0000","last_fullsized":"2024-04-25T18:47:24.973787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:49.641568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.767568+0000","last_change":"2024-04-25T18:43:16.266100+0000","last_active":"2024-04-25T18:47:24.767568+0000","last_peered":"2024-04-25T18:47:24.767568+0000","last_clean":"2024-04-25T18:47:24.767568+0000","last_became_active":"2024-04-25T18:43:16.263060+0000","last_became_peered":"2024-04-25T18:43:16.263060+0000","last_unstale":"2024-04-25T18:47:24.767568+0000","last_undegraded":"2024-04-25T18:47:24.767568+0000","last_fullsized":"2024-04-25T18:47:24.767568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:38.269244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628994+0000","last_change":"2024-04-25T18:43:12.196054+0000","last_active":"2024-04-25T18:47:24.628994+0000","last_peered":"2024-04-25T18:47:24.628994+0000","last_clean":"2024-04-25T18:47:24.628994+0000","last_became_active":"2024-04-25T18:43:12.195503+0000","last_became_peered":"2024-04-25T18:43:12.195503+0000","last_unstale":"2024-04-25T18:47:24.628994+0000","last_undegraded":"2024-04-25T18:47:24.628994+0000","last_fullsized":"2024-04-25T18:47:24.628994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:55.187176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804544+0000","last_change":"2024-04-25T18:43:16.242536+0000","last_active":"2024-04-25T18:47:24.804544+0000","last_peered":"2024-04-25T18:47:24.804544+0000","last_clean":"2024-04-25T18:47:24.804544+0000","last_became_active":"2024-04-25T18:43:16.241060+0000","last_became_peered":"2024-04-25T18:43:16.241060+0000","last_unstale":"2024-04-25T18:47:24.804544+0000","last_undegraded":"2024-04-25T18:47:24.804544+0000","last_fullsized":"2024-04-25T18:47:24.804544+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:38.306190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631518+0000","last_change":"2024-04-25T18:43:56.653377+0000","last_active":"2024-04-25T18:47:24.631518+0000","last_peered":"2024-04-25T18:47:24.631518+0000","last_clean":"2024-04-25T18:47:24.631518+0000","last_became_active":"2024-04-25T18:43:56.645516+0000","last_became_peered":"2024-04-25T18:43:56.645516+0000","last_unstale":"2024-04-25T18:47:24.631518+0000","last_undegraded":"2024-04-25T18:47:24.631518+0000","last_fullsized":"2024-04-25T18:47:24.631518+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:11.189494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804771+0000","last_change":"2024-04-25T18:43:16.247610+0000","last_active":"2024-04-25T18:47:24.804771+0000","last_peered":"2024-04-25T18:47:24.804771+0000","last_clean":"2024-04-25T18:47:24.804771+0000","last_became_active":"2024-04-25T18:43:16.247273+0000","last_became_peered":"2024-04-25T18:43:16.247273+0000","last_unstale":"2024-04-25T18:47:24.804771+0000","last_undegraded":"2024-04-25T18:47:24.804771+0000","last_fullsized":"2024-04-25T18:47:24.804771+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:26.151480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'65","reported_seq":125,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:27.707162+0000","last_change":"2024-04-25T18:43:12.202011+0000","last_active":"2024-04-25T18:47:27.707162+0000","last_peered":"2024-04-25T18:47:27.707162+0000","last_clean":"2024-04-25T18:47:27.707162+0000","last_became_active":"2024-04-25T18:43:12.201447+0000","last_became_peered":"2024-04-25T18:43:12.201447+0000","last_unstale":"2024-04-25T18:47:27.707162+0000","last_undegraded":"2024-04-25T18:47:27.707162+0000","last_fullsized":"2024-04-25T18:47:27.707162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:48.094757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804583+0000","last_change":"2024-04-25T18:43:16.249033+0000","last_active":"2024-04-25T18:47:24.804583+0000","last_peered":"2024-04-25T18:47:24.804583+0000","last_clean":"2024-04-25T18:47:24.804583+0000","last_became_active":"2024-04-25T18:43:16.248691+0000","last_became_peered":"2024-04-25T18:43:16.248691+0000","last_unstale":"2024-04-25T18:47:24.804583+0000","last_undegraded":"2024-04-25T18:47:24.804583+0000","last_fullsized":"2024-04-25T18:47:24.804583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:37.335824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631177+0000","last_change":"2024-04-25T18:43:12.205134+0000","last_active":"2024-04-25T18:47:24.631177+0000","last_peered":"2024-04-25T18:47:24.631177+0000","last_clean":"2024-04-25T18:47:24.631177+0000","last_became_active":"2024-04-25T18:43:12.204568+0000","last_became_peered":"2024-04-25T18:43:12.204568+0000","last_unstale":"2024-04-25T18:47:24.631177+0000","last_undegraded":"2024-04-25T18:47:24.631177+0000","last_fullsized":"2024-04-25T18:47:24.631177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:32.806580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804824+0000","last_change":"2024-04-25T18:43:16.247337+0000","last_active":"2024-04-25T18:47:24.804824+0000","last_peered":"2024-04-25T18:47:24.804824+0000","last_clean":"2024-04-25T18:47:24.804824+0000","last_became_active":"2024-04-25T18:43:16.246971+0000","last_became_peered":"2024-04-25T18:43:16.246971+0000","last_unstale":"2024-04-25T18:47:24.804824+0000","last_undegraded":"2024-04-25T18:47:24.804824+0000","last_fullsized":"2024-04-25T18:47:24.804824+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.882941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689499+0000","last_change":"2024-04-25T18:43:16.252196+0000","last_active":"2024-04-25T18:47:24.689499+0000","last_peered":"2024-04-25T18:47:24.689499+0000","last_clean":"2024-04-25T18:47:24.689499+0000","last_became_active":"2024-04-25T18:43:16.251715+0000","last_became_peered":"2024-04-25T18:43:16.251715+0000","last_unstale":"2024-04-25T18:47:24.689499+0000","last_undegraded":"2024-04-25T18:47:24.689499+0000","last_fullsized":"2024-04-25T18:47:24.689499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:17.218925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973649+0000","last_change":"2024-04-25T18:43:16.247632+0000","last_active":"2024-04-25T18:47:24.973649+0000","last_peered":"2024-04-25T18:47:24.973649+0000","last_clean":"2024-04-25T18:47:24.973649+0000","last_became_active":"2024-04-25T18:43:16.247305+0000","last_became_peered":"2024-04-25T18:43:16.247305+0000","last_unstale":"2024-04-25T18:47:24.973649+0000","last_undegraded":"2024-04-25T18:47:24.973649+0000","last_fullsized":"2024-04-25T18:47:24.973649+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:52.576146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973626+0000","last_change":"2024-04-25T18:43:16.266539+0000","last_active":"2024-04-25T18:47:24.973626+0000","last_peered":"2024-04-25T18:47:24.973626+0000","last_clean":"2024-04-25T18:47:24.973626+0000","last_became_active":"2024-04-25T18:43:16.266083+0000","last_became_peered":"2024-04-25T18:43:16.266083+0000","last_unstale":"2024-04-25T18:47:24.973626+0000","last_undegraded":"2024-04-25T18:47:24.973626+0000","last_fullsized":"2024-04-25T18:47:24.973626+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:49.589019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981404+0000","last_change":"2024-04-25T18:43:16.264002+0000","last_active":"2024-04-25T18:47:24.981404+0000","last_peered":"2024-04-25T18:47:24.981404+0000","last_clean":"2024-04-25T18:47:24.981404+0000","last_became_active":"2024-04-25T18:43:16.261555+0000","last_became_peered":"2024-04-25T18:43:16.261555+0000","last_unstale":"2024-04-25T18:47:24.981404+0000","last_undegraded":"2024-04-25T18:47:24.981404+0000","last_fullsized":"2024-04-25T18:47:24.981404+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:12.637040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.630913+0000","last_change":"2024-04-25T18:43:16.270914+0000","last_active":"2024-04-25T18:47:24.630913+0000","last_peered":"2024-04-25T18:47:24.630913+0000","last_clean":"2024-04-25T18:47:24.630913+0000","last_became_active":"2024-04-25T18:43:16.270117+0000","last_became_peered":"2024-04-25T18:43:16.270117+0000","last_unstale":"2024-04-25T18:47:24.630913+0000","last_undegraded":"2024-04-25T18:47:24.630913+0000","last_fullsized":"2024-04-25T18:47:24.630913+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:19.874006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689429+0000","last_change":"2024-04-25T18:43:16.247032+0000","last_active":"2024-04-25T18:47:24.689429+0000","last_peered":"2024-04-25T18:47:24.689429+0000","last_clean":"2024-04-25T18:47:24.689429+0000","last_became_active":"2024-04-25T18:43:16.246734+0000","last_became_peered":"2024-04-25T18:43:16.246734+0000","last_unstale":"2024-04-25T18:47:24.689429+0000","last_undegraded":"2024-04-25T18:47:24.689429+0000","last_fullsized":"2024-04-25T18:47:24.689429+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:39.159143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973860+0000","last_change":"2024-04-25T18:43:16.245263+0000","last_active":"2024-04-25T18:47:24.973860+0000","last_peered":"2024-04-25T18:47:24.973860+0000","last_clean":"2024-04-25T18:47:24.973860+0000","last_became_active":"2024-04-25T18:43:16.244856+0000","last_became_peered":"2024-04-25T18:43:16.244856+0000","last_unstale":"2024-04-25T18:47:24.973860+0000","last_undegraded":"2024-04-25T18:47:24.973860+0000","last_fullsized":"2024-04-25T18:47:24.973860+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:36.157927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768461+0000","last_change":"2024-04-25T18:43:16.253523+0000","last_active":"2024-04-25T18:47:24.768461+0000","last_peered":"2024-04-25T18:47:24.768461+0000","last_clean":"2024-04-25T18:47:24.768461+0000","last_became_active":"2024-04-25T18:43:16.252689+0000","last_became_peered":"2024-04-25T18:43:16.252689+0000","last_unstale":"2024-04-25T18:47:24.768461+0000","last_undegraded":"2024-04-25T18:47:24.768461+0000","last_fullsized":"2024-04-25T18:47:24.768461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.035788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644415+0000","last_change":"2024-04-25T18:43:16.255818+0000","last_active":"2024-04-25T18:47:24.644415+0000","last_peered":"2024-04-25T18:47:24.644415+0000","last_clean":"2024-04-25T18:47:24.644415+0000","last_became_active":"2024-04-25T18:43:16.255521+0000","last_became_peered":"2024-04-25T18:43:16.255521+0000","last_unstale":"2024-04-25T18:47:24.644415+0000","last_undegraded":"2024-04-25T18:47:24.644415+0000","last_fullsized":"2024-04-25T18:47:24.644415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:15.457585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.766729+0000","last_change":"2024-04-25T18:43:16.259680+0000","last_active":"2024-04-25T18:47:24.766729+0000","last_peered":"2024-04-25T18:47:24.766729+0000","last_clean":"2024-04-25T18:47:24.766729+0000","last_became_active":"2024-04-25T18:43:16.250065+0000","last_became_peered":"2024-04-25T18:43:16.250065+0000","last_unstale":"2024-04-25T18:47:24.766729+0000","last_undegraded":"2024-04-25T18:47:24.766729+0000","last_fullsized":"2024-04-25T18:47:24.766729+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:44.974919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973908+0000","last_change":"2024-04-25T18:43:16.239801+0000","last_active":"2024-04-25T18:47:24.973908+0000","last_peered":"2024-04-25T18:47:24.973908+0000","last_clean":"2024-04-25T18:47:24.973908+0000","last_became_active":"2024-04-25T18:43:16.239475+0000","last_became_peered":"2024-04-25T18:43:16.239475+0000","last_unstale":"2024-04-25T18:47:24.973908+0000","last_undegraded":"2024-04-25T18:47:24.973908+0000","last_fullsized":"2024-04-25T18:47:24.973908+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:57.407476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.630762+0000","last_change":"2024-04-25T18:43:16.270419+0000","last_active":"2024-04-25T18:47:24.630762+0000","last_peered":"2024-04-25T18:47:24.630762+0000","last_clean":"2024-04-25T18:47:24.630762+0000","last_became_active":"2024-04-25T18:43:16.269995+0000","last_became_peered":"2024-04-25T18:43:16.269995+0000","last_unstale":"2024-04-25T18:47:24.630762+0000","last_undegraded":"2024-04-25T18:47:24.630762+0000","last_fullsized":"2024-04-25T18:47:24.630762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:27.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689424+0000","last_change":"2024-04-25T18:43:16.246948+0000","last_active":"2024-04-25T18:47:24.689424+0000","last_peered":"2024-04-25T18:47:24.689424+0000","last_clean":"2024-04-25T18:47:24.689424+0000","last_became_active":"2024-04-25T18:43:16.246616+0000","last_became_peered":"2024-04-25T18:43:16.246616+0000","last_unstale":"2024-04-25T18:47:24.689424+0000","last_undegraded":"2024-04-25T18:47:24.689424+0000","last_fullsized":"2024-04-25T18:47:24.689424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:51.817448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800721,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1755,"num_read_kb":1097,"num_write":428,"num_write_kb":2364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":582,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2402163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":314,"ondisk_log_size":314,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252932,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32580,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712572,"statfs":{"total":95995035648,"available":95961673728,"internally_reserved":0,"allocated":1437696,"data_stored":900958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514569,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37372,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707780,"statfs":{"total":95995035648,"available":95956766720,"internally_reserved":0,"allocated":2084864,"data_stored":1502195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710797,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1454080,"data_stored":901194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972436,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":507898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234074,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1100751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430303,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":507367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":581509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":508638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1142784,"data_stored":583200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117105,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37204,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707948,"statfs":{"total":95995035648,"available":95956938752,"internally_reserved":0,"allocated":1789952,"data_stored":1229126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855468,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":508777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691941,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29088,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716064,"statfs":{"total":95995035648,"available":95965249536,"internally_reserved":0,"allocated":1998848,"data_stored":1428276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":74349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":76040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":404550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:47:36.777 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T18:47:36.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:47:37.271 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:37.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:37 smithi161 ceph-mon[32609]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:47:37.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:37 smithi112 ceph-mon[27864]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:47:37.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:37 smithi171 ceph-mon[33266]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:47:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:38 smithi161 ceph-mon[32609]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-25T18:47:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:38 smithi112 ceph-mon[27864]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-25T18:47:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:38 smithi171 ceph-mon[33266]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-25T18:47:39.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.616+0000 7fae6be00700 1 -- 172.21.15.112:0/3950524796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae64104ee0 msgr2=0x7fae641052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:39.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.616+0000 7fae6be00700 1 --2- 172.21.15.112:0/3950524796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae64104ee0 0x7fae641052c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fae58009920 tx=0x7fae58009c30 comp rx=0 tx=0).stop 2024-04-25T18:47:39.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.617+0000 7fae6be00700 1 -- 172.21.15.112:0/3950524796 shutdown_connections 2024-04-25T18:47:39.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.617+0000 7fae6be00700 1 --2- 172.21.15.112:0/3950524796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae64109fc0 0x7fae64111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:39.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.617+0000 7fae6be00700 1 --2- 172.21.15.112:0/3950524796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae64104ee0 0x7fae641052c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:39.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.617+0000 7fae6be00700 1 --2- 172.21.15.112:0/3950524796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae64105890 0x7fae641098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:39.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.617+0000 7fae6be00700 1 -- 172.21.15.112:0/3950524796 >> 172.21.15.112:0/3950524796 conn(0x7fae64100770 msgr2=0x7fae64102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:39.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.617+0000 7fae6be00700 1 -- 172.21.15.112:0/3950524796 shutdown_connections 2024-04-25T18:47:39.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.617+0000 7fae6be00700 1 -- 172.21.15.112:0/3950524796 wait complete. 2024-04-25T18:47:39.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.618+0000 7fae6be00700 1 Processor -- start 2024-04-25T18:47:39.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.618+0000 7fae6be00700 1 -- start start 2024-04-25T18:47:39.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.618+0000 7fae6be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae64104ee0 0x7fae6419b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:39.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.618+0000 7fae6be00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae64105890 0x7fae6419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:39.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.618+0000 7fae6be00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae64109fc0 0x7fae6419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:39.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.618+0000 7fae6be00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae64114ed0 con 0x7fae64105890 2024-04-25T18:47:39.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.618+0000 7fae6be00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fae64114d50 con 0x7fae64104ee0 2024-04-25T18:47:39.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.618+0000 7fae6be00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fae64115050 con 0x7fae64109fc0 2024-04-25T18:47:39.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.619+0000 7fae68a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae64105890 0x7fae6419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:39.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.619+0000 7fae68a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae64105890 0x7fae6419ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34692/0 (socket says 172.21.15.112:34692) 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.619+0000 7fae69400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae64104ee0 0x7fae6419b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.619+0000 7fae68a00700 1 -- 172.21.15.112:0/2263945865 learned_addr learned my addr 172.21.15.112:0/2263945865 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.619+0000 7fae69e00700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae64109fc0 0x7fae6419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.619+0000 7fae68a00700 1 -- 172.21.15.112:0/2263945865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae64109fc0 msgr2=0x7fae6419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.619+0000 7fae68a00700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae64109fc0 0x7fae6419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.619+0000 7fae68a00700 1 -- 172.21.15.112:0/2263945865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae64104ee0 msgr2=0x7fae6419b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.619+0000 7fae68a00700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae64104ee0 0x7fae6419b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.619+0000 7fae68a00700 1 -- 172.21.15.112:0/2263945865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae641a0510 con 0x7fae64105890 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.619+0000 7fae69400700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae64104ee0 0x7fae6419b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.620+0000 7fae68a00700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae64105890 0x7fae6419ba40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fae5000e9c0 tx=0x7fae5000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.620+0000 7fae62000700 1 -- 172.21.15.112:0/2263945865 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50004a30 con 0x7fae64105890 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.620+0000 7fae62000700 1 -- 172.21.15.112:0/2263945865 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5000cc70 con 0x7fae64105890 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.620+0000 7fae6be00700 1 -- 172.21.15.112:0/2263945865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae641a0800 con 0x7fae64105890 2024-04-25T18:47:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.620+0000 7fae62000700 1 -- 172.21.15.112:0/2263945865 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae500055c0 con 0x7fae64105890 2024-04-25T18:47:39.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.620+0000 7fae6be00700 1 -- 172.21.15.112:0/2263945865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae641a8080 con 0x7fae64105890 2024-04-25T18:47:39.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.621+0000 7fae6be00700 1 -- 172.21.15.112:0/2263945865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae64069730 con 0x7fae64105890 2024-04-25T18:47:39.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.623+0000 7fae62000700 1 -- 172.21.15.112:0/2263945865 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fae50004c60 con 0x7fae64105890 2024-04-25T18:47:39.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.625+0000 7fae62000700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fae4c0a3ed0 0x7fae4c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:39.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.625+0000 7fae62000700 1 -- 172.21.15.112:0/2263945865 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fae500d0cf0 con 0x7fae64105890 2024-04-25T18:47:39.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.625+0000 7fae69400700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fae4c0a3ed0 0x7fae4c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:39.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.626+0000 7fae69400700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fae4c0a3ed0 0x7fae4c0a6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fae58009810 tx=0x7fae58019040 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:39.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.629+0000 7fae62000700 1 -- 172.21.15.112:0/2263945865 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae5000ce20 con 0x7fae64105890 2024-04-25T18:47:39.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.851+0000 7fae6be00700 1 -- 172.21.15.112:0/2263945865 --> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae6419c860 con 0x7fae4c0a3ed0 2024-04-25T18:47:39.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.873+0000 7fae62000700 1 -- 172.21.15.112:0/2263945865 <== mgr.14922 v2:172.21.15.112:6800/2236031982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257775 (secure 0 0 0) 0x7fae6419c860 con 0x7fae4c0a3ed0 2024-04-25T18:47:39.876 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:47:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.879+0000 7fae6be00700 1 -- 172.21.15.112:0/2263945865 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fae4c0a3ed0 msgr2=0x7fae4c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.879+0000 7fae6be00700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fae4c0a3ed0 0x7fae4c0a6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fae58009810 tx=0x7fae58019040 comp rx=0 tx=0).stop 2024-04-25T18:47:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.879+0000 7fae6be00700 1 -- 172.21.15.112:0/2263945865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae64105890 msgr2=0x7fae6419ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.879+0000 7fae6be00700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae64105890 0x7fae6419ba40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fae5000e9c0 tx=0x7fae5000ecd0 comp rx=0 tx=0).stop 2024-04-25T18:47:39.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.879+0000 7fae6be00700 1 -- 172.21.15.112:0/2263945865 shutdown_connections 2024-04-25T18:47:39.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.879+0000 7fae6be00700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fae4c0a3ed0 0x7fae4c0a6390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:39.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.879+0000 7fae6be00700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae64109fc0 0x7fae6419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:39.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.879+0000 7fae6be00700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae64104ee0 0x7fae6419b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:39.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.879+0000 7fae6be00700 1 --2- 172.21.15.112:0/2263945865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae64105890 0x7fae6419ba40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:39.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.879+0000 7fae6be00700 1 -- 172.21.15.112:0/2263945865 >> 172.21.15.112:0/2263945865 conn(0x7fae64100770 msgr2=0x7fae64101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:39.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.880+0000 7fae6be00700 1 -- 172.21.15.112:0/2263945865 shutdown_connections 2024-04-25T18:47:39.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:39.880+0000 7fae6be00700 1 -- 172.21.15.112:0/2263945865 wait complete. 2024-04-25T18:47:39.883 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T18:47:40.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:40 smithi161 ceph-mon[32609]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:47:40.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:40 smithi112 ceph-mon[27864]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:47:40.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:40 smithi171 ceph-mon[33266]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:47:40.846 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-25T18:47:38.654245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391323,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":2022,"num_read_kb":1442,"num_write":631,"num_write_kb":5304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181318,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":964,"ondisk_log_size":964,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382568,"kb_used_data":16524,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559256,"statfs":{"total":1151940427776,"available":1151548678144,"internally_reserved":0,"allocated":16920576,"data_stored":10259889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199934,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":417,"num_read_kb":482,"num_write":154,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003978"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689752+0000","last_change":"2024-04-25T18:43:16.262618+0000","last_active":"2024-04-25T18:47:24.689752+0000","last_peered":"2024-04-25T18:47:24.689752+0000","last_clean":"2024-04-25T18:47:24.689752+0000","last_became_active":"2024-04-25T18:43:16.262215+0000","last_became_peered":"2024-04-25T18:43:16.262215+0000","last_unstale":"2024-04-25T18:47:24.689752+0000","last_undegraded":"2024-04-25T18:47:24.689752+0000","last_fullsized":"2024-04-25T18:47:24.689752+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:40.319102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981786+0000","last_change":"2024-04-25T18:43:16.254085+0000","last_active":"2024-04-25T18:47:24.981786+0000","last_peered":"2024-04-25T18:47:24.981786+0000","last_clean":"2024-04-25T18:47:24.981786+0000","last_became_active":"2024-04-25T18:43:16.248943+0000","last_became_peered":"2024-04-25T18:43:16.248943+0000","last_unstale":"2024-04-25T18:47:24.981786+0000","last_undegraded":"2024-04-25T18:47:24.981786+0000","last_fullsized":"2024-04-25T18:47:24.981786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:53.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981717+0000","last_change":"2024-04-25T18:43:16.252684+0000","last_active":"2024-04-25T18:47:24.981717+0000","last_peered":"2024-04-25T18:47:24.981717+0000","last_clean":"2024-04-25T18:47:24.981717+0000","last_became_active":"2024-04-25T18:43:16.252318+0000","last_became_peered":"2024-04-25T18:43:16.252318+0000","last_unstale":"2024-04-25T18:47:24.981717+0000","last_undegraded":"2024-04-25T18:47:24.981717+0000","last_fullsized":"2024-04-25T18:47:24.981717+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:58.904865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631697+0000","last_change":"2024-04-25T18:43:16.247059+0000","last_active":"2024-04-25T18:47:24.631697+0000","last_peered":"2024-04-25T18:47:24.631697+0000","last_clean":"2024-04-25T18:47:24.631697+0000","last_became_active":"2024-04-25T18:43:16.246655+0000","last_became_peered":"2024-04-25T18:43:16.246655+0000","last_unstale":"2024-04-25T18:47:24.631697+0000","last_undegraded":"2024-04-25T18:47:24.631697+0000","last_fullsized":"2024-04-25T18:47:24.631697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:31.026579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628963+0000","last_change":"2024-04-25T18:43:16.250033+0000","last_active":"2024-04-25T18:47:24.628963+0000","last_peered":"2024-04-25T18:47:24.628963+0000","last_clean":"2024-04-25T18:47:24.628963+0000","last_became_active":"2024-04-25T18:43:16.249017+0000","last_became_peered":"2024-04-25T18:43:16.249017+0000","last_unstale":"2024-04-25T18:47:24.628963+0000","last_undegraded":"2024-04-25T18:47:24.628963+0000","last_fullsized":"2024-04-25T18:47:24.628963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:34.304263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641422+0000","last_change":"2024-04-25T18:43:56.642410+0000","last_active":"2024-04-25T18:47:24.641422+0000","last_peered":"2024-04-25T18:47:24.641422+0000","last_clean":"2024-04-25T18:47:24.641422+0000","last_became_active":"2024-04-25T18:43:56.642009+0000","last_became_peered":"2024-04-25T18:43:56.642009+0000","last_unstale":"2024-04-25T18:47:24.641422+0000","last_undegraded":"2024-04-25T18:47:24.641422+0000","last_fullsized":"2024-04-25T18:47:24.641422+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:19.512475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804293+0000","last_change":"2024-04-25T18:43:56.647158+0000","last_active":"2024-04-25T18:47:24.804293+0000","last_peered":"2024-04-25T18:47:24.804293+0000","last_clean":"2024-04-25T18:47:24.804293+0000","last_became_active":"2024-04-25T18:43:56.646779+0000","last_became_peered":"2024-04-25T18:43:56.646779+0000","last_unstale":"2024-04-25T18:47:24.804293+0000","last_undegraded":"2024-04-25T18:47:24.804293+0000","last_fullsized":"2024-04-25T18:47:24.804293+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:04:14.055762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.766601+0000","last_change":"2024-04-25T18:43:16.247607+0000","last_active":"2024-04-25T18:47:24.766601+0000","last_peered":"2024-04-25T18:47:24.766601+0000","last_clean":"2024-04-25T18:47:24.766601+0000","last_became_active":"2024-04-25T18:43:16.247234+0000","last_became_peered":"2024-04-25T18:43:16.247234+0000","last_unstale":"2024-04-25T18:47:24.766601+0000","last_undegraded":"2024-04-25T18:47:24.766601+0000","last_fullsized":"2024-04-25T18:47:24.766601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:57.117900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973993+0000","last_change":"2024-04-25T18:43:16.247909+0000","last_active":"2024-04-25T18:47:24.973993+0000","last_peered":"2024-04-25T18:47:24.973993+0000","last_clean":"2024-04-25T18:47:24.973993+0000","last_became_active":"2024-04-25T18:43:16.247533+0000","last_became_peered":"2024-04-25T18:43:16.247533+0000","last_unstale":"2024-04-25T18:47:24.973993+0000","last_undegraded":"2024-04-25T18:47:24.973993+0000","last_fullsized":"2024-04-25T18:47:24.973993+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:02.257832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641756+0000","last_change":"2024-04-25T18:43:16.252088+0000","last_active":"2024-04-25T18:47:24.641756+0000","last_peered":"2024-04-25T18:47:24.641756+0000","last_clean":"2024-04-25T18:47:24.641756+0000","last_became_active":"2024-04-25T18:43:16.251580+0000","last_became_peered":"2024-04-25T18:43:16.251580+0000","last_unstale":"2024-04-25T18:47:24.641756+0000","last_undegraded":"2024-04-25T18:47:24.641756+0000","last_fullsized":"2024-04-25T18:47:24.641756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:22.992087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804685+0000","last_change":"2024-04-25T18:43:16.239696+0000","last_active":"2024-04-25T18:47:24.804685+0000","last_peered":"2024-04-25T18:47:24.804685+0000","last_clean":"2024-04-25T18:47:24.804685+0000","last_became_active":"2024-04-25T18:43:16.239354+0000","last_became_peered":"2024-04-25T18:43:16.239354+0000","last_unstale":"2024-04-25T18:47:24.804685+0000","last_undegraded":"2024-04-25T18:47:24.804685+0000","last_fullsized":"2024-04-25T18:47:24.804685+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:39.682138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631515+0000","last_change":"2024-04-25T18:43:16.242487+0000","last_active":"2024-04-25T18:47:24.631515+0000","last_peered":"2024-04-25T18:47:24.631515+0000","last_clean":"2024-04-25T18:47:24.631515+0000","last_became_active":"2024-04-25T18:43:16.242158+0000","last_became_peered":"2024-04-25T18:43:16.242158+0000","last_unstale":"2024-04-25T18:47:24.631515+0000","last_undegraded":"2024-04-25T18:47:24.631515+0000","last_fullsized":"2024-04-25T18:47:24.631515+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:36.477440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.630730+0000","last_change":"2024-04-25T18:43:16.244154+0000","last_active":"2024-04-25T18:47:24.630730+0000","last_peered":"2024-04-25T18:47:24.630730+0000","last_clean":"2024-04-25T18:47:24.630730+0000","last_became_active":"2024-04-25T18:43:16.243829+0000","last_became_peered":"2024-04-25T18:43:16.243829+0000","last_unstale":"2024-04-25T18:47:24.630730+0000","last_undegraded":"2024-04-25T18:47:24.630730+0000","last_fullsized":"2024-04-25T18:47:24.630730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:43.262751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981216+0000","last_change":"2024-04-25T18:43:16.261227+0000","last_active":"2024-04-25T18:47:24.981216+0000","last_peered":"2024-04-25T18:47:24.981216+0000","last_clean":"2024-04-25T18:47:24.981216+0000","last_became_active":"2024-04-25T18:43:16.250468+0000","last_became_peered":"2024-04-25T18:43:16.250468+0000","last_unstale":"2024-04-25T18:47:24.981216+0000","last_undegraded":"2024-04-25T18:47:24.981216+0000","last_fullsized":"2024-04-25T18:47:24.981216+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:44.805643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981094+0000","last_change":"2024-04-25T18:43:16.257956+0000","last_active":"2024-04-25T18:47:24.981094+0000","last_peered":"2024-04-25T18:47:24.981094+0000","last_clean":"2024-04-25T18:47:24.981094+0000","last_became_active":"2024-04-25T18:43:16.255126+0000","last_became_peered":"2024-04-25T18:43:16.255126+0000","last_unstale":"2024-04-25T18:47:24.981094+0000","last_undegraded":"2024-04-25T18:47:24.981094+0000","last_fullsized":"2024-04-25T18:47:24.981094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:30.047165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973501+0000","last_change":"2024-04-25T18:43:16.256208+0000","last_active":"2024-04-25T18:47:24.973501+0000","last_peered":"2024-04-25T18:47:24.973501+0000","last_clean":"2024-04-25T18:47:24.973501+0000","last_became_active":"2024-04-25T18:43:16.255614+0000","last_became_peered":"2024-04-25T18:43:16.255614+0000","last_unstale":"2024-04-25T18:47:24.973501+0000","last_undegraded":"2024-04-25T18:47:24.973501+0000","last_fullsized":"2024-04-25T18:47:24.973501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:46.700209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689815+0000","last_change":"2024-04-25T18:43:16.262730+0000","last_active":"2024-04-25T18:47:24.689815+0000","last_peered":"2024-04-25T18:47:24.689815+0000","last_clean":"2024-04-25T18:47:24.689815+0000","last_became_active":"2024-04-25T18:43:16.262192+0000","last_became_peered":"2024-04-25T18:43:16.262192+0000","last_unstale":"2024-04-25T18:47:24.689815+0000","last_undegraded":"2024-04-25T18:47:24.689815+0000","last_fullsized":"2024-04-25T18:47:24.689815+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:29.031294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.763135+0000","last_change":"2024-04-25T18:43:16.261161+0000","last_active":"2024-04-25T18:47:24.763135+0000","last_peered":"2024-04-25T18:47:24.763135+0000","last_clean":"2024-04-25T18:47:24.763135+0000","last_became_active":"2024-04-25T18:43:16.258217+0000","last_became_peered":"2024-04-25T18:43:16.258217+0000","last_unstale":"2024-04-25T18:47:24.763135+0000","last_undegraded":"2024-04-25T18:47:24.763135+0000","last_fullsized":"2024-04-25T18:47:24.763135+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:50.564609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973806+0000","last_change":"2024-04-25T18:43:16.266660+0000","last_active":"2024-04-25T18:47:24.973806+0000","last_peered":"2024-04-25T18:47:24.973806+0000","last_clean":"2024-04-25T18:47:24.973806+0000","last_became_active":"2024-04-25T18:43:16.266295+0000","last_became_peered":"2024-04-25T18:43:16.266295+0000","last_unstale":"2024-04-25T18:47:24.973806+0000","last_undegraded":"2024-04-25T18:47:24.973806+0000","last_fullsized":"2024-04-25T18:47:24.973806+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:42.406585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.763304+0000","last_change":"2024-04-25T18:43:12.200172+0000","last_active":"2024-04-25T18:47:24.763304+0000","last_peered":"2024-04-25T18:47:24.763304+0000","last_clean":"2024-04-25T18:47:24.763304+0000","last_became_active":"2024-04-25T18:43:12.199760+0000","last_became_peered":"2024-04-25T18:43:12.199760+0000","last_unstale":"2024-04-25T18:47:24.763304+0000","last_undegraded":"2024-04-25T18:47:24.763304+0000","last_fullsized":"2024-04-25T18:47:24.763304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:57.460274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768028+0000","last_change":"2024-04-25T18:43:16.255989+0000","last_active":"2024-04-25T18:47:24.768028+0000","last_peered":"2024-04-25T18:47:24.768028+0000","last_clean":"2024-04-25T18:47:24.768028+0000","last_became_active":"2024-04-25T18:43:16.255114+0000","last_became_peered":"2024-04-25T18:43:16.255114+0000","last_unstale":"2024-04-25T18:47:24.768028+0000","last_undegraded":"2024-04-25T18:47:24.768028+0000","last_fullsized":"2024-04-25T18:47:24.768028+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:52.888876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768305+0000","last_change":"2024-04-25T18:43:12.208194+0000","last_active":"2024-04-25T18:47:24.768305+0000","last_peered":"2024-04-25T18:47:24.768305+0000","last_clean":"2024-04-25T18:47:24.768305+0000","last_became_active":"2024-04-25T18:43:12.207801+0000","last_became_peered":"2024-04-25T18:43:12.207801+0000","last_unstale":"2024-04-25T18:47:24.768305+0000","last_undegraded":"2024-04-25T18:47:24.768305+0000","last_fullsized":"2024-04-25T18:47:24.768305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:24.742486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981924+0000","last_change":"2024-04-25T18:43:16.268960+0000","last_active":"2024-04-25T18:47:24.981924+0000","last_peered":"2024-04-25T18:47:24.981924+0000","last_clean":"2024-04-25T18:47:24.981924+0000","last_became_active":"2024-04-25T18:43:16.263626+0000","last_became_peered":"2024-04-25T18:43:16.263626+0000","last_unstale":"2024-04-25T18:47:24.981924+0000","last_undegraded":"2024-04-25T18:47:24.981924+0000","last_fullsized":"2024-04-25T18:47:24.981924+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:57.721918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629378+0000","last_change":"2024-04-25T18:43:57.110208+0000","last_active":"2024-04-25T18:47:24.629378+0000","last_peered":"2024-04-25T18:47:24.629378+0000","last_clean":"2024-04-25T18:47:24.629378+0000","last_became_active":"2024-04-25T18:43:57.106027+0000","last_became_peered":"2024-04-25T18:43:57.106027+0000","last_unstale":"2024-04-25T18:47:24.629378+0000","last_undegraded":"2024-04-25T18:47:24.629378+0000","last_fullsized":"2024-04-25T18:47:24.629378+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:20.819961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973596+0000","last_change":"2024-04-25T18:43:16.250414+0000","last_active":"2024-04-25T18:47:24.973596+0000","last_peered":"2024-04-25T18:47:24.973596+0000","last_clean":"2024-04-25T18:47:24.973596+0000","last_became_active":"2024-04-25T18:43:16.249832+0000","last_became_peered":"2024-04-25T18:43:16.249832+0000","last_unstale":"2024-04-25T18:47:24.973596+0000","last_undegraded":"2024-04-25T18:47:24.973596+0000","last_fullsized":"2024-04-25T18:47:24.973596+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:04.577854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629313+0000","last_change":"2024-04-25T18:43:12.195099+0000","last_active":"2024-04-25T18:47:24.629313+0000","last_peered":"2024-04-25T18:47:24.629313+0000","last_clean":"2024-04-25T18:47:24.629313+0000","last_became_active":"2024-04-25T18:43:12.194726+0000","last_became_peered":"2024-04-25T18:43:12.194726+0000","last_unstale":"2024-04-25T18:47:24.629313+0000","last_undegraded":"2024-04-25T18:47:24.629313+0000","last_fullsized":"2024-04-25T18:47:24.629313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.866905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644755+0000","last_change":"2024-04-25T18:43:16.245208+0000","last_active":"2024-04-25T18:47:24.644755+0000","last_peered":"2024-04-25T18:47:24.644755+0000","last_clean":"2024-04-25T18:47:24.644755+0000","last_became_active":"2024-04-25T18:43:16.244836+0000","last_became_peered":"2024-04-25T18:43:16.244836+0000","last_unstale":"2024-04-25T18:47:24.644755+0000","last_undegraded":"2024-04-25T18:47:24.644755+0000","last_fullsized":"2024-04-25T18:47:24.644755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:35.917060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":277,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:34.661143+0000","last_change":"2024-04-25T18:43:12.198252+0000","last_active":"2024-04-25T18:47:34.661143+0000","last_peered":"2024-04-25T18:47:34.661143+0000","last_clean":"2024-04-25T18:47:34.661143+0000","last_became_active":"2024-04-25T18:43:12.197691+0000","last_became_peered":"2024-04-25T18:43:12.197691+0000","last_unstale":"2024-04-25T18:47:34.661143+0000","last_undegraded":"2024-04-25T18:47:34.661143+0000","last_fullsized":"2024-04-25T18:47:34.661143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.900012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628891+0000","last_change":"2024-04-25T18:43:16.255576+0000","last_active":"2024-04-25T18:47:24.628891+0000","last_peered":"2024-04-25T18:47:24.628891+0000","last_clean":"2024-04-25T18:47:24.628891+0000","last_became_active":"2024-04-25T18:43:16.255195+0000","last_became_peered":"2024-04-25T18:43:16.255195+0000","last_unstale":"2024-04-25T18:47:24.628891+0000","last_undegraded":"2024-04-25T18:47:24.628891+0000","last_fullsized":"2024-04-25T18:47:24.628891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:41.510872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641395+0000","last_change":"2024-04-25T18:43:12.200786+0000","last_active":"2024-04-25T18:47:24.641395+0000","last_peered":"2024-04-25T18:47:24.641395+0000","last_clean":"2024-04-25T18:47:24.641395+0000","last_became_active":"2024-04-25T18:43:12.200409+0000","last_became_peered":"2024-04-25T18:43:12.200409+0000","last_unstale":"2024-04-25T18:47:24.641395+0000","last_undegraded":"2024-04-25T18:47:24.641395+0000","last_fullsized":"2024-04-25T18:47:24.641395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:52.177787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644567+0000","last_change":"2024-04-25T18:43:16.255563+0000","last_active":"2024-04-25T18:47:24.644567+0000","last_peered":"2024-04-25T18:47:24.644567+0000","last_clean":"2024-04-25T18:47:24.644567+0000","last_became_active":"2024-04-25T18:43:16.255174+0000","last_became_peered":"2024-04-25T18:43:16.255174+0000","last_unstale":"2024-04-25T18:47:24.644567+0000","last_undegraded":"2024-04-25T18:47:24.644567+0000","last_fullsized":"2024-04-25T18:47:24.644567+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:32:15.735051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631536+0000","last_change":"2024-04-25T18:43:12.210242+0000","last_active":"2024-04-25T18:47:24.631536+0000","last_peered":"2024-04-25T18:47:24.631536+0000","last_clean":"2024-04-25T18:47:24.631536+0000","last_became_active":"2024-04-25T18:43:12.209600+0000","last_became_peered":"2024-04-25T18:43:12.209600+0000","last_unstale":"2024-04-25T18:47:24.631536+0000","last_undegraded":"2024-04-25T18:47:24.631536+0000","last_fullsized":"2024-04-25T18:47:24.631536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:21.886712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.762109+0000","last_change":"2024-04-25T18:43:56.649102+0000","last_active":"2024-04-25T18:47:24.762109+0000","last_peered":"2024-04-25T18:47:24.762109+0000","last_clean":"2024-04-25T18:47:24.762109+0000","last_became_active":"2024-04-25T18:43:56.647127+0000","last_became_peered":"2024-04-25T18:43:56.647127+0000","last_unstale":"2024-04-25T18:47:24.762109+0000","last_undegraded":"2024-04-25T18:47:24.762109+0000","last_fullsized":"2024-04-25T18:47:24.762109+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:43.111539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644363+0000","last_change":"2024-04-25T18:43:12.198957+0000","last_active":"2024-04-25T18:47:24.644363+0000","last_peered":"2024-04-25T18:47:24.644363+0000","last_clean":"2024-04-25T18:47:24.644363+0000","last_became_active":"2024-04-25T18:43:12.198605+0000","last_became_peered":"2024-04-25T18:43:12.198605+0000","last_unstale":"2024-04-25T18:47:24.644363+0000","last_undegraded":"2024-04-25T18:47:24.644363+0000","last_fullsized":"2024-04-25T18:47:24.644363+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:11.759430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.762246+0000","last_change":"2024-04-25T18:43:16.266829+0000","last_active":"2024-04-25T18:47:24.762246+0000","last_peered":"2024-04-25T18:47:24.762246+0000","last_clean":"2024-04-25T18:47:24.762246+0000","last_became_active":"2024-04-25T18:43:16.266256+0000","last_became_peered":"2024-04-25T18:43:16.266256+0000","last_unstale":"2024-04-25T18:47:24.762246+0000","last_undegraded":"2024-04-25T18:47:24.762246+0000","last_fullsized":"2024-04-25T18:47:24.762246+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:43.343613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":269,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:34.660385+0000","last_change":"2024-04-25T18:43:12.201641+0000","last_active":"2024-04-25T18:47:34.660385+0000","last_peered":"2024-04-25T18:47:34.660385+0000","last_clean":"2024-04-25T18:47:34.660385+0000","last_became_active":"2024-04-25T18:43:12.201274+0000","last_became_peered":"2024-04-25T18:43:12.201274+0000","last_unstale":"2024-04-25T18:47:34.660385+0000","last_undegraded":"2024-04-25T18:47:34.660385+0000","last_fullsized":"2024-04-25T18:47:34.660385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:47.683850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.974010+0000","last_change":"2024-04-25T18:43:16.259551+0000","last_active":"2024-04-25T18:47:24.974010+0000","last_peered":"2024-04-25T18:47:24.974010+0000","last_clean":"2024-04-25T18:47:24.974010+0000","last_became_active":"2024-04-25T18:43:16.259165+0000","last_became_peered":"2024-04-25T18:43:16.259165+0000","last_unstale":"2024-04-25T18:47:24.974010+0000","last_undegraded":"2024-04-25T18:47:24.974010+0000","last_fullsized":"2024-04-25T18:47:24.974010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:23.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804701+0000","last_change":"2024-04-25T18:43:12.197534+0000","last_active":"2024-04-25T18:47:24.804701+0000","last_peered":"2024-04-25T18:47:24.804701+0000","last_clean":"2024-04-25T18:47:24.804701+0000","last_became_active":"2024-04-25T18:43:12.197106+0000","last_became_peered":"2024-04-25T18:43:12.197106+0000","last_unstale":"2024-04-25T18:47:24.804701+0000","last_undegraded":"2024-04-25T18:47:24.804701+0000","last_fullsized":"2024-04-25T18:47:24.804701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:35.399044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981722+0000","last_change":"2024-04-25T18:43:16.257668+0000","last_active":"2024-04-25T18:47:24.981722+0000","last_peered":"2024-04-25T18:47:24.981722+0000","last_clean":"2024-04-25T18:47:24.981722+0000","last_became_active":"2024-04-25T18:43:16.253609+0000","last_became_peered":"2024-04-25T18:43:16.253609+0000","last_unstale":"2024-04-25T18:47:24.981722+0000","last_undegraded":"2024-04-25T18:47:24.981722+0000","last_fullsized":"2024-04-25T18:47:24.981722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:36.048386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644803+0000","last_change":"2024-04-25T18:43:12.207322+0000","last_active":"2024-04-25T18:47:24.644803+0000","last_peered":"2024-04-25T18:47:24.644803+0000","last_clean":"2024-04-25T18:47:24.644803+0000","last_became_active":"2024-04-25T18:43:12.206947+0000","last_became_peered":"2024-04-25T18:43:12.206947+0000","last_unstale":"2024-04-25T18:47:24.644803+0000","last_undegraded":"2024-04-25T18:47:24.644803+0000","last_fullsized":"2024-04-25T18:47:24.644803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:20.536598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973667+0000","last_change":"2024-04-25T18:43:16.239575+0000","last_active":"2024-04-25T18:47:24.973667+0000","last_peered":"2024-04-25T18:47:24.973667+0000","last_clean":"2024-04-25T18:47:24.973667+0000","last_became_active":"2024-04-25T18:43:16.239208+0000","last_became_peered":"2024-04-25T18:43:16.239208+0000","last_unstale":"2024-04-25T18:47:24.973667+0000","last_undegraded":"2024-04-25T18:47:24.973667+0000","last_fullsized":"2024-04-25T18:47:24.973667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:58.195855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":551,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:34.757272+0000","last_change":"2024-04-25T18:43:12.197199+0000","last_active":"2024-04-25T18:47:34.757272+0000","last_peered":"2024-04-25T18:47:34.757272+0000","last_clean":"2024-04-25T18:47:34.757272+0000","last_became_active":"2024-04-25T18:43:12.196776+0000","last_became_peered":"2024-04-25T18:43:12.196776+0000","last_unstale":"2024-04-25T18:47:34.757272+0000","last_undegraded":"2024-04-25T18:47:34.757272+0000","last_fullsized":"2024-04-25T18:47:34.757272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:54.149782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":74349,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":481,"num_read_kb":313,"num_write":42,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629197+0000","last_change":"2024-04-25T18:43:16.250993+0000","last_active":"2024-04-25T18:47:24.629197+0000","last_peered":"2024-04-25T18:47:24.629197+0000","last_clean":"2024-04-25T18:47:24.629197+0000","last_became_active":"2024-04-25T18:43:16.250216+0000","last_became_peered":"2024-04-25T18:43:16.250216+0000","last_unstale":"2024-04-25T18:47:24.629197+0000","last_undegraded":"2024-04-25T18:47:24.629197+0000","last_fullsized":"2024-04-25T18:47:24.629197+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:37.996463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973654+0000","last_change":"2024-04-25T18:43:12.204606+0000","last_active":"2024-04-25T18:47:24.973654+0000","last_peered":"2024-04-25T18:47:24.973654+0000","last_clean":"2024-04-25T18:47:24.973654+0000","last_became_active":"2024-04-25T18:43:12.204306+0000","last_became_peered":"2024-04-25T18:43:12.204306+0000","last_unstale":"2024-04-25T18:47:24.973654+0000","last_undegraded":"2024-04-25T18:47:24.973654+0000","last_fullsized":"2024-04-25T18:47:24.973654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:29.903775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.632833+0000","last_change":"2024-04-25T18:43:16.250093+0000","last_active":"2024-04-25T18:47:24.632833+0000","last_peered":"2024-04-25T18:47:24.632833+0000","last_clean":"2024-04-25T18:47:24.632833+0000","last_became_active":"2024-04-25T18:43:16.249567+0000","last_became_peered":"2024-04-25T18:43:16.249567+0000","last_unstale":"2024-04-25T18:47:24.632833+0000","last_undegraded":"2024-04-25T18:47:24.632833+0000","last_fullsized":"2024-04-25T18:47:24.632833+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:35.498641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689370+0000","last_change":"2024-04-25T18:43:56.652488+0000","last_active":"2024-04-25T18:47:24.689370+0000","last_peered":"2024-04-25T18:47:24.689370+0000","last_clean":"2024-04-25T18:47:24.689370+0000","last_became_active":"2024-04-25T18:43:56.647745+0000","last_became_peered":"2024-04-25T18:43:56.647745+0000","last_unstale":"2024-04-25T18:47:24.689370+0000","last_undegraded":"2024-04-25T18:47:24.689370+0000","last_fullsized":"2024-04-25T18:47:24.689370+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:33.474470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804304+0000","last_change":"2024-04-25T18:43:56.642673+0000","last_active":"2024-04-25T18:47:24.804304+0000","last_peered":"2024-04-25T18:47:24.804304+0000","last_clean":"2024-04-25T18:47:24.804304+0000","last_became_active":"2024-04-25T18:43:56.642313+0000","last_became_peered":"2024-04-25T18:43:56.642313+0000","last_unstale":"2024-04-25T18:47:24.804304+0000","last_undegraded":"2024-04-25T18:47:24.804304+0000","last_fullsized":"2024-04-25T18:47:24.804304+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:45.805738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'4","reported_seq":270,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:33.700953+0000","last_change":"2024-04-25T18:43:12.197341+0000","last_active":"2024-04-25T18:47:33.700953+0000","last_peered":"2024-04-25T18:47:33.700953+0000","last_clean":"2024-04-25T18:47:33.700953+0000","last_became_active":"2024-04-25T18:43:12.196952+0000","last_became_peered":"2024-04-25T18:43:12.196952+0000","last_unstale":"2024-04-25T18:47:33.700953+0000","last_undegraded":"2024-04-25T18:47:33.700953+0000","last_fullsized":"2024-04-25T18:47:33.700953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:52.902579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":238,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804357+0000","last_change":"2024-04-25T18:43:16.245436+0000","last_active":"2024-04-25T18:47:24.804357+0000","last_peered":"2024-04-25T18:47:24.804357+0000","last_clean":"2024-04-25T18:47:24.804357+0000","last_became_active":"2024-04-25T18:43:16.244943+0000","last_became_peered":"2024-04-25T18:43:16.244943+0000","last_unstale":"2024-04-25T18:47:24.804357+0000","last_undegraded":"2024-04-25T18:47:24.804357+0000","last_fullsized":"2024-04-25T18:47:24.804357+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:03.402999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628839+0000","last_change":"2024-04-25T18:43:12.197642+0000","last_active":"2024-04-25T18:47:24.628839+0000","last_peered":"2024-04-25T18:47:24.628839+0000","last_clean":"2024-04-25T18:47:24.628839+0000","last_became_active":"2024-04-25T18:43:12.196395+0000","last_became_peered":"2024-04-25T18:43:12.196395+0000","last_unstale":"2024-04-25T18:47:24.628839+0000","last_undegraded":"2024-04-25T18:47:24.628839+0000","last_fullsized":"2024-04-25T18:47:24.628839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:11.783422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644371+0000","last_change":"2024-04-25T18:43:16.245344+0000","last_active":"2024-04-25T18:47:24.644371+0000","last_peered":"2024-04-25T18:47:24.644371+0000","last_clean":"2024-04-25T18:47:24.644371+0000","last_became_active":"2024-04-25T18:43:16.244984+0000","last_became_peered":"2024-04-25T18:43:16.244984+0000","last_unstale":"2024-04-25T18:47:24.644371+0000","last_undegraded":"2024-04-25T18:47:24.644371+0000","last_fullsized":"2024-04-25T18:47:24.644371+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:04.822629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629525+0000","last_change":"2024-04-25T18:43:12.199685+0000","last_active":"2024-04-25T18:47:24.629525+0000","last_peered":"2024-04-25T18:47:24.629525+0000","last_clean":"2024-04-25T18:47:24.629525+0000","last_became_active":"2024-04-25T18:43:12.199261+0000","last_became_peered":"2024-04-25T18:43:12.199261+0000","last_unstale":"2024-04-25T18:47:24.629525+0000","last_undegraded":"2024-04-25T18:47:24.629525+0000","last_fullsized":"2024-04-25T18:47:24.629525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:09.784616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689634+0000","last_change":"2024-04-25T18:43:16.247745+0000","last_active":"2024-04-25T18:47:24.689634+0000","last_peered":"2024-04-25T18:47:24.689634+0000","last_clean":"2024-04-25T18:47:24.689634+0000","last_became_active":"2024-04-25T18:43:16.247311+0000","last_became_peered":"2024-04-25T18:43:16.247311+0000","last_unstale":"2024-04-25T18:47:24.689634+0000","last_undegraded":"2024-04-25T18:47:24.689634+0000","last_fullsized":"2024-04-25T18:47:24.689634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:29.925464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.763129+0000","last_change":"2024-04-25T18:43:12.210209+0000","last_active":"2024-04-25T18:47:24.763129+0000","last_peered":"2024-04-25T18:47:24.763129+0000","last_clean":"2024-04-25T18:47:24.763129+0000","last_became_active":"2024-04-25T18:43:12.209708+0000","last_became_peered":"2024-04-25T18:43:12.209708+0000","last_unstale":"2024-04-25T18:47:24.763129+0000","last_undegraded":"2024-04-25T18:47:24.763129+0000","last_fullsized":"2024-04-25T18:47:24.763129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:07:18.331518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'5","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641664+0000","last_change":"2024-04-25T18:43:16.263616+0000","last_active":"2024-04-25T18:47:24.641664+0000","last_peered":"2024-04-25T18:47:24.641664+0000","last_clean":"2024-04-25T18:47:24.641664+0000","last_became_active":"2024-04-25T18:43:16.263287+0000","last_became_peered":"2024-04-25T18:43:16.263287+0000","last_unstale":"2024-04-25T18:47:24.641664+0000","last_undegraded":"2024-04-25T18:47:24.641664+0000","last_fullsized":"2024-04-25T18:47:24.641664+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:46.366457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804594+0000","last_change":"2024-04-25T18:43:12.203705+0000","last_active":"2024-04-25T18:47:24.804594+0000","last_peered":"2024-04-25T18:47:24.804594+0000","last_clean":"2024-04-25T18:47:24.804594+0000","last_became_active":"2024-04-25T18:43:12.203371+0000","last_became_peered":"2024-04-25T18:43:12.203371+0000","last_unstale":"2024-04-25T18:47:24.804594+0000","last_undegraded":"2024-04-25T18:47:24.804594+0000","last_fullsized":"2024-04-25T18:47:24.804594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:07.586581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.641542+0000","last_change":"2024-04-25T18:43:56.647597+0000","last_active":"2024-04-25T18:47:24.641542+0000","last_peered":"2024-04-25T18:47:24.641542+0000","last_clean":"2024-04-25T18:47:24.641542+0000","last_became_active":"2024-04-25T18:43:56.647121+0000","last_became_peered":"2024-04-25T18:43:56.647121+0000","last_unstale":"2024-04-25T18:47:24.641542+0000","last_undegraded":"2024-04-25T18:47:24.641542+0000","last_fullsized":"2024-04-25T18:47:24.641542+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:07.143806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644686+0000","last_change":"2024-04-25T18:43:16.247133+0000","last_active":"2024-04-25T18:47:24.644686+0000","last_peered":"2024-04-25T18:47:24.644686+0000","last_clean":"2024-04-25T18:47:24.644686+0000","last_became_active":"2024-04-25T18:43:16.246232+0000","last_became_peered":"2024-04-25T18:43:16.246232+0000","last_unstale":"2024-04-25T18:47:24.644686+0000","last_undegraded":"2024-04-25T18:47:24.644686+0000","last_fullsized":"2024-04-25T18:47:24.644686+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.288404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'5","reported_seq":269,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:33.700114+0000","last_change":"2024-04-25T18:43:12.209908+0000","last_active":"2024-04-25T18:47:33.700114+0000","last_peered":"2024-04-25T18:47:33.700114+0000","last_clean":"2024-04-25T18:47:33.700114+0000","last_became_active":"2024-04-25T18:43:12.209042+0000","last_became_peered":"2024-04-25T18:43:12.209042+0000","last_unstale":"2024-04-25T18:47:33.700114+0000","last_undegraded":"2024-04-25T18:47:33.700114+0000","last_fullsized":"2024-04-25T18:47:33.700114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:36.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.767262+0000","last_change":"2024-04-25T18:43:16.242364+0000","last_active":"2024-04-25T18:47:24.767262+0000","last_peered":"2024-04-25T18:47:24.767262+0000","last_clean":"2024-04-25T18:47:24.767262+0000","last_became_active":"2024-04-25T18:43:16.242019+0000","last_became_peered":"2024-04-25T18:43:16.242019+0000","last_unstale":"2024-04-25T18:47:24.767262+0000","last_undegraded":"2024-04-25T18:47:24.767262+0000","last_fullsized":"2024-04-25T18:47:24.767262+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:13.238129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631512+0000","last_change":"2024-04-25T18:43:12.206606+0000","last_active":"2024-04-25T18:47:24.631512+0000","last_peered":"2024-04-25T18:47:24.631512+0000","last_clean":"2024-04-25T18:47:24.631512+0000","last_became_active":"2024-04-25T18:43:12.206238+0000","last_became_peered":"2024-04-25T18:43:12.206238+0000","last_unstale":"2024-04-25T18:47:24.631512+0000","last_undegraded":"2024-04-25T18:47:24.631512+0000","last_fullsized":"2024-04-25T18:47:24.631512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:07.781590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"95'148","reported_seq":410,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:26.771988+0000","last_change":"2024-04-25T18:41:40.527363+0000","last_active":"2024-04-25T18:47:26.771988+0000","last_peered":"2024-04-25T18:47:26.771988+0000","last_clean":"2024-04-25T18:47:26.771988+0000","last_became_active":"2024-04-25T18:41:40.526907+0000","last_became_peered":"2024-04-25T18:41:40.526907+0000","last_unstale":"2024-04-25T18:47:26.771988+0000","last_undegraded":"2024-04-25T18:47:26.771988+0000","last_fullsized":"2024-04-25T18:47:26.771988+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:37:45.963565+0000","last_clean_scrub_stamp":"2024-04-25T18:37:45.963565+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:16.689693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.630889+0000","last_change":"2024-04-25T18:43:16.247552+0000","last_active":"2024-04-25T18:47:24.630889+0000","last_peered":"2024-04-25T18:47:24.630889+0000","last_clean":"2024-04-25T18:47:24.630889+0000","last_became_active":"2024-04-25T18:43:16.247178+0000","last_became_peered":"2024-04-25T18:43:16.247178+0000","last_unstale":"2024-04-25T18:47:24.630889+0000","last_undegraded":"2024-04-25T18:47:24.630889+0000","last_fullsized":"2024-04-25T18:47:24.630889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:50.072986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804356+0000","last_change":"2024-04-25T18:43:56.636197+0000","last_active":"2024-04-25T18:47:24.804356+0000","last_peered":"2024-04-25T18:47:24.804356+0000","last_clean":"2024-04-25T18:47:24.804356+0000","last_became_active":"2024-04-25T18:43:56.633431+0000","last_became_peered":"2024-04-25T18:43:56.633431+0000","last_unstale":"2024-04-25T18:47:24.804356+0000","last_undegraded":"2024-04-25T18:47:24.804356+0000","last_fullsized":"2024-04-25T18:47:24.804356+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:12.463489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.629270+0000","last_change":"2024-04-25T18:43:12.202101+0000","last_active":"2024-04-25T18:47:24.629270+0000","last_peered":"2024-04-25T18:47:24.629270+0000","last_clean":"2024-04-25T18:47:24.629270+0000","last_became_active":"2024-04-25T18:43:12.201727+0000","last_became_peered":"2024-04-25T18:43:12.201727+0000","last_unstale":"2024-04-25T18:47:24.629270+0000","last_undegraded":"2024-04-25T18:47:24.629270+0000","last_fullsized":"2024-04-25T18:47:24.629270+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:33.132072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631089+0000","last_change":"2024-04-25T18:43:16.255433+0000","last_active":"2024-04-25T18:47:24.631089+0000","last_peered":"2024-04-25T18:47:24.631089+0000","last_clean":"2024-04-25T18:47:24.631089+0000","last_became_active":"2024-04-25T18:43:16.254866+0000","last_became_peered":"2024-04-25T18:43:16.254866+0000","last_unstale":"2024-04-25T18:47:24.631089+0000","last_undegraded":"2024-04-25T18:47:24.631089+0000","last_fullsized":"2024-04-25T18:47:24.631089+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:07.876973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"95'102","reported_seq":236,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:28.169230+0000","last_change":"2024-04-25T18:43:56.656998+0000","last_active":"2024-04-25T18:47:28.169230+0000","last_peered":"2024-04-25T18:47:28.169230+0000","last_clean":"2024-04-25T18:47:28.169230+0000","last_became_active":"2024-04-25T18:43:56.647456+0000","last_became_peered":"2024-04-25T18:43:56.647456+0000","last_unstale":"2024-04-25T18:47:28.169230+0000","last_undegraded":"2024-04-25T18:47:28.169230+0000","last_fullsized":"2024-04-25T18:47:28.169230+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:00.719426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768597+0000","last_change":"2024-04-25T18:43:16.252826+0000","last_active":"2024-04-25T18:47:24.768597+0000","last_peered":"2024-04-25T18:47:24.768597+0000","last_clean":"2024-04-25T18:47:24.768597+0000","last_became_active":"2024-04-25T18:43:16.252435+0000","last_became_peered":"2024-04-25T18:43:16.252435+0000","last_unstale":"2024-04-25T18:47:24.768597+0000","last_undegraded":"2024-04-25T18:47:24.768597+0000","last_fullsized":"2024-04-25T18:47:24.768597+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:13.401006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631915+0000","last_change":"2024-04-25T18:43:12.376121+0000","last_active":"2024-04-25T18:47:24.631915+0000","last_peered":"2024-04-25T18:47:24.631915+0000","last_clean":"2024-04-25T18:47:24.631915+0000","last_became_active":"2024-04-25T18:43:12.375757+0000","last_became_peered":"2024-04-25T18:43:12.375757+0000","last_unstale":"2024-04-25T18:47:24.631915+0000","last_undegraded":"2024-04-25T18:47:24.631915+0000","last_fullsized":"2024-04-25T18:47:24.631915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:15.610442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981907+0000","last_change":"2024-04-25T18:43:16.266504+0000","last_active":"2024-04-25T18:47:24.981907+0000","last_peered":"2024-04-25T18:47:24.981907+0000","last_clean":"2024-04-25T18:47:24.981907+0000","last_became_active":"2024-04-25T18:43:16.265048+0000","last_became_peered":"2024-04-25T18:43:16.265048+0000","last_unstale":"2024-04-25T18:47:24.981907+0000","last_undegraded":"2024-04-25T18:47:24.981907+0000","last_fullsized":"2024-04-25T18:47:24.981907+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:36.924872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973593+0000","last_change":"2024-04-25T18:43:12.207529+0000","last_active":"2024-04-25T18:47:24.973593+0000","last_peered":"2024-04-25T18:47:24.973593+0000","last_clean":"2024-04-25T18:47:24.973593+0000","last_became_active":"2024-04-25T18:43:12.207122+0000","last_became_peered":"2024-04-25T18:43:12.207122+0000","last_unstale":"2024-04-25T18:47:24.973593+0000","last_undegraded":"2024-04-25T18:47:24.973593+0000","last_fullsized":"2024-04-25T18:47:24.973593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:58:44.518497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628853+0000","last_change":"2024-04-25T18:43:16.249616+0000","last_active":"2024-04-25T18:47:24.628853+0000","last_peered":"2024-04-25T18:47:24.628853+0000","last_clean":"2024-04-25T18:47:24.628853+0000","last_became_active":"2024-04-25T18:43:16.249252+0000","last_became_peered":"2024-04-25T18:43:16.249252+0000","last_unstale":"2024-04-25T18:47:24.628853+0000","last_undegraded":"2024-04-25T18:47:24.628853+0000","last_fullsized":"2024-04-25T18:47:24.628853+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:34.721513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.766937+0000","last_change":"2024-04-25T18:43:12.209304+0000","last_active":"2024-04-25T18:47:24.766937+0000","last_peered":"2024-04-25T18:47:24.766937+0000","last_clean":"2024-04-25T18:47:24.766937+0000","last_became_active":"2024-04-25T18:43:12.208887+0000","last_became_peered":"2024-04-25T18:43:12.208887+0000","last_unstale":"2024-04-25T18:47:24.766937+0000","last_undegraded":"2024-04-25T18:47:24.766937+0000","last_fullsized":"2024-04-25T18:47:24.766937+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:22.049809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689380+0000","last_change":"2024-04-25T18:43:16.262482+0000","last_active":"2024-04-25T18:47:24.689380+0000","last_peered":"2024-04-25T18:47:24.689380+0000","last_clean":"2024-04-25T18:47:24.689380+0000","last_became_active":"2024-04-25T18:43:16.262108+0000","last_became_peered":"2024-04-25T18:43:16.262108+0000","last_unstale":"2024-04-25T18:47:24.689380+0000","last_undegraded":"2024-04-25T18:47:24.689380+0000","last_fullsized":"2024-04-25T18:47:24.689380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:14.777939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:25.252200+0000","last_change":"2024-04-25T18:43:12.205742+0000","last_active":"2024-04-25T18:47:25.252200+0000","last_peered":"2024-04-25T18:47:25.252200+0000","last_clean":"2024-04-25T18:47:25.252200+0000","last_became_active":"2024-04-25T18:43:12.205314+0000","last_became_peered":"2024-04-25T18:43:12.205314+0000","last_unstale":"2024-04-25T18:47:25.252200+0000","last_undegraded":"2024-04-25T18:47:25.252200+0000","last_fullsized":"2024-04-25T18:47:25.252200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:49.641568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.767568+0000","last_change":"2024-04-25T18:43:16.266100+0000","last_active":"2024-04-25T18:47:24.767568+0000","last_peered":"2024-04-25T18:47:24.767568+0000","last_clean":"2024-04-25T18:47:24.767568+0000","last_became_active":"2024-04-25T18:43:16.263060+0000","last_became_peered":"2024-04-25T18:43:16.263060+0000","last_unstale":"2024-04-25T18:47:24.767568+0000","last_undegraded":"2024-04-25T18:47:24.767568+0000","last_fullsized":"2024-04-25T18:47:24.767568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:25:38.269244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.628994+0000","last_change":"2024-04-25T18:43:12.196054+0000","last_active":"2024-04-25T18:47:24.628994+0000","last_peered":"2024-04-25T18:47:24.628994+0000","last_clean":"2024-04-25T18:47:24.628994+0000","last_became_active":"2024-04-25T18:43:12.195503+0000","last_became_peered":"2024-04-25T18:43:12.195503+0000","last_unstale":"2024-04-25T18:47:24.628994+0000","last_undegraded":"2024-04-25T18:47:24.628994+0000","last_fullsized":"2024-04-25T18:47:24.628994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:55.187176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804598+0000","last_change":"2024-04-25T18:43:16.242536+0000","last_active":"2024-04-25T18:47:24.804598+0000","last_peered":"2024-04-25T18:47:24.804598+0000","last_clean":"2024-04-25T18:47:24.804598+0000","last_became_active":"2024-04-25T18:43:16.241060+0000","last_became_peered":"2024-04-25T18:43:16.241060+0000","last_unstale":"2024-04-25T18:47:24.804598+0000","last_undegraded":"2024-04-25T18:47:24.804598+0000","last_fullsized":"2024-04-25T18:47:24.804598+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:38.306190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631518+0000","last_change":"2024-04-25T18:43:56.653377+0000","last_active":"2024-04-25T18:47:24.631518+0000","last_peered":"2024-04-25T18:47:24.631518+0000","last_clean":"2024-04-25T18:47:24.631518+0000","last_became_active":"2024-04-25T18:43:56.645516+0000","last_became_peered":"2024-04-25T18:43:56.645516+0000","last_unstale":"2024-04-25T18:47:24.631518+0000","last_undegraded":"2024-04-25T18:47:24.631518+0000","last_fullsized":"2024-04-25T18:47:24.631518+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:11.189494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804771+0000","last_change":"2024-04-25T18:43:16.247610+0000","last_active":"2024-04-25T18:47:24.804771+0000","last_peered":"2024-04-25T18:47:24.804771+0000","last_clean":"2024-04-25T18:47:24.804771+0000","last_became_active":"2024-04-25T18:43:16.247273+0000","last_became_peered":"2024-04-25T18:43:16.247273+0000","last_unstale":"2024-04-25T18:47:24.804771+0000","last_undegraded":"2024-04-25T18:47:24.804771+0000","last_fullsized":"2024-04-25T18:47:24.804771+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:26.151480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'65","reported_seq":125,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:27.707162+0000","last_change":"2024-04-25T18:43:12.202011+0000","last_active":"2024-04-25T18:47:27.707162+0000","last_peered":"2024-04-25T18:47:27.707162+0000","last_clean":"2024-04-25T18:47:27.707162+0000","last_became_active":"2024-04-25T18:43:12.201447+0000","last_became_peered":"2024-04-25T18:43:12.201447+0000","last_unstale":"2024-04-25T18:47:27.707162+0000","last_undegraded":"2024-04-25T18:47:27.707162+0000","last_fullsized":"2024-04-25T18:47:27.707162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:48.094757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804583+0000","last_change":"2024-04-25T18:43:16.249033+0000","last_active":"2024-04-25T18:47:24.804583+0000","last_peered":"2024-04-25T18:47:24.804583+0000","last_clean":"2024-04-25T18:47:24.804583+0000","last_became_active":"2024-04-25T18:43:16.248691+0000","last_became_peered":"2024-04-25T18:43:16.248691+0000","last_unstale":"2024-04-25T18:47:24.804583+0000","last_undegraded":"2024-04-25T18:47:24.804583+0000","last_fullsized":"2024-04-25T18:47:24.804583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:37.335824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631177+0000","last_change":"2024-04-25T18:43:12.205134+0000","last_active":"2024-04-25T18:47:24.631177+0000","last_peered":"2024-04-25T18:47:24.631177+0000","last_clean":"2024-04-25T18:47:24.631177+0000","last_became_active":"2024-04-25T18:43:12.204568+0000","last_became_peered":"2024-04-25T18:43:12.204568+0000","last_unstale":"2024-04-25T18:47:24.631177+0000","last_undegraded":"2024-04-25T18:47:24.631177+0000","last_fullsized":"2024-04-25T18:47:24.631177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:11.158797+0000","last_clean_scrub_stamp":"2024-04-25T18:43:11.158797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:32.806580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.804824+0000","last_change":"2024-04-25T18:43:16.247337+0000","last_active":"2024-04-25T18:47:24.804824+0000","last_peered":"2024-04-25T18:47:24.804824+0000","last_clean":"2024-04-25T18:47:24.804824+0000","last_became_active":"2024-04-25T18:43:16.246971+0000","last_became_peered":"2024-04-25T18:43:16.246971+0000","last_unstale":"2024-04-25T18:47:24.804824+0000","last_undegraded":"2024-04-25T18:47:24.804824+0000","last_fullsized":"2024-04-25T18:47:24.804824+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.882941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689499+0000","last_change":"2024-04-25T18:43:16.252196+0000","last_active":"2024-04-25T18:47:24.689499+0000","last_peered":"2024-04-25T18:47:24.689499+0000","last_clean":"2024-04-25T18:47:24.689499+0000","last_became_active":"2024-04-25T18:43:16.251715+0000","last_became_peered":"2024-04-25T18:43:16.251715+0000","last_unstale":"2024-04-25T18:47:24.689499+0000","last_undegraded":"2024-04-25T18:47:24.689499+0000","last_fullsized":"2024-04-25T18:47:24.689499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:17.218925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973649+0000","last_change":"2024-04-25T18:43:16.247632+0000","last_active":"2024-04-25T18:47:24.973649+0000","last_peered":"2024-04-25T18:47:24.973649+0000","last_clean":"2024-04-25T18:47:24.973649+0000","last_became_active":"2024-04-25T18:43:16.247305+0000","last_became_peered":"2024-04-25T18:43:16.247305+0000","last_unstale":"2024-04-25T18:47:24.973649+0000","last_undegraded":"2024-04-25T18:47:24.973649+0000","last_fullsized":"2024-04-25T18:47:24.973649+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:52.576146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973626+0000","last_change":"2024-04-25T18:43:16.266539+0000","last_active":"2024-04-25T18:47:24.973626+0000","last_peered":"2024-04-25T18:47:24.973626+0000","last_clean":"2024-04-25T18:47:24.973626+0000","last_became_active":"2024-04-25T18:43:16.266083+0000","last_became_peered":"2024-04-25T18:43:16.266083+0000","last_unstale":"2024-04-25T18:47:24.973626+0000","last_undegraded":"2024-04-25T18:47:24.973626+0000","last_fullsized":"2024-04-25T18:47:24.973626+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:49.589019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.981404+0000","last_change":"2024-04-25T18:43:16.264002+0000","last_active":"2024-04-25T18:47:24.981404+0000","last_peered":"2024-04-25T18:47:24.981404+0000","last_clean":"2024-04-25T18:47:24.981404+0000","last_became_active":"2024-04-25T18:43:16.261555+0000","last_became_peered":"2024-04-25T18:43:16.261555+0000","last_unstale":"2024-04-25T18:47:24.981404+0000","last_undegraded":"2024-04-25T18:47:24.981404+0000","last_fullsized":"2024-04-25T18:47:24.981404+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:35:12.637040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.631289+0000","last_change":"2024-04-25T18:43:16.270914+0000","last_active":"2024-04-25T18:47:24.631289+0000","last_peered":"2024-04-25T18:47:24.631289+0000","last_clean":"2024-04-25T18:47:24.631289+0000","last_became_active":"2024-04-25T18:43:16.270117+0000","last_became_peered":"2024-04-25T18:43:16.270117+0000","last_unstale":"2024-04-25T18:47:24.631289+0000","last_undegraded":"2024-04-25T18:47:24.631289+0000","last_fullsized":"2024-04-25T18:47:24.631289+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:19.874006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689510+0000","last_change":"2024-04-25T18:43:16.247032+0000","last_active":"2024-04-25T18:47:24.689510+0000","last_peered":"2024-04-25T18:47:24.689510+0000","last_clean":"2024-04-25T18:47:24.689510+0000","last_became_active":"2024-04-25T18:43:16.246734+0000","last_became_peered":"2024-04-25T18:43:16.246734+0000","last_unstale":"2024-04-25T18:47:24.689510+0000","last_undegraded":"2024-04-25T18:47:24.689510+0000","last_fullsized":"2024-04-25T18:47:24.689510+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:39.159143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973860+0000","last_change":"2024-04-25T18:43:16.245263+0000","last_active":"2024-04-25T18:47:24.973860+0000","last_peered":"2024-04-25T18:47:24.973860+0000","last_clean":"2024-04-25T18:47:24.973860+0000","last_became_active":"2024-04-25T18:43:16.244856+0000","last_became_peered":"2024-04-25T18:43:16.244856+0000","last_unstale":"2024-04-25T18:47:24.973860+0000","last_undegraded":"2024-04-25T18:47:24.973860+0000","last_fullsized":"2024-04-25T18:47:24.973860+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:48:36.157927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.768461+0000","last_change":"2024-04-25T18:43:16.253523+0000","last_active":"2024-04-25T18:47:24.768461+0000","last_peered":"2024-04-25T18:47:24.768461+0000","last_clean":"2024-04-25T18:47:24.768461+0000","last_became_active":"2024-04-25T18:43:16.252689+0000","last_became_peered":"2024-04-25T18:43:16.252689+0000","last_unstale":"2024-04-25T18:47:24.768461+0000","last_undegraded":"2024-04-25T18:47:24.768461+0000","last_fullsized":"2024-04-25T18:47:24.768461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:31.035788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.644415+0000","last_change":"2024-04-25T18:43:16.255818+0000","last_active":"2024-04-25T18:47:24.644415+0000","last_peered":"2024-04-25T18:47:24.644415+0000","last_clean":"2024-04-25T18:47:24.644415+0000","last_became_active":"2024-04-25T18:43:16.255521+0000","last_became_peered":"2024-04-25T18:43:16.255521+0000","last_unstale":"2024-04-25T18:47:24.644415+0000","last_undegraded":"2024-04-25T18:47:24.644415+0000","last_fullsized":"2024-04-25T18:47:24.644415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:15.457585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.766729+0000","last_change":"2024-04-25T18:43:16.259680+0000","last_active":"2024-04-25T18:47:24.766729+0000","last_peered":"2024-04-25T18:47:24.766729+0000","last_clean":"2024-04-25T18:47:24.766729+0000","last_became_active":"2024-04-25T18:43:16.250065+0000","last_became_peered":"2024-04-25T18:43:16.250065+0000","last_unstale":"2024-04-25T18:47:24.766729+0000","last_undegraded":"2024-04-25T18:47:24.766729+0000","last_fullsized":"2024-04-25T18:47:24.766729+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:44.974919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.973908+0000","last_change":"2024-04-25T18:43:16.239801+0000","last_active":"2024-04-25T18:47:24.973908+0000","last_peered":"2024-04-25T18:47:24.973908+0000","last_clean":"2024-04-25T18:47:24.973908+0000","last_became_active":"2024-04-25T18:43:16.239475+0000","last_became_peered":"2024-04-25T18:43:16.239475+0000","last_unstale":"2024-04-25T18:47:24.973908+0000","last_undegraded":"2024-04-25T18:47:24.973908+0000","last_fullsized":"2024-04-25T18:47:24.973908+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:57.407476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.630762+0000","last_change":"2024-04-25T18:43:16.270419+0000","last_active":"2024-04-25T18:47:24.630762+0000","last_peered":"2024-04-25T18:47:24.630762+0000","last_clean":"2024-04-25T18:47:24.630762+0000","last_became_active":"2024-04-25T18:43:16.269995+0000","last_became_peered":"2024-04-25T18:43:16.269995+0000","last_unstale":"2024-04-25T18:47:24.630762+0000","last_undegraded":"2024-04-25T18:47:24.630762+0000","last_fullsized":"2024-04-25T18:47:24.630762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:27.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:47:24.689424+0000","last_change":"2024-04-25T18:43:16.246948+0000","last_active":"2024-04-25T18:47:24.689424+0000","last_peered":"2024-04-25T18:47:24.689424+0000","last_clean":"2024-04-25T18:47:24.689424+0000","last_became_active":"2024-04-25T18:43:16.246616+0000","last_became_peered":"2024-04-25T18:43:16.246616+0000","last_unstale":"2024-04-25T18:47:24.689424+0000","last_undegraded":"2024-04-25T18:47:24.689424+0000","last_fullsized":"2024-04-25T18:47:24.689424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:43:15.201931+0000","last_clean_scrub_stamp":"2024-04-25T18:43:15.201931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:51.817448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800721,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1785,"num_read_kb":1112,"num_write":428,"num_write_kb":2364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":582,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2402163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":314,"ondisk_log_size":314,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32580,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712572,"statfs":{"total":95995035648,"available":95961673728,"internally_reserved":0,"allocated":1437696,"data_stored":900958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514570,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37372,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707780,"statfs":{"total":95995035648,"available":95956766720,"internally_reserved":0,"allocated":2084864,"data_stored":1502195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710798,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1454080,"data_stored":901194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972437,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":507898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234075,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1100751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":507367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":581509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":508638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1142784,"data_stored":583200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117106,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37204,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707948,"statfs":{"total":95995035648,"available":95956938752,"internally_reserved":0,"allocated":1789952,"data_stored":1229126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":508777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691942,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29088,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716064,"statfs":{"total":95995035648,"available":95965249536,"internally_reserved":0,"allocated":1998848,"data_stored":1428276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":74349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":76040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":404550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:47:40.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:47:41.337 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:41.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:41 smithi161 ceph-mon[32609]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:47:41.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:41 smithi112 ceph-mon[27864]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:47:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.558+0000 7fbb68a00640 1 -- 172.21.15.112:0/1162205838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb64074c40 msgr2=0x7fbb6410e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.558+0000 7fbb68a00640 1 --2- 172.21.15.112:0/1162205838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb64074c40 0x7fbb6410e0d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009900 tx=0x7fbb4c02a480 comp rx=0 tx=0).stop 2024-04-25T18:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.558+0000 7fbb68a00640 1 -- 172.21.15.112:0/1162205838 shutdown_connections 2024-04-25T18:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.558+0000 7fbb68a00640 1 --2- 172.21.15.112:0/1162205838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6410e610 0x7fbb64110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.558+0000 7fbb68a00640 1 --2- 172.21.15.112:0/1162205838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb64074c40 0x7fbb6410e0d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.558+0000 7fbb68a00640 1 --2- 172.21.15.112:0/1162205838 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb64074320 0x7fbb64074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.558+0000 7fbb68a00640 1 -- 172.21.15.112:0/1162205838 >> 172.21.15.112:0/1162205838 conn(0x7fbb640fd8a0 msgr2=0x7fbb640ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.558+0000 7fbb68a00640 1 -- 172.21.15.112:0/1162205838 shutdown_connections 2024-04-25T18:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.558+0000 7fbb68a00640 1 -- 172.21.15.112:0/1162205838 wait complete. 2024-04-25T18:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.559+0000 7fbb68a00640 1 Processor -- start 2024-04-25T18:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb68a00640 1 -- start start 2024-04-25T18:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb68a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb64074320 0x7fbb641a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb68a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb64074c40 0x7fbb641a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb68a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb6410e610 0x7fbb6419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb68a00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64114220 con 0x7fbb64074c40 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb68a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641140a0 con 0x7fbb64074320 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb68a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641143a0 con 0x7fbb6410e610 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb5aa00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb64074c40 0x7fbb641a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb5aa00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb64074c40 0x7fbb641a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36984/0 (socket says 172.21.15.112:36984) 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb5aa00640 1 -- 172.21.15.112:0/3985878694 learned_addr learned my addr 172.21.15.112:0/3985878694 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb63e00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb6410e610 0x7fbb6419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.560+0000 7fbb63400640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb64074320 0x7fbb641a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.561+0000 7fbb5aa00640 1 -- 172.21.15.112:0/3985878694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb6410e610 msgr2=0x7fbb6419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.561+0000 7fbb5aa00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb6410e610 0x7fbb6419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.561+0000 7fbb5aa00640 1 -- 172.21.15.112:0/3985878694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb64074320 msgr2=0x7fbb641a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.561+0000 7fbb5aa00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb64074320 0x7fbb641a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.561+0000 7fbb5aa00640 1 -- 172.21.15.112:0/3985878694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6419ad80 con 0x7fbb64074c40 2024-04-25T18:47:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.561+0000 7fbb63e00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb6410e610 0x7fbb6419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.561+0000 7fbb5aa00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb64074c40 0x7fbb641a0990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c005940 tx=0x7fbb4c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.561+0000 7fbb60c00640 1 -- 172.21.15.112:0/3985878694 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c03f030 con 0x7fbb64074c40 2024-04-25T18:47:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.561+0000 7fbb60c00640 1 -- 172.21.15.112:0/3985878694 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4c0044c0 con 0x7fbb64074c40 2024-04-25T18:47:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.561+0000 7fbb68a00640 1 -- 172.21.15.112:0/3985878694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6419b010 con 0x7fbb64074c40 2024-04-25T18:47:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.561+0000 7fbb60c00640 1 -- 172.21.15.112:0/3985878694 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c035070 con 0x7fbb64074c40 2024-04-25T18:47:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.562+0000 7fbb68a00640 1 -- 172.21.15.112:0/3985878694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb641a7310 con 0x7fbb64074c40 2024-04-25T18:47:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb60c00640 1 -- 172.21.15.112:0/3985878694 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fbb4c042030 con 0x7fbb64074c40 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb6ce73740 1 Processor -- start 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb6ce73740 1 -- start start 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 0x55944caafd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 0x55944caba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cabcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55944ca15900 con 0x55944cabab90 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55944c9bbef0 con 0x55944cab02d0 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55944ca118c0 con 0x55944caaf9b0 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb63e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 0x55944caafd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb5aa00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cabcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.565+0000 7fbb63e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 0x55944caafd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:39434/0 (socket says 172.21.15.112:39434) 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.566+0000 7fbb63e00640 1 -- 172.21.15.112:0/2556158076 learned_addr learned my addr 172.21.15.112:0/2556158076 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.566+0000 7fbb63400640 1 --2- 172.21.15.112:0/2556158076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 0x55944caba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.566+0000 7fbb63400640 1 -- 172.21.15.112:0/2556158076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 msgr2=0x55944caafd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.566+0000 7fbb63400640 1 --2- 172.21.15.112:0/2556158076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 0x55944caafd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.566+0000 7fbb63400640 1 -- 172.21.15.112:0/2556158076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 msgr2=0x55944cabcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.566+0000 7fbb63400640 1 --2- 172.21.15.112:0/2556158076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cabcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.566+0000 7fbb63400640 1 -- 172.21.15.112:0/2556158076 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55944c728210 con 0x55944cab02d0 2024-04-25T18:47:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.566+0000 7fbb63e00640 1 --2- 172.21.15.112:0/2556158076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 0x55944caafd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.566+0000 7fbb63400640 1 --2- 172.21.15.112:0/2556158076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 0x55944caba650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009c20 tx=0x7fbb54011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a59978eee741b77e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.566+0000 7fbb60c00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fbb480a3e30 0x7fbb480a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.567+0000 7fbb60c00640 1 -- 172.21.15.112:0/3985878694 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fbb4c0f0ce0 con 0x7fbb64074c40 2024-04-25T18:47:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.567+0000 7fbb53e00640 1 -- 172.21.15.112:0/2556158076 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5401e070 con 0x55944cab02d0 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.567+0000 7fbb63e00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fbb480a3e30 0x7fbb480a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.567+0000 7fbb53e00640 1 -- 172.21.15.112:0/2556158076 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb54009dd0 con 0x55944cab02d0 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.567+0000 7fbb53e00640 1 -- 172.21.15.112:0/2556158076 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb540187d0 con 0x55944cab02d0 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.567+0000 7fbb6ce73740 1 -- 172.21.15.112:0/2556158076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 msgr2=0x55944caba650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.567+0000 7fbb63e00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fbb480a3e30 0x7fbb480a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009fd0 tx=0x7fbb5c009500 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.567+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/2556158076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 0x55944caba650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009c20 tx=0x7fbb54011810 comp rx=0 tx=0).stop 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.568+0000 7fbb6ce73740 1 -- 172.21.15.112:0/2556158076 shutdown_connections 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.568+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/2556158076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cabcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.568+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/2556158076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 0x55944caba650 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.568+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/2556158076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 0x55944caafd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.568+0000 7fbb6ce73740 1 -- 172.21.15.112:0/2556158076 >> 172.21.15.112:0/2556158076 conn(0x55944caabec0 msgr2=0x55944caac2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.568+0000 7fbb6ce73740 1 -- 172.21.15.112:0/2556158076 shutdown_connections 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.568+0000 7fbb6ce73740 1 -- 172.21.15.112:0/2556158076 wait complete. 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.568+0000 7fbb6ce73740 1 -- start start 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.568+0000 7fbb6ce73740 1 Processor -- start 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 0x55944cab7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 0x55944cab8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cab8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55944cabd7c0 con 0x55944cabab90 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55944c9cf310 con 0x55944cab02d0 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55944cabd940 con 0x55944caaf9b0 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb6ce73740 10 client.0 _ll_register_callbacks cb 0x55944c99dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb5aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 0x55944cab8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb63e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cab8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb63400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 0x55944cab7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb5aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 0x55944cab8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:32828/0 (socket says 172.21.15.112:32828) 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb5aa00640 1 -- 172.21.15.112:0/1090344192 learned_addr learned my addr 172.21.15.112:0/1090344192 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb63e00640 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cab8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36994/0 (socket says 172.21.15.112:36994) 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb5aa00640 1 -- 172.21.15.112:0/1090344192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 msgr2=0x55944cab7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb5aa00640 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 0x55944cab7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb5aa00640 1 -- 172.21.15.112:0/1090344192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 msgr2=0x55944cab8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.569+0000 7fbb5aa00640 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cab8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.570+0000 7fbb5aa00640 1 -- 172.21.15.112:0/1090344192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55944c7f4620 con 0x55944cab02d0 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.570+0000 7fbb5aa00640 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 0x55944cab8050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0bc8b0 tx=0x7fbb4c0f4a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=871377a601e0339 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.570+0000 7fbb50c00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.570+0000 7fbb50c00640 1 -- 172.21.15.112:0/1090344192 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c049070 con 0x55944cab02d0 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.570+0000 7fbb50c00640 1 -- 172.21.15.112:0/1090344192 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fbb4c0c52e0 con 0x55944cab02d0 2024-04-25T18:47:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.570+0000 7fbb6ce73740 1 -- 172.21.15.112:0/1090344192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb54009dd0 con 0x55944cab02d0 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.570+0000 7fbb50c00640 1 -- 172.21.15.112:0/1090344192 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbb4c0bc6f0 con 0x55944cab02d0 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.570+0000 7fbb6ce73740 10 client.25169 fetch_fsmap learned FSMap version 26 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.570+0000 7fbb6ce73740 1 -- 172.21.15.112:0/1090344192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55944cac9a00 con 0x55944cab02d0 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.570+0000 7fbb50c00640 1 -- 172.21.15.112:0/1090344192 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb54009dd0 con 0x55944cab02d0 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.571+0000 7fbb50c00640 1 -- 172.21.15.112:0/1090344192 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbb4c0ee6b0 con 0x55944cab02d0 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.571+0000 7fbb6ce73740 10 client.25169 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.571+0000 7fbb6ce73740 10 client.25169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.571+0000 7fbb6ce73740 20 client.25169 populate_metadata read hostname 'smithi112' 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.571+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55944cacaeb0 0x55944cacd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.571+0000 7fbb6ce73740 4 client.25169 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.571+0000 7fbb6ce73740 1 -- 172.21.15.112:0/1090344192 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55944ca298b0 con 0x55944cacaeb0 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.571+0000 7fbb63400640 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55944cacaeb0 0x55944cacd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.572+0000 7fbb63400640 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55944cacaeb0 0x55944cacd2c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbde75d8449efa0b server_cookie=9842abc0a12bd084 in_seq=0 out_seq=0 2024-04-25T18:47:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.572+0000 7fbb50c00640 10 client.25169 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb50c00640 1 -- 172.21.15.112:0/1090344192 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55944ca298b0 con 0x55944cacaeb0 2024-04-25T18:47:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb50c00640 10 client.25169 handle_command_reply: tid=0 2024-04-25T18:47:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 1 client.25169 shutdown 2024-04-25T18:47:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 1 -- 172.21.15.112:0/1090344192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 msgr2=0x55944cab8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 0x55944cab8050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0bc8b0 tx=0x7fbb4c0f4a30 comp rx=0 tx=0).stop 2024-04-25T18:47:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 1 -- 172.21.15.112:0/1090344192 shutdown_connections 2024-04-25T18:47:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55944cacaeb0 0x55944cacd2c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cab8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cab02d0 0x55944cab8050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/1090344192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944caaf9b0 0x55944cab7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 1 -- 172.21.15.112:0/1090344192 >> 172.21.15.112:0/1090344192 conn(0x55944caabec0 msgr2=0x55944cab1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb50c00640 0 client.25169 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 1 -- 172.21.15.112:0/1090344192 shutdown_connections 2024-04-25T18:47:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 1 -- 172.21.15.112:0/1090344192 wait complete. 2024-04-25T18:47:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.573+0000 7fbb6ce73740 20 client.25169 trim_cache size 0 max 16384 2024-04-25T18:47:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.589+0000 7fbb6ce73740 1 Processor -- start 2024-04-25T18:47:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.589+0000 7fbb6ce73740 1 -- start start 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.590+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944caaf9b0 0x55944cac6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.590+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 0x55944cad2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.590+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cad3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.590+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641b2400 con 0x55944cabab90 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.590+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641afcd0 con 0x55944caaf9b0 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.590+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641adb20 con 0x55944cab02d0 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.590+0000 7fbb63e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cad3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.590+0000 7fbb63e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cad3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36998/0 (socket says 172.21.15.112:36998) 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.590+0000 7fbb63e00640 1 -- 172.21.15.112:0/2026343986 learned_addr learned my addr 172.21.15.112:0/2026343986 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.590+0000 7fbb5aa00640 1 --2- 172.21.15.112:0/2026343986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 0x55944cad2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.590+0000 7fbb63400640 1 --2- 172.21.15.112:0/2026343986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944caaf9b0 0x55944cac6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.591+0000 7fbb63e00640 1 -- 172.21.15.112:0/2026343986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 msgr2=0x55944cad2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.591+0000 7fbb63e00640 1 --2- 172.21.15.112:0/2026343986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 0x55944cad2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.591+0000 7fbb63e00640 1 -- 172.21.15.112:0/2026343986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944caaf9b0 msgr2=0x55944cac6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.591+0000 7fbb63e00640 1 --2- 172.21.15.112:0/2026343986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944caaf9b0 0x55944cac6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.591+0000 7fbb63e00640 1 -- 172.21.15.112:0/2026343986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55944cab6060 con 0x55944cabab90 2024-04-25T18:47:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.591+0000 7fbb5aa00640 1 --2- 172.21.15.112:0/2026343986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 0x55944cad2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.591+0000 7fbb63e00640 1 --2- 172.21.15.112:0/2026343986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cad3110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c011410 tx=0x7fbb5c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4801738b8ecae69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.591+0000 7fbb53400640 1 -- 172.21.15.112:0/2026343986 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c004b20 con 0x55944cabab90 2024-04-25T18:47:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.591+0000 7fbb53400640 1 -- 172.21.15.112:0/2026343986 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb5c004cc0 con 0x55944cabab90 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.591+0000 7fbb53400640 1 -- 172.21.15.112:0/2026343986 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c005310 con 0x55944cabab90 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.592+0000 7fbb6ce73740 1 -- 172.21.15.112:0/2026343986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 msgr2=0x55944cad3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.592+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/2026343986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cad3110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c011410 tx=0x7fbb5c01a000 comp rx=0 tx=0).stop 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.592+0000 7fbb6ce73740 1 -- 172.21.15.112:0/2026343986 shutdown_connections 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.592+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/2026343986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944cabab90 0x55944cad3110 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.592+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/2026343986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 0x55944cad2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.592+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/2026343986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944caaf9b0 0x55944cac6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.592+0000 7fbb6ce73740 1 -- 172.21.15.112:0/2026343986 >> 172.21.15.112:0/2026343986 conn(0x55944caabec0 msgr2=0x55944cab2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.592+0000 7fbb6ce73740 1 -- 172.21.15.112:0/2026343986 shutdown_connections 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.592+0000 7fbb6ce73740 1 -- 172.21.15.112:0/2026343986 wait complete. 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.592+0000 7fbb6ce73740 1 -- start start 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb6ce73740 1 Processor -- start 2024-04-25T18:47:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944caaf9b0 0x55944cac2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 0x55944cac0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb6ce73740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cabab90 0x55944cac0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641ae190 con 0x55944caaf9b0 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641af600 con 0x55944cabab90 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb6ce73740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641ae860 con 0x55944cab02d0 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb6ce73740 10 client.0 _ll_register_callbacks cb 0x7fbb4c009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb63400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944caaf9b0 0x55944cac2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb5aa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 0x55944cac0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb63e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cabab90 0x55944cac0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb63400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944caaf9b0 0x55944cac2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37012/0 (socket says 172.21.15.112:37012) 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.593+0000 7fbb63400640 1 -- 172.21.15.112:0/4098015688 learned_addr learned my addr 172.21.15.112:0/4098015688 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb5aa00640 1 -- 172.21.15.112:0/4098015688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cabab90 msgr2=0x55944cac0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb5aa00640 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cabab90 0x55944cac0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb5aa00640 1 -- 172.21.15.112:0/4098015688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944caaf9b0 msgr2=0x55944cac2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb5aa00640 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944caaf9b0 0x55944cac2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb5aa00640 1 -- 172.21.15.112:0/4098015688 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55944cac1050 con 0x55944cab02d0 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb63400640 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944caaf9b0 0x55944cac2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb5aa00640 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 0x55944cac0310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0bae60 tx=0x7fbb4c0c54f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404e35829bfe87c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb50c00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb50c00640 1 -- 172.21.15.112:0/4098015688 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c049040 con 0x55944cab02d0 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb50c00640 1 -- 172.21.15.112:0/4098015688 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fbb4c0f4480 con 0x55944cab02d0 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb6ce73740 1 -- 172.21.15.112:0/4098015688 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb5c004cc0 con 0x55944cab02d0 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb50c00640 1 -- 172.21.15.112:0/4098015688 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbb4c0a2050 con 0x55944cab02d0 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb6ce73740 10 client.25180 fetch_fsmap learned FSMap version 26 2024-04-25T18:47:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.594+0000 7fbb6ce73740 1 -- 172.21.15.112:0/4098015688 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55944cac1050 con 0x55944cab02d0 2024-04-25T18:47:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.595+0000 7fbb50c00640 1 -- 172.21.15.112:0/4098015688 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb5c004cc0 con 0x55944cab02d0 2024-04-25T18:47:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.595+0000 7fbb50c00640 1 -- 172.21.15.112:0/4098015688 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbb4c0eeb80 con 0x55944cab02d0 2024-04-25T18:47:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.595+0000 7fbb6ce73740 10 client.25180 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:47:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.595+0000 7fbb6ce73740 10 client.25180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:47:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.595+0000 7fbb6ce73740 20 client.25180 populate_metadata read hostname 'smithi112' 2024-04-25T18:47:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.595+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55944cb31880 0x55944cb33ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.595+0000 7fbb6ce73740 4 client.25180 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:47:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.595+0000 7fbb6ce73740 1 -- 172.21.15.112:0/4098015688 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55944ca23320 con 0x55944cb31880 2024-04-25T18:47:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.595+0000 7fbb63400640 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55944cb31880 0x55944cb33ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.596+0000 7fbb63400640 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55944cb31880 0x55944cb33ca0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e3e599ca5e19f0f server_cookie=886f9d4d8dc47865 in_seq=0 out_seq=0 2024-04-25T18:47:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.596+0000 7fbb50c00640 10 client.25180 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb50c00640 1 -- 172.21.15.112:0/4098015688 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55944ca23320 con 0x55944cb31880 2024-04-25T18:47:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb50c00640 10 client.25180 handle_command_reply: tid=0 2024-04-25T18:47:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 1 client.25180 shutdown 2024-04-25T18:47:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 1 -- 172.21.15.112:0/4098015688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 msgr2=0x55944cac0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 0x55944cac0310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0bae60 tx=0x7fbb4c0c54f0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 1 -- 172.21.15.112:0/4098015688 shutdown_connections 2024-04-25T18:47:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55944cabab90 0x55944cac0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55944cab02d0 0x55944cac0310 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55944cb31880 0x55944cb33ca0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 1 --2- 172.21.15.112:0/4098015688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55944caaf9b0 0x55944cac2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb50c00640 0 client.25180 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 1 -- 172.21.15.112:0/4098015688 >> 172.21.15.112:0/4098015688 conn(0x55944caabec0 msgr2=0x55944cab7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 1 -- 172.21.15.112:0/4098015688 shutdown_connections 2024-04-25T18:47:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 1 -- 172.21.15.112:0/4098015688 wait complete. 2024-04-25T18:47:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.608+0000 7fbb6ce73740 20 client.25180 trim_cache size 0 max 16384 2024-04-25T18:47:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:47:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:47:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:47:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.609+0000 7fbb68a00640 1 -- 172.21.15.112:0/3985878694 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fbb480a3e30 msgr2=0x7fbb480a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.609+0000 7fbb68a00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fbb480a3e30 0x7fbb480a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009fd0 tx=0x7fbb5c009500 comp rx=0 tx=0).stop 2024-04-25T18:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.609+0000 7fbb68a00640 1 -- 172.21.15.112:0/3985878694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb64074c40 msgr2=0x7fbb641a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.609+0000 7fbb68a00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb64074c40 0x7fbb641a0990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c005940 tx=0x7fbb4c0040c0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.609+0000 7fbb68a00640 1 -- 172.21.15.112:0/3985878694 shutdown_connections 2024-04-25T18:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.609+0000 7fbb68a00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fbb480a3e30 0x7fbb480a62f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.609+0000 7fbb68a00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb6410e610 0x7fbb6419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.609+0000 7fbb68a00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb64074c40 0x7fbb641a0990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.609+0000 7fbb68a00640 1 --2- 172.21.15.112:0/3985878694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb64074320 0x7fbb641a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.609+0000 7fbb68a00640 1 -- 172.21.15.112:0/3985878694 >> 172.21.15.112:0/3985878694 conn(0x7fbb640fd8a0 msgr2=0x7fbb640fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.609+0000 7fbb68a00640 1 -- 172.21.15.112:0/3985878694 shutdown_connections 2024-04-25T18:47:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:41.610+0000 7fbb68a00640 1 -- 172.21.15.112:0/3985878694 wait complete. 2024-04-25T18:47:41.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:41 smithi171 ceph-mon[33266]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:47:42.417 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:42 smithi112 ceph-mon[27864]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:47:42.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:42 smithi171 ceph-mon[33266]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:47:42.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:42 smithi161 ceph-mon[32609]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:47:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.791+0000 7f93a9c00700 1 -- 172.21.15.112:0/1571811167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93a4102af0 msgr2=0x7f93a4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.791+0000 7f93a9c00700 1 --2- 172.21.15.112:0/1571811167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93a4102af0 0x7f93a4102f70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f938c0098f0 tx=0x7f938c009c00 comp rx=0 tx=0).stop 2024-04-25T18:47:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.791+0000 7f93a9c00700 1 -- 172.21.15.112:0/1571811167 shutdown_connections 2024-04-25T18:47:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.791+0000 7f93a9c00700 1 --2- 172.21.15.112:0/1571811167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41034b0 0x7f93a410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.791+0000 7f93a9c00700 1 --2- 172.21.15.112:0/1571811167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a4107940 0x7f93a4107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.791+0000 7f93a9c00700 1 --2- 172.21.15.112:0/1571811167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93a4102af0 0x7f93a4102f70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.791+0000 7f93a9c00700 1 -- 172.21.15.112:0/1571811167 >> 172.21.15.112:0/1571811167 conn(0x7f93a4079ba0 msgr2=0x7f93a4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.791+0000 7f93a9c00700 1 -- 172.21.15.112:0/1571811167 shutdown_connections 2024-04-25T18:47:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.791+0000 7f93a9c00700 1 -- 172.21.15.112:0/1571811167 wait complete. 2024-04-25T18:47:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.793+0000 7f93a9c00700 1 Processor -- start 2024-04-25T18:47:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.793+0000 7f93a9c00700 1 -- start start 2024-04-25T18:47:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.794+0000 7f93a9c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a4102af0 0x7f93a4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.794+0000 7f93a9c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41034b0 0x7f93a4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.794+0000 7f93a9c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93a4107940 0x7f93a40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.794+0000 7f93a9c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93a4112cb0 con 0x7f93a41034b0 2024-04-25T18:47:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.794+0000 7f93a9c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f93a4112b30 con 0x7f93a4102af0 2024-04-25T18:47:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.794+0000 7f93a9c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f93a4112e30 con 0x7f93a4107940 2024-04-25T18:47:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.794+0000 7f939a000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41034b0 0x7f93a4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.794+0000 7f93a2a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a4102af0 0x7f93a4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.794+0000 7f93a2a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a4102af0 0x7f93a4074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:32882/0 (socket says 172.21.15.112:32882) 2024-04-25T18:47:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.794+0000 7f93a2a00700 1 -- 172.21.15.112:0/1583026689 learned_addr learned my addr 172.21.15.112:0/1583026689 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.794+0000 7f93a3400700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93a4107940 0x7f93a40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.795+0000 7f939a000700 1 -- 172.21.15.112:0/1583026689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93a4107940 msgr2=0x7f93a40757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.795+0000 7f939a000700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93a4107940 0x7f93a40757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.795+0000 7f939a000700 1 -- 172.21.15.112:0/1583026689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a4102af0 msgr2=0x7f93a4074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.795+0000 7f939a000700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a4102af0 0x7f93a4074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.795+0000 7f939a000700 1 -- 172.21.15.112:0/1583026689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a41a76f0 con 0x7f93a41034b0 2024-04-25T18:47:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.795+0000 7f93a2a00700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a4102af0 0x7f93a4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.795+0000 7f939a000700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41034b0 0x7f93a4075280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f938c00c010 tx=0x7f938c00fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.796+0000 7f939be00700 1 -- 172.21.15.112:0/1583026689 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0050c0 con 0x7f93a41034b0 2024-04-25T18:47:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.796+0000 7f939be00700 1 -- 172.21.15.112:0/1583026689 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c00b560 con 0x7f93a41034b0 2024-04-25T18:47:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.796+0000 7f93a9c00700 1 -- 172.21.15.112:0/1583026689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a41a7980 con 0x7f93a41034b0 2024-04-25T18:47:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.796+0000 7f939be00700 1 -- 172.21.15.112:0/1583026689 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0039a0 con 0x7f93a41034b0 2024-04-25T18:47:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.796+0000 7f93a9c00700 1 -- 172.21.15.112:0/1583026689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a41a7e30 con 0x7f93a41034b0 2024-04-25T18:47:43.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.797+0000 7f93a9c00700 1 -- 172.21.15.112:0/1583026689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a4069730 con 0x7f93a41034b0 2024-04-25T18:47:43.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.799+0000 7f939be00700 1 -- 172.21.15.112:0/1583026689 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f938c00ba50 con 0x7f93a41034b0 2024-04-25T18:47:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.801+0000 7f939be00700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f93880a3f50 0x7f93880a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.801+0000 7f939be00700 1 -- 172.21.15.112:0/1583026689 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f938c0d1a60 con 0x7f93a41034b0 2024-04-25T18:47:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.801+0000 7f93a2a00700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f93880a3f50 0x7f93880a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.802+0000 7f93a2a00700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f93880a3f50 0x7f93880a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9394009c10 tx=0x7f939400b040 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:43.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:43.806+0000 7f939be00700 1 -- 172.21.15.112:0/1583026689 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f938c013030 con 0x7f93a41034b0 2024-04-25T18:47:44.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.080+0000 7f93a9c00700 1 -- 172.21.15.112:0/1583026689 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93a402df80 con 0x7f93a41034b0 2024-04-25T18:47:44.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.082+0000 7f939be00700 1 -- 172.21.15.112:0/1583026689 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7f938c09dee0 con 0x7f93a41034b0 2024-04-25T18:47:44.084 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:47:44.084 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:47:25.174463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24493,24511,24512]},"id":1}]} 2024-04-25T18:47:44.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.085+0000 7f93a9c00700 1 -- 172.21.15.112:0/1583026689 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f93880a3f50 msgr2=0x7f93880a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:44.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.085+0000 7f93a9c00700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f93880a3f50 0x7f93880a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9394009c10 tx=0x7f939400b040 comp rx=0 tx=0).stop 2024-04-25T18:47:44.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.085+0000 7f93a9c00700 1 -- 172.21.15.112:0/1583026689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41034b0 msgr2=0x7f93a4075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:44.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.085+0000 7f93a9c00700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41034b0 0x7f93a4075280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f938c00c010 tx=0x7f938c00fa20 comp rx=0 tx=0).stop 2024-04-25T18:47:44.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.086+0000 7f93a9c00700 1 -- 172.21.15.112:0/1583026689 shutdown_connections 2024-04-25T18:47:44.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.086+0000 7f93a9c00700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f93880a3f50 0x7f93880a6410 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:44.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.086+0000 7f93a9c00700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93a4107940 0x7f93a40757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:44.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.086+0000 7f93a9c00700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a4102af0 0x7f93a4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:44.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.086+0000 7f93a9c00700 1 --2- 172.21.15.112:0/1583026689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41034b0 0x7f93a4075280 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:44.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.086+0000 7f93a9c00700 1 -- 172.21.15.112:0/1583026689 >> 172.21.15.112:0/1583026689 conn(0x7f93a4079ba0 msgr2=0x7f93a4100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:44.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.086+0000 7f93a9c00700 1 -- 172.21.15.112:0/1583026689 shutdown_connections 2024-04-25T18:47:44.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:44.086+0000 7f93a9c00700 1 -- 172.21.15.112:0/1583026689 wait complete. 2024-04-25T18:47:44.088 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-04-25T18:47:44.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:44 smithi112 ceph-mon[27864]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-25T18:47:44.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:44 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1583026689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:47:44.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:44 smithi171 ceph-mon[33266]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-25T18:47:44.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:44 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1583026689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:47:44.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:44 smithi161 ceph-mon[32609]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-25T18:47:44.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:44 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1583026689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:47:44.985 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:47:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:46.484 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:46 smithi171 ceph-mon[33266]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:47:46.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:46 smithi112 ceph-mon[27864]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:47:46.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:46 smithi161 ceph-mon[32609]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:47:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.774+0000 7f6036200700 1 -- 172.21.15.112:0/1981164488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6030101940 msgr2=0x7f6030101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:47.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.774+0000 7f6036200700 1 --2- 172.21.15.112:0/1981164488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6030101940 0x7f6030101dc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f601c009990 tx=0x7f601c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:47:47.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.774+0000 7f6036200700 1 -- 172.21.15.112:0/1981164488 shutdown_connections 2024-04-25T18:47:47.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.774+0000 7f6036200700 1 --2- 172.21.15.112:0/1981164488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6030102300 0x7f603010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:47.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.774+0000 7f6036200700 1 --2- 172.21.15.112:0/1981164488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6030107990 0x7f6030107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:47.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.774+0000 7f6036200700 1 --2- 172.21.15.112:0/1981164488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6030101940 0x7f6030101dc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:47.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.774+0000 7f6036200700 1 -- 172.21.15.112:0/1981164488 >> 172.21.15.112:0/1981164488 conn(0x7f60300788f0 msgr2=0x7f6030078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:47.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.774+0000 7f6036200700 1 -- 172.21.15.112:0/1981164488 shutdown_connections 2024-04-25T18:47:47.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.774+0000 7f6036200700 1 -- 172.21.15.112:0/1981164488 wait complete. 2024-04-25T18:47:47.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.775+0000 7f6036200700 1 Processor -- start 2024-04-25T18:47:47.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.775+0000 7f6036200700 1 -- start start 2024-04-25T18:47:47.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.776+0000 7f6036200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6030101940 0x7f60301050e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:47.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.776+0000 7f6036200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6030102300 0x7f60301034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:47.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.776+0000 7f6036200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6030107990 0x7f6030103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:47.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.776+0000 7f6036200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6030111b50 con 0x7f6030101940 2024-04-25T18:47:47.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.776+0000 7f6036200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f60301119d0 con 0x7f6030102300 2024-04-25T18:47:47.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.776+0000 7f6036200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6030111cd0 con 0x7f6030107990 2024-04-25T18:47:47.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.776+0000 7f602f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6030101940 0x7f60301050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:47.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.776+0000 7f602f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6030101940 0x7f60301050e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37056/0 (socket says 172.21.15.112:37056) 2024-04-25T18:47:47.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.776+0000 7f602f400700 1 -- 172.21.15.112:0/2168490495 learned_addr learned my addr 172.21.15.112:0/2168490495 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:47.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.776+0000 7f602ea00700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6030102300 0x7f60301034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:47.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.776+0000 7f602fe00700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6030107990 0x7f6030103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:47.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.777+0000 7f602f400700 1 -- 172.21.15.112:0/2168490495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6030107990 msgr2=0x7f6030103b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:47.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.777+0000 7f602f400700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6030107990 0x7f6030103b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:47.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.777+0000 7f602f400700 1 -- 172.21.15.112:0/2168490495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6030102300 msgr2=0x7f60301034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:47.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.777+0000 7f602f400700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6030102300 0x7f60301034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:47.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.777+0000 7f602f400700 1 -- 172.21.15.112:0/2168490495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60301abba0 con 0x7f6030101940 2024-04-25T18:47:47.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.777+0000 7f602ea00700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6030102300 0x7f60301034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:47.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.777+0000 7f602f400700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6030101940 0x7f60301050e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f602000e9c0 tx=0x7f602000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:47.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.777+0000 7f6027e00700 1 -- 172.21.15.112:0/2168490495 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602000cbd0 con 0x7f6030101940 2024-04-25T18:47:47.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.777+0000 7f6036200700 1 -- 172.21.15.112:0/2168490495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60301abe30 con 0x7f6030101940 2024-04-25T18:47:47.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.777+0000 7f6027e00700 1 -- 172.21.15.112:0/2168490495 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602000ee60 con 0x7f6030101940 2024-04-25T18:47:47.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.778+0000 7f6036200700 1 -- 172.21.15.112:0/2168490495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60301ac290 con 0x7f6030101940 2024-04-25T18:47:47.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.778+0000 7f6027e00700 1 -- 172.21.15.112:0/2168490495 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020005580 con 0x7f6030101940 2024-04-25T18:47:47.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.779+0000 7f6036200700 1 -- 172.21.15.112:0/2168490495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6030103060 con 0x7f6030101940 2024-04-25T18:47:47.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.781+0000 7f6027e00700 1 -- 172.21.15.112:0/2168490495 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f6020004750 con 0x7f6030101940 2024-04-25T18:47:47.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.782+0000 7f6027e00700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f60140a3ed0 0x7f60140a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:47.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.782+0000 7f6027e00700 1 -- 172.21.15.112:0/2168490495 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f60200d0e30 con 0x7f6030101940 2024-04-25T18:47:47.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.783+0000 7f602ea00700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f60140a3ed0 0x7f60140a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:47.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.783+0000 7f602ea00700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f60140a3ed0 0x7f60140a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6030104cf0 tx=0x7f601c01a040 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:47.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:47.795+0000 7f6027e00700 1 -- 172.21.15.112:0/2168490495 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f602009d220 con 0x7f6030101940 2024-04-25T18:47:48.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.073+0000 7f6036200700 1 -- 172.21.15.112:0/2168490495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60301ac520 con 0x7f6030101940 2024-04-25T18:47:48.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.074+0000 7f6027e00700 1 -- 172.21.15.112:0/2168490495 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7f602009cb20 con 0x7f6030101940 2024-04-25T18:47:48.079 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:47:48.079 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:47:25.174463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24493,24511,24512]},"id":1}]} 2024-04-25T18:47:48.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 -- 172.21.15.112:0/2168490495 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f60140a3ed0 msgr2=0x7f60140a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:48.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f60140a3ed0 0x7f60140a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6030104cf0 tx=0x7f601c01a040 comp rx=0 tx=0).stop 2024-04-25T18:47:48.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 -- 172.21.15.112:0/2168490495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6030101940 msgr2=0x7f60301050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:48.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6030101940 0x7f60301050e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f602000e9c0 tx=0x7f602000ecd0 comp rx=0 tx=0).stop 2024-04-25T18:47:48.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 -- 172.21.15.112:0/2168490495 shutdown_connections 2024-04-25T18:47:48.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f60140a3ed0 0x7f60140a6390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:48.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6030107990 0x7f6030103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:48.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6030101940 0x7f60301050e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:48.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 --2- 172.21.15.112:0/2168490495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6030102300 0x7f60301034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:48.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 -- 172.21.15.112:0/2168490495 >> 172.21.15.112:0/2168490495 conn(0x7f60300788f0 msgr2=0x7f6030105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:48.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 -- 172.21.15.112:0/2168490495 shutdown_connections 2024-04-25T18:47:48.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:48.077+0000 7f6036200700 1 -- 172.21.15.112:0/2168490495 wait complete. 2024-04-25T18:47:48.081 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-04-25T18:47:48.481 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:48 smithi112 ceph-mon[27864]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:47:48.481 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:48 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2168490495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:47:48.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:48 smithi171 ceph-mon[33266]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:47:48.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:48 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2168490495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:47:48.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:48 smithi161 ceph-mon[32609]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:47:48.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:48 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2168490495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:47:49.028 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:47:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:50.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:50 smithi112 ceph-mon[27864]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T18:47:50.523 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:50 smithi161 ceph-mon[32609]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T18:47:50.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:50 smithi171 ceph-mon[33266]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T18:47:51.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.821+0000 7f9787400700 1 -- 172.21.15.112:0/761276846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9780102a90 msgr2=0x7f9780102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:51.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.821+0000 7f9787400700 1 --2- 172.21.15.112:0/761276846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9780102a90 0x7f9780102f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f976c009990 tx=0x7f976c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:47:51.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.821+0000 7f9787400700 1 -- 172.21.15.112:0/761276846 shutdown_connections 2024-04-25T18:47:51.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.821+0000 7f9787400700 1 --2- 172.21.15.112:0/761276846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9780103450 0x7f978010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:51.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.821+0000 7f9787400700 1 --2- 172.21.15.112:0/761276846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9780108a50 0x7f9780108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:51.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.821+0000 7f9787400700 1 --2- 172.21.15.112:0/761276846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9780102a90 0x7f9780102f10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:51.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.821+0000 7f9787400700 1 -- 172.21.15.112:0/761276846 >> 172.21.15.112:0/761276846 conn(0x7f9780079ba0 msgr2=0x7f9780079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:51.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.821+0000 7f9787400700 1 -- 172.21.15.112:0/761276846 shutdown_connections 2024-04-25T18:47:51.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.821+0000 7f9787400700 1 -- 172.21.15.112:0/761276846 wait complete. 2024-04-25T18:47:51.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.822+0000 7f9787400700 1 Processor -- start 2024-04-25T18:47:51.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.822+0000 7f9787400700 1 -- start start 2024-04-25T18:47:51.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.822+0000 7f9787400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9780102a90 0x7f97801991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:51.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.822+0000 7f9787400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9780103450 0x7f9780199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:51.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.822+0000 7f9787400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9780108a50 0x7f978019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:51.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.822+0000 7f9787400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9780112c50 con 0x7f9780102a90 2024-04-25T18:47:51.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.822+0000 7f9787400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9780112ad0 con 0x7f9780103450 2024-04-25T18:47:51.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.822+0000 7f9787400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9780112dd0 con 0x7f9780108a50 2024-04-25T18:47:51.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9784a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9780102a90 0x7f97801991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:51.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9777e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9780103450 0x7f9780199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:51.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9784a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9780102a90 0x7f97801991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38058/0 (socket says 172.21.15.112:38058) 2024-04-25T18:47:51.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9777e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9780103450 0x7f9780199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48280/0 (socket says 172.21.15.112:48280) 2024-04-25T18:47:51.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9784a00700 1 -- 172.21.15.112:0/1601686450 learned_addr learned my addr 172.21.15.112:0/1601686450 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:51.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9785400700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9780108a50 0x7f978019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:51.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9777e00700 1 -- 172.21.15.112:0/1601686450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9780108a50 msgr2=0x7f978019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:51.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9777e00700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9780108a50 0x7f978019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:51.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9777e00700 1 -- 172.21.15.112:0/1601686450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9780102a90 msgr2=0x7f97801991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:51.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9777e00700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9780102a90 0x7f97801991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:51.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9777e00700 1 -- 172.21.15.112:0/1601686450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978019e1f0 con 0x7f9780103450 2024-04-25T18:47:51.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.823+0000 7f9785400700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9780108a50 0x7f978019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:47:51.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.824+0000 7f9777e00700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9780103450 0x7f9780199720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f976c000c00 tx=0x7f976c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:51.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.824+0000 7f977e000700 1 -- 172.21.15.112:0/1601686450 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c026070 con 0x7f9780103450 2024-04-25T18:47:51.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.824+0000 7f977e000700 1 -- 172.21.15.112:0/1601686450 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f976c021410 con 0x7f9780103450 2024-04-25T18:47:51.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.824+0000 7f9787400700 1 -- 172.21.15.112:0/1601686450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978019e480 con 0x7f9780103450 2024-04-25T18:47:51.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.824+0000 7f977e000700 1 -- 172.21.15.112:0/1601686450 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c017650 con 0x7f9780103450 2024-04-25T18:47:51.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.824+0000 7f9787400700 1 -- 172.21.15.112:0/1601686450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97801a5d60 con 0x7f9780103450 2024-04-25T18:47:51.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.825+0000 7f9787400700 1 -- 172.21.15.112:0/1601686450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9780069730 con 0x7f9780103450 2024-04-25T18:47:51.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.827+0000 7f977e000700 1 -- 172.21.15.112:0/1601686450 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f976c029070 con 0x7f9780103450 2024-04-25T18:47:51.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.828+0000 7f977e000700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f97640a3f50 0x7f97640a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:51.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.829+0000 7f977e000700 1 -- 172.21.15.112:0/1601686450 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f976c0d1870 con 0x7f9780103450 2024-04-25T18:47:51.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.829+0000 7f9784a00700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f97640a3f50 0x7f97640a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:51.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.829+0000 7f9784a00700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f97640a3f50 0x7f97640a6410 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9770009ce0 tx=0x7f9770009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:51.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:51.831+0000 7f977e000700 1 -- 172.21.15.112:0/1601686450 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f976c013070 con 0x7f9780103450 2024-04-25T18:47:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.058+0000 7f9787400700 1 -- 172.21.15.112:0/1601686450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f978002df80 con 0x7f9780103450 2024-04-25T18:47:52.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.060+0000 7f977e000700 1 -- 172.21.15.112:0/1601686450 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+23089 (secure 0 0 0) 0x7f976c09ded0 con 0x7f9780103450 2024-04-25T18:47:52.063 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:47:52.063 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":95,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","created":"2024-04-25T18:33:37.379089+0000","modified":"2024-04-25T18:47:24.619335+0000","last_up_change":"2024-04-25T18:42:00.961646+0000","last_in_change":"2024-04-25T18:41:40.647162+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:37:45.106199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:43:11.017975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:43:15.116917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efb4bc01-e29b-4402-8a7a-3387d1731ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6803","nonce":232254535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6805","nonce":232254535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6809","nonce":232254535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6807","nonce":232254535}]},"public_addr":"172.21.15.112:6803/232254535","cluster_addr":"172.21.15.112:6805/232254535","heartbeat_back_addr":"172.21.15.112:6809/232254535","heartbeat_front_addr":"172.21.15.112:6807/232254535","state":["exists","up"]},{"osd":1,"uuid":"65a83fa2-96a6-4743-8c17-21531f5e885e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6801","nonce":664224744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6803","nonce":664224744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6807","nonce":664224744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6805","nonce":664224744}]},"public_addr":"172.21.15.161:6801/664224744","cluster_addr":"172.21.15.161:6803/664224744","heartbeat_back_addr":"172.21.15.161:6807/664224744","heartbeat_front_addr":"172.21.15.161:6805/664224744","state":["exists","up"]},{"osd":2,"uuid":"7562ec4c-c126-48ec-a51c-56fa461a2e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6801","nonce":345172073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6803","nonce":345172073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6807","nonce":345172073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6805","nonce":345172073}]},"public_addr":"172.21.15.171:6801/345172073","cluster_addr":"172.21.15.171:6803/345172073","heartbeat_back_addr":"172.21.15.171:6807/345172073","heartbeat_front_addr":"172.21.15.171:6805/345172073","state":["exists","up"]},{"osd":3,"uuid":"d7aae8cd-fec7-46d9-8773-25eaaca3cc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6811","nonce":1472407156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6813","nonce":1472407156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6817","nonce":1472407156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6815","nonce":1472407156}]},"public_addr":"172.21.15.112:6811/1472407156","cluster_addr":"172.21.15.112:6813/1472407156","heartbeat_back_addr":"172.21.15.112:6817/1472407156","heartbeat_front_addr":"172.21.15.112:6815/1472407156","state":["exists","up"]},{"osd":4,"uuid":"b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6809","nonce":373204306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6811","nonce":373204306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6815","nonce":373204306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6813","nonce":373204306}]},"public_addr":"172.21.15.161:6809/373204306","cluster_addr":"172.21.15.161:6811/373204306","heartbeat_back_addr":"172.21.15.161:6815/373204306","heartbeat_front_addr":"172.21.15.161:6813/373204306","state":["exists","up"]},{"osd":5,"uuid":"56124d12-bc1f-4cba-8430-938248851776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6809","nonce":2643512313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6811","nonce":2643512313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6815","nonce":2643512313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6813","nonce":2643512313}]},"public_addr":"172.21.15.171:6809/2643512313","cluster_addr":"172.21.15.171:6811/2643512313","heartbeat_back_addr":"172.21.15.171:6815/2643512313","heartbeat_front_addr":"172.21.15.171:6813/2643512313","state":["exists","up"]},{"osd":6,"uuid":"9563a9d3-352b-4686-975c-282ec167248b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6819","nonce":1933654933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6821","nonce":1933654933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6825","nonce":1933654933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6823","nonce":1933654933}]},"public_addr":"172.21.15.112:6819/1933654933","cluster_addr":"172.21.15.112:6821/1933654933","heartbeat_back_addr":"172.21.15.112:6825/1933654933","heartbeat_front_addr":"172.21.15.112:6823/1933654933","state":["exists","up"]},{"osd":7,"uuid":"f2d06e86-6aac-40f4-956e-18fe0b0e3543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6817","nonce":3060201138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6819","nonce":3060201138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6823","nonce":3060201138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6821","nonce":3060201138}]},"public_addr":"172.21.15.161:6817/3060201138","cluster_addr":"172.21.15.161:6819/3060201138","heartbeat_back_addr":"172.21.15.161:6823/3060201138","heartbeat_front_addr":"172.21.15.161:6821/3060201138","state":["exists","up"]},{"osd":8,"uuid":"f006e45f-2bb9-45d0-9020-be7c33c6a825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6817","nonce":835756571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6819","nonce":835756571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6823","nonce":835756571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6821","nonce":835756571}]},"public_addr":"172.21.15.171:6817/835756571","cluster_addr":"172.21.15.171:6819/835756571","heartbeat_back_addr":"172.21.15.171:6823/835756571","heartbeat_front_addr":"172.21.15.171:6821/835756571","state":["exists","up"]},{"osd":9,"uuid":"c5a96491-4151-4633-b3bc-2c214fe51c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6827","nonce":349758528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6829","nonce":349758528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6833","nonce":349758528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6831","nonce":349758528}]},"public_addr":"172.21.15.112:6827/349758528","cluster_addr":"172.21.15.112:6829/349758528","heartbeat_back_addr":"172.21.15.112:6833/349758528","heartbeat_front_addr":"172.21.15.112:6831/349758528","state":["exists","up"]},{"osd":10,"uuid":"244eda3e-25a0-490e-879f-c1e2726c1be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6825","nonce":3013292765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6827","nonce":3013292765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6831","nonce":3013292765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6829","nonce":3013292765}]},"public_addr":"172.21.15.161:6825/3013292765","cluster_addr":"172.21.15.161:6827/3013292765","heartbeat_back_addr":"172.21.15.161:6831/3013292765","heartbeat_front_addr":"172.21.15.161:6829/3013292765","state":["exists","up"]},{"osd":11,"uuid":"356e5c54-bbc3-45ee-bda2-85164ce3cb92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6825","nonce":1010336458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6827","nonce":1010336458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6831","nonce":1010336458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6829","nonce":1010336458}]},"public_addr":"172.21.15.171:6825/1010336458","cluster_addr":"172.21.15.171:6827/1010336458","heartbeat_back_addr":"172.21.15.171:6831/1010336458","heartbeat_front_addr":"172.21.15.171:6829/1010336458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:36:51.024650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:17.026638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:41.239726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:15.148968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:41.029118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:06.225858+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:41.639444+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:07.753400+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:32.292085+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:07.159926+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:33.749782+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:58.594447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/2665612708":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/698167759":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/722306877":"2024-04-26T18:47:24.619297+0000","172.21.15.112:6801/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/181862246":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/3393454453":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6800/605196912":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/3195850527":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/2936464788":"2024-04-26T18:47:24.619297+0000","172.21.15.112:6801/3468460974":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3392048544":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6801/605196912":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/1329430933":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/735578423":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/3468460974":"2024-04-26T18:34:20.605775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:47:52.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.063+0000 7f9787400700 1 -- 172.21.15.112:0/1601686450 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f97640a3f50 msgr2=0x7f97640a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.063+0000 7f9787400700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f97640a3f50 0x7f97640a6410 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9770009ce0 tx=0x7f9770009260 comp rx=0 tx=0).stop 2024-04-25T18:47:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.063+0000 7f9787400700 1 -- 172.21.15.112:0/1601686450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9780103450 msgr2=0x7f9780199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.063+0000 7f9787400700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9780103450 0x7f9780199720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f976c000c00 tx=0x7f976c0047b0 comp rx=0 tx=0).stop 2024-04-25T18:47:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.063+0000 7f9787400700 1 -- 172.21.15.112:0/1601686450 shutdown_connections 2024-04-25T18:47:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.063+0000 7f9787400700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f97640a3f50 0x7f97640a6410 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.063+0000 7f9787400700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9780108a50 0x7f978019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.063+0000 7f9787400700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9780102a90 0x7f97801991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.063+0000 7f9787400700 1 --2- 172.21.15.112:0/1601686450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9780103450 0x7f9780199720 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.063+0000 7f9787400700 1 -- 172.21.15.112:0/1601686450 >> 172.21.15.112:0/1601686450 conn(0x7f9780079ba0 msgr2=0x7f97801017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.064+0000 7f9787400700 1 -- 172.21.15.112:0/1601686450 shutdown_connections 2024-04-25T18:47:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:52.064+0000 7f9787400700 1 -- 172.21.15.112:0/1601686450 wait complete. 2024-04-25T18:47:52.464 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:52 smithi112 ceph-mon[27864]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T18:47:52.464 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:52 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1601686450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:47:52.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:52 smithi171 ceph-mon[33266]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T18:47:52.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:52 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1601686450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:47:52.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:52 smithi161 ceph-mon[32609]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T18:47:52.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:52 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1601686450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:47:52.966 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-25T18:47:53.417 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:54 smithi112 ceph-mon[27864]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:47:54.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:54 smithi171 ceph-mon[33266]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:47:54.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:54 smithi161 ceph-mon[32609]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:47:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.752+0000 7f09d3e00640 1 -- 172.21.15.112:0/2075554028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d4074320 msgr2=0x7f09d4074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.752+0000 7f09d3e00640 1 --2- 172.21.15.112:0/2075554028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d4074320 0x7f09d4074700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009900 tx=0x7f09c402a490 comp rx=0 tx=0).stop 2024-04-25T18:47:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.752+0000 7f09d3e00640 1 -- 172.21.15.112:0/2075554028 shutdown_connections 2024-04-25T18:47:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.752+0000 7f09d3e00640 1 --2- 172.21.15.112:0/2075554028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09d410e600 0x7f09d4110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.752+0000 7f09d3e00640 1 --2- 172.21.15.112:0/2075554028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09d4074c40 0x7f09d410e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.752+0000 7f09d3e00640 1 --2- 172.21.15.112:0/2075554028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d4074320 0x7f09d4074700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.752+0000 7f09d3e00640 1 -- 172.21.15.112:0/2075554028 >> 172.21.15.112:0/2075554028 conn(0x7f09d40fd890 msgr2=0x7f09d40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.752+0000 7f09d3e00640 1 -- 172.21.15.112:0/2075554028 shutdown_connections 2024-04-25T18:47:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.752+0000 7f09d3e00640 1 -- 172.21.15.112:0/2075554028 wait complete. 2024-04-25T18:47:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.753+0000 7f09d3e00640 1 Processor -- start 2024-04-25T18:47:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.753+0000 7f09d3e00640 1 -- start start 2024-04-25T18:47:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.753+0000 7f09d3e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09d4074320 0x7f09d41a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.753+0000 7f09d3e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09d4074c40 0x7f09d41a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.753+0000 7f09d3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d410e600 0x7f09d419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.753+0000 7f09d3e00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4114280 con 0x7f09d4074c40 2024-04-25T18:47:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.753+0000 7f09d3e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4114100 con 0x7f09d410e600 2024-04-25T18:47:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.753+0000 7f09d3e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4114400 con 0x7f09d4074320 2024-04-25T18:47:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d2000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09d4074c40 0x7f09d41a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d2a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09d4074320 0x7f09d41a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d2000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09d4074c40 0x7f09d41a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38078/0 (socket says 172.21.15.112:38078) 2024-04-25T18:47:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d2a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09d4074320 0x7f09d41a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:38412/0 (socket says 172.21.15.112:38412) 2024-04-25T18:47:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d2000640 1 -- 172.21.15.112:0/3851131934 learned_addr learned my addr 172.21.15.112:0/3851131934 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d3400640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d410e600 0x7f09d419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d2000640 1 -- 172.21.15.112:0/3851131934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09d4074320 msgr2=0x7f09d41a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d2000640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09d4074320 0x7f09d41a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d2000640 1 -- 172.21.15.112:0/3851131934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d410e600 msgr2=0x7f09d419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d2000640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d410e600 0x7f09d419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d2000640 1 -- 172.21.15.112:0/3851131934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d419ad60 con 0x7f09d4074c40 2024-04-25T18:47:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.754+0000 7f09d2000640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09d4074c40 0x7f09d41a0970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09c000ea60 tx=0x7f09c000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.755+0000 7f09cb400640 1 -- 172.21.15.112:0/3851131934 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c000cde0 con 0x7f09d4074c40 2024-04-25T18:47:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.755+0000 7f09d3e00640 1 -- 172.21.15.112:0/3851131934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d419b050 con 0x7f09d4074c40 2024-04-25T18:47:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.755+0000 7f09cb400640 1 -- 172.21.15.112:0/3851131934 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09c00035f0 con 0x7f09d4074c40 2024-04-25T18:47:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.755+0000 7f09cb400640 1 -- 172.21.15.112:0/3851131934 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0010690 con 0x7f09d4074c40 2024-04-25T18:47:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.755+0000 7f09d3e00640 1 -- 172.21.15.112:0/3851131934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d41a72f0 con 0x7f09d4074c40 2024-04-25T18:47:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.758+0000 7f09dc7eb740 1 Processor -- start 2024-04-25T18:47:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.758+0000 7f09dc7eb740 1 -- start start 2024-04-25T18:47:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 0x5589cb808d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 0x5589cb813650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb815f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589cb76e900 con 0x5589cb813b90 2024-04-25T18:47:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5589cb714ef0 con 0x5589cb8092d0 2024-04-25T18:47:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5589cb76a8c0 con 0x5589cb8089b0 2024-04-25T18:47:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09cb400640 1 -- 172.21.15.112:0/3851131934 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f09c00037a0 con 0x7f09d4074c40 2024-04-25T18:47:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09d2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 0x5589cb813650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09d2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 0x5589cb813650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48312/0 (socket says 172.21.15.112:48312) 2024-04-25T18:47:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09d3400640 1 --2- 172.21.15.112:0/2338716775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 0x5589cb808d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09d2a00640 1 -- 172.21.15.112:0/2338716775 learned_addr learned my addr 172.21.15.112:0/2338716775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09d2000640 1 --2- 172.21.15.112:0/2338716775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb815f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09d2a00640 1 -- 172.21.15.112:0/2338716775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 msgr2=0x5589cb808d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09d2a00640 1 --2- 172.21.15.112:0/2338716775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 0x5589cb808d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09d2a00640 1 -- 172.21.15.112:0/2338716775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 msgr2=0x5589cb815f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09d2a00640 1 --2- 172.21.15.112:0/2338716775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb815f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09d2a00640 1 -- 172.21.15.112:0/2338716775 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589cb481210 con 0x5589cb8092d0 2024-04-25T18:47:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.759+0000 7f09d3400640 1 --2- 172.21.15.112:0/2338716775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 0x5589cb808d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.760+0000 7f09d2a00640 1 --2- 172.21.15.112:0/2338716775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 0x5589cb813650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f09c4000c00 tx=0x7f09c4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8f727385085b8b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.760+0000 7f09bfe00640 1 -- 172.21.15.112:0/2338716775 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4035040 con 0x5589cb8092d0 2024-04-25T18:47:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.760+0000 7f09bfe00640 1 -- 172.21.15.112:0/2338716775 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09c40054f0 con 0x5589cb8092d0 2024-04-25T18:47:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.760+0000 7f09bfe00640 1 -- 172.21.15.112:0/2338716775 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c40035f0 con 0x5589cb8092d0 2024-04-25T18:47:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.760+0000 7f09cb400640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f09b80a3e30 0x7f09b80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09cb400640 1 -- 172.21.15.112:0/3851131934 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f09c00d0db0 con 0x7f09d4074c40 2024-04-25T18:47:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09d3400640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f09b80a3e30 0x7f09b80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2338716775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 msgr2=0x5589cb813650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2338716775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 0x5589cb813650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f09c4000c00 tx=0x7f09c4004040 comp rx=0 tx=0).stop 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2338716775 shutdown_connections 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2338716775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb815f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2338716775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 0x5589cb813650 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2338716775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 0x5589cb808d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2338716775 >> 172.21.15.112:0/2338716775 conn(0x5589cb804ec0 msgr2=0x5589cb8052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2338716775 shutdown_connections 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09d3400640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f09b80a3e30 0x7f09b80a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009fd0 tx=0x7f09cc009500 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.761+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2338716775 wait complete. 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.762+0000 7f09dc7eb740 1 -- start start 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.762+0000 7f09dc7eb740 1 Processor -- start 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.762+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 0x5589cb810af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8092d0 0x5589cb811030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb811570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589cb816760 con 0x5589cb813b90 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5589cb728310 con 0x5589cb8089b0 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5589cb8168e0 con 0x5589cb8092d0 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09dc7eb740 10 client.0 _ll_register_callbacks cb 0x5589cb6f6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09d3400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb811570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09d2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 0x5589cb810af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09d2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 0x5589cb810af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48320/0 (socket says 172.21.15.112:48320) 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09d2a00640 1 -- 172.21.15.112:0/3690084546 learned_addr learned my addr 172.21.15.112:0/3690084546 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09d2000640 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8092d0 0x5589cb811030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09d2a00640 1 -- 172.21.15.112:0/3690084546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8092d0 msgr2=0x5589cb811030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09d2a00640 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8092d0 0x5589cb811030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09d2a00640 1 -- 172.21.15.112:0/3690084546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 msgr2=0x5589cb811570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09d2a00640 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb811570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.763+0000 7f09d2a00640 1 -- 172.21.15.112:0/3690084546 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589cb54d620 con 0x5589cb8089b0 2024-04-25T18:47:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.764+0000 7f09d2a00640 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 0x5589cb810af0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005ab0 tx=0x7f09c40049a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e6dbd7553b2dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.764+0000 7f09bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:47:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.764+0000 7f09bcc00640 1 -- 172.21.15.112:0/3690084546 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4052060 con 0x5589cb8089b0 2024-04-25T18:47:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.764+0000 7f09dc7eb740 1 -- 172.21.15.112:0/3690084546 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09c40054f0 con 0x5589cb8089b0 2024-04-25T18:47:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.764+0000 7f09bcc00640 1 -- 172.21.15.112:0/3690084546 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f09c4004c30 con 0x5589cb8089b0 2024-04-25T18:47:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.764+0000 7f09bcc00640 1 -- 172.21.15.112:0/3690084546 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f09c4035070 con 0x5589cb8089b0 2024-04-25T18:47:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.764+0000 7f09dc7eb740 10 client.25205 fetch_fsmap learned FSMap version 26 2024-04-25T18:47:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.764+0000 7f09dc7eb740 1 -- 172.21.15.112:0/3690084546 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589cb822980 con 0x5589cb8089b0 2024-04-25T18:47:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.765+0000 7f09bcc00640 1 -- 172.21.15.112:0/3690084546 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09c40054f0 con 0x5589cb8089b0 2024-04-25T18:47:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.765+0000 7f09bcc00640 1 -- 172.21.15.112:0/3690084546 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f09c40039c0 con 0x5589cb8089b0 2024-04-25T18:47:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.765+0000 7f09dc7eb740 10 client.25205 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:47:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.765+0000 7f09dc7eb740 10 client.25205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:47:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.765+0000 7f09dc7eb740 20 client.25205 populate_metadata read hostname 'smithi112' 2024-04-25T18:47:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.765+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5589cb823cd0 0x5589cb8260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.765+0000 7f09dc7eb740 4 client.25205 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.765+0000 7f09dc7eb740 1 -- 172.21.15.112:0/3690084546 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589cb7828b0 con 0x5589cb823cd0 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.765+0000 7f09d2000640 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5589cb823cd0 0x5589cb8260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.766+0000 7f09d2000640 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5589cb823cd0 0x5589cb8260c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9048a68be23c2803 server_cookie=a04d4371dfdb44ed in_seq=0 out_seq=0 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.766+0000 7f09bcc00640 10 client.25205 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09bcc00640 1 -- 172.21.15.112:0/3690084546 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5589cb7828b0 con 0x5589cb823cd0 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09bcc00640 10 client.25205 handle_command_reply: tid=0 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 1 client.25205 shutdown 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 1 -- 172.21.15.112:0/3690084546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 msgr2=0x5589cb810af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 0x5589cb810af0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005ab0 tx=0x7f09c40049a0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 1 -- 172.21.15.112:0/3690084546 shutdown_connections 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5589cb823cd0 0x5589cb8260c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb811570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09bcc00640 0 client.25205 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8092d0 0x5589cb811030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/3690084546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 0x5589cb810af0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 1 -- 172.21.15.112:0/3690084546 >> 172.21.15.112:0/3690084546 conn(0x5589cb804ec0 msgr2=0x5589cb80abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 1 -- 172.21.15.112:0/3690084546 shutdown_connections 2024-04-25T18:47:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 1 -- 172.21.15.112:0/3690084546 wait complete. 2024-04-25T18:47:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.767+0000 7f09dc7eb740 20 client.25205 trim_cache size 0 max 16384 2024-04-25T18:47:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.783+0000 7f09dc7eb740 1 Processor -- start 2024-04-25T18:47:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.783+0000 7f09dc7eb740 1 -- start start 2024-04-25T18:47:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 0x5589cb81f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 0x5589cb82b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb82bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41b2580 con 0x5589cb813b90 2024-04-25T18:47:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41afe50 con 0x5589cb8092d0 2024-04-25T18:47:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41adca0 con 0x5589cb8089b0 2024-04-25T18:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09d2a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 0x5589cb81f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09d2a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 0x5589cb81f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:38448/0 (socket says 172.21.15.112:38448) 2024-04-25T18:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09d2a00640 1 -- 172.21.15.112:0/2234299848 learned_addr learned my addr 172.21.15.112:0/2234299848 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09d2000640 1 --2- 172.21.15.112:0/2234299848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 0x5589cb82b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09d3400640 1 --2- 172.21.15.112:0/2234299848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb82bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09d2a00640 1 -- 172.21.15.112:0/2234299848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 msgr2=0x5589cb82b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09d2a00640 1 --2- 172.21.15.112:0/2234299848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 0x5589cb82b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09d2a00640 1 -- 172.21.15.112:0/2234299848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 msgr2=0x5589cb82bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09d2a00640 1 --2- 172.21.15.112:0/2234299848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb82bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.784+0000 7f09d2a00640 1 -- 172.21.15.112:0/2234299848 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589cb809100 con 0x5589cb8089b0 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.785+0000 7f09d2000640 1 --2- 172.21.15.112:0/2234299848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 0x5589cb82b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.785+0000 7f09d2a00640 1 --2- 172.21.15.112:0/2234299848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 0x5589cb81f620 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009900 tx=0x7f09c4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8158632f9e851e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.785+0000 7f09bf400640 1 -- 172.21.15.112:0/2234299848 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4052070 con 0x5589cb8089b0 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.785+0000 7f09bf400640 1 -- 172.21.15.112:0/2234299848 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09c40054f0 con 0x5589cb8089b0 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.785+0000 7f09bf400640 1 -- 172.21.15.112:0/2234299848 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4004eb0 con 0x5589cb8089b0 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.786+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2234299848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 msgr2=0x5589cb81f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.786+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2234299848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 0x5589cb81f620 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009900 tx=0x7f09c4005850 comp rx=0 tx=0).stop 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.786+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2234299848 shutdown_connections 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.786+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2234299848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb813b90 0x5589cb82bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.786+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2234299848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8092d0 0x5589cb82b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.786+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2234299848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb8089b0 0x5589cb81f620 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.786+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2234299848 >> 172.21.15.112:0/2234299848 conn(0x5589cb804ec0 msgr2=0x5589cb80a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.786+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2234299848 shutdown_connections 2024-04-25T18:47:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.786+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2234299848 wait complete. 2024-04-25T18:47:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.786+0000 7f09dc7eb740 1 -- start start 2024-04-25T18:47:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.786+0000 7f09dc7eb740 1 Processor -- start 2024-04-25T18:47:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 0x5589cb81bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb8092d0 0x5589cb81c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09dc7eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb813b90 0x5589cb818f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41ae310 con 0x5589cb8092d0 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41af780 con 0x5589cb8089b0 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09dc7eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41ae9e0 con 0x5589cb813b90 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09dc7eb740 10 client.0 _ll_register_callbacks cb 0x7f09c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09d2000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb8092d0 0x5589cb81c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09d2000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb8092d0 0x5589cb81c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38108/0 (socket says 172.21.15.112:38108) 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09d2000640 1 -- 172.21.15.112:0/2184298222 learned_addr learned my addr 172.21.15.112:0/2184298222 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09d3400640 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb813b90 0x5589cb818f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.787+0000 7f09d2a00640 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 0x5589cb81bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09d2000640 1 -- 172.21.15.112:0/2184298222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb813b90 msgr2=0x5589cb818f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09d2000640 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb813b90 0x5589cb818f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09d2000640 1 -- 172.21.15.112:0/2184298222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 msgr2=0x5589cb81bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09d2000640 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 0x5589cb81bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09d2000640 1 -- 172.21.15.112:0/2184298222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589cb8197a0 con 0x5589cb8092d0 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09d2000640 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb8092d0 0x5589cb81c020 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09c0099470 tx=0x7f09c00d4840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad15f29c4380b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09bcc00640 1 -- 172.21.15.112:0/2184298222 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c009c400 con 0x5589cb8092d0 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09bcc00640 1 -- 172.21.15.112:0/2184298222 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f09c0003ac0 con 0x5589cb8092d0 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2184298222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09c40054f0 con 0x5589cb8092d0 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09bcc00640 1 -- 172.21.15.112:0/2184298222 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f09c009d1a0 con 0x5589cb8092d0 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09dc7eb740 10 client.15078 fetch_fsmap learned FSMap version 26 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.788+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2184298222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589cb889bb0 con 0x5589cb8092d0 2024-04-25T18:47:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.789+0000 7f09bcc00640 1 -- 172.21.15.112:0/2184298222 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09c40054f0 con 0x5589cb8092d0 2024-04-25T18:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.789+0000 7f09bcc00640 1 -- 172.21.15.112:0/2184298222 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f09c0099110 con 0x5589cb8092d0 2024-04-25T18:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.789+0000 7f09dc7eb740 10 client.15078 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.789+0000 7f09dc7eb740 10 client.15078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.789+0000 7f09dc7eb740 20 client.15078 populate_metadata read hostname 'smithi112' 2024-04-25T18:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.789+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5589cb88a2e0 0x5589cb88c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.789+0000 7f09dc7eb740 4 client.15078 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.789+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2184298222 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589cb77c320 con 0x5589cb88a2e0 2024-04-25T18:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.789+0000 7f09d2a00640 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5589cb88a2e0 0x5589cb88c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.790+0000 7f09d2a00640 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5589cb88a2e0 0x5589cb88c700 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99bcd37090ca1942 server_cookie=175a0f6655ea40bc in_seq=0 out_seq=0 2024-04-25T18:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.790+0000 7f09bcc00640 10 client.15078 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.796+0000 7f09bcc00640 1 -- 172.21.15.112:0/2184298222 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589cb77c320 con 0x5589cb88a2e0 2024-04-25T18:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.796+0000 7f09bcc00640 10 client.15078 handle_command_reply: tid=0 2024-04-25T18:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.796+0000 7f09dc7eb740 1 client.15078 shutdown 2024-04-25T18:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.796+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2184298222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb8092d0 msgr2=0x5589cb81c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.796+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb8092d0 0x5589cb81c020 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09c0099470 tx=0x7f09c00d4840 comp rx=0 tx=0).stop 2024-04-25T18:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2184298222 shutdown_connections 2024-04-25T18:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589cb813b90 0x5589cb818f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5589cb88a2e0 0x5589cb88c700 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589cb8092d0 0x5589cb81c020 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09dc7eb740 1 --2- 172.21.15.112:0/2184298222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5589cb8089b0 0x5589cb81bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2184298222 >> 172.21.15.112:0/2184298222 conn(0x5589cb804ec0 msgr2=0x5589cb810a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09bcc00640 0 client.15078 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2184298222 shutdown_connections 2024-04-25T18:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09dc7eb740 1 -- 172.21.15.112:0/2184298222 wait complete. 2024-04-25T18:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09dc7eb740 20 client.15078 trim_cache size 0 max 16384 2024-04-25T18:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09d3e00640 1 -- 172.21.15.112:0/3851131934 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f09b80a3e30 msgr2=0x7f09b80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.797+0000 7f09d3e00640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f09b80a3e30 0x7f09b80a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009fd0 tx=0x7f09cc009500 comp rx=0 tx=0).stop 2024-04-25T18:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.798+0000 7f09d3e00640 1 -- 172.21.15.112:0/3851131934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09d4074c40 msgr2=0x7f09d41a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.798+0000 7f09d3e00640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09d4074c40 0x7f09d41a0970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09c000ea60 tx=0x7f09c000ef20 comp rx=0 tx=0).stop 2024-04-25T18:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.798+0000 7f09d3e00640 1 -- 172.21.15.112:0/3851131934 shutdown_connections 2024-04-25T18:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.798+0000 7f09d3e00640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f09b80a3e30 0x7f09b80a62f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.798+0000 7f09d3e00640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d410e600 0x7f09d419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.798+0000 7f09d3e00640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09d4074c40 0x7f09d41a0970 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.798+0000 7f09d3e00640 1 --2- 172.21.15.112:0/3851131934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09d4074320 0x7f09d41a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.798+0000 7f09d3e00640 1 -- 172.21.15.112:0/3851131934 >> 172.21.15.112:0/3851131934 conn(0x7f09d40fd890 msgr2=0x7f09d40ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.798+0000 7f09d3e00640 1 -- 172.21.15.112:0/3851131934 shutdown_connections 2024-04-25T18:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:47:54.798+0000 7f09d3e00640 1 -- 172.21.15.112:0/3851131934 wait complete. 2024-04-25T18:47:56.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:56 smithi171 ceph-mon[33266]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T18:47:56.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:56 smithi161 ceph-mon[32609]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T18:47:56.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:56 smithi112 ceph-mon[27864]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T18:47:56.835 INFO:teuthology.orchestra.run.smithi112.stdout:{"run_dir":"/var/run/ceph"} 2024-04-25T18:47:56.835 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-25T18:47:56.836 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 59754 2024-04-25T18:47:56.836 DEBUG:tasks.quiescer:Yielding 2024-04-25T18:47:56.836 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T18:47:56.850 INFO:tasks.workunit:Pulling workunits from ref 7797745e6d8b3ac036504dc61491d368615024da 2024-04-25T18:47:56.850 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T18:47:56.850 INFO:tasks.workunit:timeout=3h 2024-04-25T18:47:56.850 INFO:tasks.workunit:cleanup=True 2024-04-25T18:47:56.851 DEBUG:teuthology.orchestra.run.smithi112:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T18:47:56.851 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:47:56.900 INFO:teuthology.orchestra.run.smithi112.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T18:47:56.901 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T18:47:56.901 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-25T18:47:56.901 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:47:56.901 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T18:47:56.901 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 18:44:01.937925002 +0000 2024-04-25T18:47:56.901 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 18:44:01.937925002 +0000 2024-04-25T18:47:56.901 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 18:47:24.117662055 +0000 2024-04-25T18:47:56.901 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 18:44:01.937925002 +0000 2024-04-25T18:47:56.901 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T18:47:56.901 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T18:47:57.026 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7797745e6d8b3ac036504dc61491d368615024da 2024-04-25T18:47:57.073 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T18:47:57.280 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:47:58 smithi171 ceph-mon[33266]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T18:47:58.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:47:58 smithi161 ceph-mon[32609]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T18:47:58.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:47:58 smithi112 ceph-mon[27864]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T18:47:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.724+0000 7f2585a00700 1 -- 172.21.15.112:0/2185306362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2580105890 msgr2=0x7f25801098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.724+0000 7f2585a00700 1 --2- 172.21.15.112:0/2185306362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2580105890 0x7f25801098e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f256c0099c0 tx=0x7f256c009cd0 comp rx=0 tx=0).stop 2024-04-25T18:47:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.725+0000 7f2585a00700 1 -- 172.21.15.112:0/2185306362 shutdown_connections 2024-04-25T18:47:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.725+0000 7f2585a00700 1 --2- 172.21.15.112:0/2185306362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2580109fc0 0x7f2580111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:59.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.725+0000 7f2585a00700 1 --2- 172.21.15.112:0/2185306362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2580104ee0 0x7f25801052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.725+0000 7f2585a00700 1 --2- 172.21.15.112:0/2185306362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2580105890 0x7f25801098e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.725+0000 7f2585a00700 1 -- 172.21.15.112:0/2185306362 >> 172.21.15.112:0/2185306362 conn(0x7f2580100770 msgr2=0x7f2580102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.725+0000 7f2585a00700 1 -- 172.21.15.112:0/2185306362 shutdown_connections 2024-04-25T18:47:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.725+0000 7f2585a00700 1 -- 172.21.15.112:0/2185306362 wait complete. 2024-04-25T18:47:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.726+0000 7f2585a00700 1 Processor -- start 2024-04-25T18:47:59.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.726+0000 7f2585a00700 1 -- start start 2024-04-25T18:47:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.727+0000 7f2585a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2580104ee0 0x7f258019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.727+0000 7f2585a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2580105890 0x7f258019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.727+0000 7f2585a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2580109fc0 0x7f258019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.727+0000 7f2585a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2580114ea0 con 0x7f2580109fc0 2024-04-25T18:47:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.727+0000 7f2585a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2580114d20 con 0x7f2580104ee0 2024-04-25T18:47:59.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.727+0000 7f2585a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2580115020 con 0x7f2580105890 2024-04-25T18:47:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.727+0000 7f257ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2580104ee0 0x7f258019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.727+0000 7f257e000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2580105890 0x7f258019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.727+0000 7f257ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2580104ee0 0x7f258019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48358/0 (socket says 172.21.15.112:48358) 2024-04-25T18:47:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.727+0000 7f257ea00700 1 -- 172.21.15.112:0/472948882 learned_addr learned my addr 172.21.15.112:0/472948882 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:47:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.728+0000 7f257f400700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2580109fc0 0x7f258019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.728+0000 7f257e000700 1 -- 172.21.15.112:0/472948882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2580104ee0 msgr2=0x7f258019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.728+0000 7f257e000700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2580104ee0 0x7f258019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:59.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.728+0000 7f257e000700 1 -- 172.21.15.112:0/472948882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2580109fc0 msgr2=0x7f258019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:59.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.728+0000 7f257e000700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2580109fc0 0x7f258019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:59.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.728+0000 7f257e000700 1 -- 172.21.15.112:0/472948882 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25801a0380 con 0x7f2580105890 2024-04-25T18:47:59.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.728+0000 7f257e000700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2580105890 0x7f258019b8b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f256c0098e0 tx=0x7f256c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:59.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.729+0000 7f2577400700 1 -- 172.21.15.112:0/472948882 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c026070 con 0x7f2580105890 2024-04-25T18:47:59.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.729+0000 7f2585a00700 1 -- 172.21.15.112:0/472948882 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25801a0610 con 0x7f2580105890 2024-04-25T18:47:59.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.729+0000 7f2585a00700 1 -- 172.21.15.112:0/472948882 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25801a7ef0 con 0x7f2580105890 2024-04-25T18:47:59.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.729+0000 7f2577400700 1 -- 172.21.15.112:0/472948882 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f256c003c70 con 0x7f2580105890 2024-04-25T18:47:59.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.729+0000 7f2577400700 1 -- 172.21.15.112:0/472948882 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c00faa0 con 0x7f2580105890 2024-04-25T18:47:59.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.730+0000 7f2585a00700 1 -- 172.21.15.112:0/472948882 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2560005330 con 0x7f2580105890 2024-04-25T18:47:59.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.734+0000 7f2577400700 1 -- 172.21.15.112:0/472948882 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f256c029070 con 0x7f2580105890 2024-04-25T18:47:59.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.735+0000 7f2577400700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f25640a3f50 0x7f25640a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:59.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.736+0000 7f2577400700 1 -- 172.21.15.112:0/472948882 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f256c0d16a0 con 0x7f2580105890 2024-04-25T18:47:59.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.736+0000 7f257ea00700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f25640a3f50 0x7f25640a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:59.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.736+0000 7f2577400700 1 -- 172.21.15.112:0/472948882 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f256c030080 con 0x7f2580105890 2024-04-25T18:47:59.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:47:59.737+0000 7f257ea00700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f25640a3f50 0x7f25640a6410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2570009d20 tx=0x7f2570009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:00.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.019+0000 7f2585a00700 1 -- 172.21.15.112:0/472948882 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2560005620 con 0x7f2580105890 2024-04-25T18:48:00.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.020+0000 7f2577400700 1 -- 172.21.15.112:0/472948882 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7f256c013070 con 0x7f2580105890 2024-04-25T18:48:00.022 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:48:00.022 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:47:25.174463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24493,24511,24512]},"id":1}]} 2024-04-25T18:48:00.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.023+0000 7f2585a00700 1 -- 172.21.15.112:0/472948882 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f25640a3f50 msgr2=0x7f25640a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:00.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.023+0000 7f2585a00700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f25640a3f50 0x7f25640a6410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2570009d20 tx=0x7f2570009260 comp rx=0 tx=0).stop 2024-04-25T18:48:00.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.023+0000 7f2585a00700 1 -- 172.21.15.112:0/472948882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2580105890 msgr2=0x7f258019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:00.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.023+0000 7f2585a00700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2580105890 0x7f258019b8b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f256c0098e0 tx=0x7f256c004270 comp rx=0 tx=0).stop 2024-04-25T18:48:00.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.024+0000 7f2585a00700 1 -- 172.21.15.112:0/472948882 shutdown_connections 2024-04-25T18:48:00.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.024+0000 7f2585a00700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f25640a3f50 0x7f25640a6410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:00.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.024+0000 7f2585a00700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2580109fc0 0x7f258019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:00.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.024+0000 7f2585a00700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2580104ee0 0x7f258019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:00.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.024+0000 7f2585a00700 1 --2- 172.21.15.112:0/472948882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2580105890 0x7f258019b8b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:00.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.024+0000 7f2585a00700 1 -- 172.21.15.112:0/472948882 >> 172.21.15.112:0/472948882 conn(0x7f2580100770 msgr2=0x7f2580102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:00.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.024+0000 7f2585a00700 1 -- 172.21.15.112:0/472948882 shutdown_connections 2024-04-25T18:48:00.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:00.024+0000 7f2585a00700 1 -- 172.21.15.112:0/472948882 wait complete. 2024-04-25T18:48:00.028 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-04-25T18:48:00.467 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:00 smithi112 ceph-mon[27864]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:48:00.467 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:00 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/472948882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:48:00.467 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:48:00 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:48:00.232+0000 7fc988c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:48:00.467 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:48:00 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:48:00.232+0000 7fc988c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:48:00.467 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:48:00 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:48:00.232+0000 7fc988c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:48:00.467 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:48:00 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:48:00.232+0000 7fc988c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:48:00.467 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:48:00 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:48:00.232+0000 7fc988c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:48:00.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:00 smithi171 ceph-mon[33266]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:48:00.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:00 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/472948882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:48:00.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:00 smithi161 ceph-mon[32609]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:48:00.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:00 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/472948882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:48:01.209 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T18:43:19.347467+0000', 'modified': '2024-04-25T18:47:25.174463+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24511, 'mds_2': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/4007863322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 4007863322}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 4007863322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/3522712423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 3522712423}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 3522712423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'g', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6839/3241331981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 3241331981}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 3241331981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.171:6839/1642228003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 1642228003}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 1642228003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.112:6841/1028859850', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6840', 'nonce': 1028859850}, {'type': 'v1', 'addr': '172.21.15.112:6841', 'nonce': 1028859850}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.161:6839/2315334000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2315334000}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2315334000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24493, 24511, 24512]} 2024-04-25T18:48:01.209 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T18:48:01.209 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T18:48:01.676 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:48:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:02 smithi112 ceph-mon[27864]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:48:02.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:02 smithi112 ceph-mon[27864]: mgrmap e25: x(active, since 36s), standbys: y, z 2024-04-25T18:48:02.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:02 smithi171 ceph-mon[33266]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:48:02.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:02 smithi171 ceph-mon[33266]: mgrmap e25: x(active, since 36s), standbys: y, z 2024-04-25T18:48:02.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:02 smithi161 ceph-mon[32609]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:48:02.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:02 smithi161 ceph-mon[32609]: mgrmap e25: x(active, since 36s), standbys: y, z 2024-04-25T18:48:04.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.109+0000 7f31bbc00700 1 -- 172.21.15.112:0/3468341466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b4108a80 msgr2=0x7f31b4108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.109+0000 7f31bbc00700 1 --2- 172.21.15.112:0/3468341466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b4108a80 0x7f31b4108e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009990 tx=0x7f31a8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.109+0000 7f31bbc00700 1 -- 172.21.15.112:0/3468341466 shutdown_connections 2024-04-25T18:48:04.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.109+0000 7f31bbc00700 1 --2- 172.21.15.112:0/3468341466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31b4103450 0x7f31b410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.109+0000 7f31bbc00700 1 --2- 172.21.15.112:0/3468341466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b4108a80 0x7f31b4108e60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.109+0000 7f31bbc00700 1 --2- 172.21.15.112:0/3468341466 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31b4102a90 0x7f31b4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.109+0000 7f31bbc00700 1 -- 172.21.15.112:0/3468341466 >> 172.21.15.112:0/3468341466 conn(0x7f31b4079bf0 msgr2=0x7f31b407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:04.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.109+0000 7f31bbc00700 1 -- 172.21.15.112:0/3468341466 shutdown_connections 2024-04-25T18:48:04.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.109+0000 7f31bbc00700 1 -- 172.21.15.112:0/3468341466 wait complete. 2024-04-25T18:48:04.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.110+0000 7f31bbc00700 1 Processor -- start 2024-04-25T18:48:04.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.110+0000 7f31bbc00700 1 -- start start 2024-04-25T18:48:04.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.111+0000 7f31bbc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b4102a90 0x7f31b4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.111+0000 7f31bbc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31b4103450 0x7f31b4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.111+0000 7f31bbc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31b4108a80 0x7f31b40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.111+0000 7f31bbc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4112c80 con 0x7f31b4102a90 2024-04-25T18:48:04.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.111+0000 7f31bbc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4112b00 con 0x7f31b4103450 2024-04-25T18:48:04.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.111+0000 7f31bbc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4112e00 con 0x7f31b4108a80 2024-04-25T18:48:04.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.111+0000 7f31b9200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b4102a90 0x7f31b4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.111+0000 7f31b9200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b4102a90 0x7f31b4074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34448/0 (socket says 172.21.15.112:34448) 2024-04-25T18:48:04.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.111+0000 7f31b9200700 1 -- 172.21.15.112:0/1213852352 learned_addr learned my addr 172.21.15.112:0/1213852352 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.111+0000 7f31b3e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31b4103450 0x7f31b4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.112+0000 7f31b9c00700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31b4108a80 0x7f31b40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.112+0000 7f31b3e00700 1 -- 172.21.15.112:0/1213852352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31b4108a80 msgr2=0x7f31b40757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.112+0000 7f31b3e00700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31b4108a80 0x7f31b40757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.112+0000 7f31b3e00700 1 -- 172.21.15.112:0/1213852352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b4102a90 msgr2=0x7f31b4074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.112+0000 7f31b3e00700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b4102a90 0x7f31b4074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.112+0000 7f31b3e00700 1 -- 172.21.15.112:0/1213852352 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b41a76e0 con 0x7f31b4103450 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.112+0000 7f31b9200700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b4102a90 0x7f31b4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.112+0000 7f31b9c00700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31b4108a80 0x7f31b40757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.112+0000 7f31b3e00700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31b4103450 0x7f31b4075280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f31a000e9f0 tx=0x7f31a000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.112+0000 7f31b1600700 1 -- 172.21.15.112:0/1213852352 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a000cbd0 con 0x7f31b4103450 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.113+0000 7f31b1600700 1 -- 172.21.15.112:0/1213852352 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a00044d0 con 0x7f31b4103450 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.113+0000 7f31bbc00700 1 -- 172.21.15.112:0/1213852352 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b41a79d0 con 0x7f31b4103450 2024-04-25T18:48:04.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.113+0000 7f31b1600700 1 -- 172.21.15.112:0/1213852352 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0005020 con 0x7f31b4103450 2024-04-25T18:48:04.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.113+0000 7f31bbc00700 1 -- 172.21.15.112:0/1213852352 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b41a7ec0 con 0x7f31b4103450 2024-04-25T18:48:04.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.116+0000 7f31b1600700 1 -- 172.21.15.112:0/1213852352 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f31a00051e0 con 0x7f31b4103450 2024-04-25T18:48:04.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.117+0000 7f31b1600700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f319c0a3f60 0x7f319c0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.117+0000 7f31b1600700 1 -- 172.21.15.112:0/1213852352 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f31a00d0b00 con 0x7f31b4103450 2024-04-25T18:48:04.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.118+0000 7f31b9200700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f319c0a3f60 0x7f319c0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.118+0000 7f31c6532b80 1 Processor -- start 2024-04-25T18:48:04.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.118+0000 7f31c6532b80 1 -- start start 2024-04-25T18:48:04.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.118+0000 7f31b9200700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f319c0a3f60 0x7f319c0a6420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f31a8006010 tx=0x7f31a800b280 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:04.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.119+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959a2f0 0x5558e959a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.119+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 0x5558e95a5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.119+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e95a5ab0 0x5558e95a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.119+0000 7f31c6532b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558e94ede40 con 0x5558e959ac10 2024-04-25T18:48:04.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.119+0000 7f31c6532b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5558e94946e0 con 0x5558e959a2f0 2024-04-25T18:48:04.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.119+0000 7f31c6532b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5558e94e9e00 con 0x5558e95a5ab0 2024-04-25T18:48:04.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.119+0000 7f31b9200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959a2f0 0x5558e959a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.119+0000 7f31b3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 0x5558e95a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.119+0000 7f31b9200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959a2f0 0x5558e959a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:37080/0 (socket says 172.21.15.112:37080) 2024-04-25T18:48:04.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.119+0000 7f31b9200700 1 -- 172.21.15.112:0/2022532705 learned_addr learned my addr 172.21.15.112:0/2022532705 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:04.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.119+0000 7f31b9c00700 1 --2- 172.21.15.112:0/2022532705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e95a5ab0 0x5558e95a7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.120+0000 7f31b3e00700 1 -- 172.21.15.112:0/2022532705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e95a5ab0 msgr2=0x5558e95a7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:04.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.120+0000 7f31b3e00700 1 --2- 172.21.15.112:0/2022532705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e95a5ab0 0x5558e95a7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.120+0000 7f31b3e00700 1 -- 172.21.15.112:0/2022532705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959a2f0 msgr2=0x5558e959a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:04.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.120+0000 7f31b3e00700 1 --2- 172.21.15.112:0/2022532705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959a2f0 0x5558e959a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.120+0000 7f31b3e00700 1 -- 172.21.15.112:0/2022532705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558e948a630 con 0x5558e959ac10 2024-04-25T18:48:04.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.120+0000 7f31b3e00700 1 --2- 172.21.15.112:0/2022532705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 0x5558e95a5570 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f31a00ce010 tx=0x7f31a0098c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd70dc1b837c27b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:04.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.120+0000 7f31a6000700 1 -- 172.21.15.112:0/2022532705 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a001c040 con 0x5558e959ac10 2024-04-25T18:48:04.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.120+0000 7f31a6000700 1 -- 172.21.15.112:0/2022532705 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a009c860 con 0x5558e959ac10 2024-04-25T18:48:04.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.120+0000 7f31a6000700 1 -- 172.21.15.112:0/2022532705 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a009ccc0 con 0x5558e959ac10 2024-04-25T18:48:04.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.121+0000 7f31c6532b80 1 -- 172.21.15.112:0/2022532705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 msgr2=0x5558e95a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.121+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2022532705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 0x5558e95a5570 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f31a00ce010 tx=0x7f31a0098c10 comp rx=0 tx=0).stop 2024-04-25T18:48:04.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.121+0000 7f31c6532b80 1 -- 172.21.15.112:0/2022532705 shutdown_connections 2024-04-25T18:48:04.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.121+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2022532705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e95a5ab0 0x5558e95a7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.121+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2022532705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959a2f0 0x5558e959a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.121+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2022532705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 0x5558e95a5570 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.121+0000 7f31c6532b80 1 -- 172.21.15.112:0/2022532705 >> 172.21.15.112:0/2022532705 conn(0x5558e94c9550 msgr2=0x5558e9596790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:04.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.122+0000 7f31c6532b80 1 -- 172.21.15.112:0/2022532705 shutdown_connections 2024-04-25T18:48:04.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.122+0000 7f31c6532b80 1 -- 172.21.15.112:0/2022532705 wait complete. 2024-04-25T18:48:04.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.122+0000 7f31c6532b80 1 -- start start 2024-04-25T18:48:04.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.122+0000 7f31c6532b80 1 Processor -- start 2024-04-25T18:48:04.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.123+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959a2f0 0x5558e95a5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.123+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 0x5558e95a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.123+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e95a5ab0 0x5558e95a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.123+0000 7f31c6532b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558e95a85a0 con 0x5558e959a2f0 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.123+0000 7f31c6532b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5558e94a7810 con 0x5558e959ac10 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.123+0000 7f31c6532b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5558e95a8720 con 0x5558e95a5ab0 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.123+0000 7f31c6532b80 10 client.0 _ll_register_callbacks cb 0x5558e94866d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.123+0000 7f31b9200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959a2f0 0x5558e95a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.123+0000 7f31b9200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959a2f0 0x5558e95a5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34466/0 (socket says 172.21.15.112:34466) 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.123+0000 7f31b9200700 1 -- 172.21.15.112:0/2339127575 learned_addr learned my addr 172.21.15.112:0/2339127575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.124+0000 7f31b3e00700 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 0x5558e95a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.124+0000 7f31b9200700 1 -- 172.21.15.112:0/2339127575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e95a5ab0 msgr2=0x5558e95a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.124+0000 7f31b9c00700 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e95a5ab0 0x5558e95a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.124+0000 7f31b9200700 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e95a5ab0 0x5558e95a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.124+0000 7f31b9200700 1 -- 172.21.15.112:0/2339127575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 msgr2=0x5558e95a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.124+0000 7f31b9200700 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 0x5558e95a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.124+0000 7f31b9200700 1 -- 172.21.15.112:0/2339127575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558e943b880 con 0x5558e959a2f0 2024-04-25T18:48:04.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.124+0000 7f31b3e00700 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 0x5558e95a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:48:04.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.124+0000 7f31b9c00700 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e95a5ab0 0x5558e95a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:48:04.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.124+0000 7f31b9200700 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959a2f0 0x5558e95a5280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f31a8003700 tx=0x7f31a8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6712f15e7dc02e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:04.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.124+0000 7f319aa00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:48:04.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.125+0000 7f319aa00700 1 -- 172.21.15.112:0/2339127575 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a800f930 con 0x5558e959a2f0 2024-04-25T18:48:04.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.125+0000 7f319aa00700 1 -- 172.21.15.112:0/2339127575 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f31a800fad0 con 0x5558e959a2f0 2024-04-25T18:48:04.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.125+0000 7f31c6532b80 1 -- 172.21.15.112:0/2339127575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31a009c860 con 0x5558e959a2f0 2024-04-25T18:48:04.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.125+0000 7f319aa00700 1 -- 172.21.15.112:0/2339127575 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f31a803c050 con 0x5558e959a2f0 2024-04-25T18:48:04.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.125+0000 7f31c6532b80 10 client.15102 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:04.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.125+0000 7f31c6532b80 1 -- 172.21.15.112:0/2339127575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5558e9484aa0 con 0x5558e959a2f0 2024-04-25T18:48:04.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.125+0000 7f319aa00700 1 -- 172.21.15.112:0/2339127575 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a009c860 con 0x5558e959a2f0 2024-04-25T18:48:04.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.126+0000 7f319aa00700 1 -- 172.21.15.112:0/2339127575 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f31a802e400 con 0x5558e959a2f0 2024-04-25T18:48:04.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.126+0000 7f31c6532b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:04.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.126+0000 7f31c6532b80 10 client.15102 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:48:04.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.126+0000 7f31c6532b80 20 client.15102 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:04.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.126+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x5558e95b5320 0x5558e95b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.126+0000 7f31c6532b80 4 client.15102 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:48:04.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.126+0000 7f31c6532b80 1 -- 172.21.15.112:0/2339127575 --> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558e94fb630 con 0x5558e95b5320 2024-04-25T18:48:04.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.127+0000 7f31b3e00700 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x5558e95b5320 0x5558e95b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.127+0000 7f31b3e00700 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x5558e95b5320 0x5558e95b6790 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=520117f14392db06 server_cookie=51299734be672e2c in_seq=0 out_seq=0 2024-04-25T18:48:04.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.127+0000 7f319aa00700 10 client.15102 ms_handle_connect on v2:172.21.15.171:6838/1642228003 2024-04-25T18:48:04.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.128+0000 7f319aa00700 1 -- 172.21.15.112:0/2339127575 <== mds.2 v2:172.21.15.171:6838/1642228003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558e94fb630 con 0x5558e95b5320 2024-04-25T18:48:04.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.128+0000 7f319aa00700 10 client.15102 handle_command_reply: tid=0 2024-04-25T18:48:04.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.128+0000 7f31c6532b80 1 client.15102 shutdown 2024-04-25T18:48:04.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.128+0000 7f31c6532b80 1 -- 172.21.15.112:0/2339127575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959a2f0 msgr2=0x5558e95a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.128+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959a2f0 0x5558e95a5280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f31a8003700 tx=0x7f31a8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.128+0000 7f31c6532b80 1 -- 172.21.15.112:0/2339127575 shutdown_connections 2024-04-25T18:48:04.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.129+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x5558e95b5320 0x5558e95b6790 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.129+0000 7f319aa00700 0 client.15102 ms_handle_reset on v2:172.21.15.171:6838/1642228003 2024-04-25T18:48:04.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.129+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e95a5ab0 0x5558e95a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.129+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959a2f0 0x5558e95a5280 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.129+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2339127575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 0x5558e95a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.129+0000 7f31c6532b80 1 -- 172.21.15.112:0/2339127575 >> 172.21.15.112:0/2339127575 conn(0x5558e94c9550 msgr2=0x5558e959cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:04.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.129+0000 7f31c6532b80 1 -- 172.21.15.112:0/2339127575 shutdown_connections 2024-04-25T18:48:04.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.129+0000 7f31c6532b80 1 -- 172.21.15.112:0/2339127575 wait complete. 2024-04-25T18:48:04.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.129+0000 7f31c6532b80 20 client.15102 trim_cache size 0 max 16384 2024-04-25T18:48:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.151+0000 7f31c6532b80 1 Processor -- start 2024-04-25T18:48:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.151+0000 7f31c6532b80 1 -- start start 2024-04-25T18:48:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.152+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e959a2f0 0x5558e9596920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.152+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 0x5558e95a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.152+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e95a5ab0 0x5558e95a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.152+0000 7f31c6532b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558e95b4ad0 con 0x5558e95a5ab0 2024-04-25T18:48:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.152+0000 7f31c6532b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5558e95b51a0 con 0x5558e959ac10 2024-04-25T18:48:04.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.152+0000 7f31c6532b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5558e95b5870 con 0x5558e959a2f0 2024-04-25T18:48:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.152+0000 7f31b9c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e95a5ab0 0x5558e95a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.152+0000 7f31b9c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e95a5ab0 0x5558e95a28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34478/0 (socket says 172.21.15.112:34478) 2024-04-25T18:48:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.152+0000 7f31b9c00700 1 -- 172.21.15.112:0/2973593213 learned_addr learned my addr 172.21.15.112:0/2973593213 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.152+0000 7f31b3e00700 1 --2- 172.21.15.112:0/2973593213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 0x5558e95a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.152+0000 7f31b9200700 1 --2- 172.21.15.112:0/2973593213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e959a2f0 0x5558e9596920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.153+0000 7f31b3e00700 1 -- 172.21.15.112:0/2973593213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e959a2f0 msgr2=0x5558e9596920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.153+0000 7f31b3e00700 1 --2- 172.21.15.112:0/2973593213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e959a2f0 0x5558e9596920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.153+0000 7f31b3e00700 1 -- 172.21.15.112:0/2973593213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e95a5ab0 msgr2=0x5558e95a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.153+0000 7f31b3e00700 1 --2- 172.21.15.112:0/2973593213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e95a5ab0 0x5558e95a28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.153+0000 7f31b3e00700 1 -- 172.21.15.112:0/2973593213 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558e9597ce0 con 0x5558e959ac10 2024-04-25T18:48:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.153+0000 7f31b9c00700 1 --2- 172.21.15.112:0/2973593213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e95a5ab0 0x5558e95a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.153+0000 7f31b3e00700 1 --2- 172.21.15.112:0/2973593213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 0x5558e95a2360 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f31a00ce100 tx=0x7f31a0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d81aa4321cf8c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.153+0000 7f31a5600700 1 -- 172.21.15.112:0/2973593213 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a001c040 con 0x5558e959ac10 2024-04-25T18:48:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.153+0000 7f31a5600700 1 -- 172.21.15.112:0/2973593213 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a009ab60 con 0x5558e959ac10 2024-04-25T18:48:04.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.153+0000 7f31a5600700 1 -- 172.21.15.112:0/2973593213 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a00a5040 con 0x5558e959ac10 2024-04-25T18:48:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.154+0000 7f31c6532b80 1 -- 172.21.15.112:0/2973593213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 msgr2=0x5558e95a2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.154+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2973593213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 0x5558e95a2360 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f31a00ce100 tx=0x7f31a0004ab0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.154+0000 7f31c6532b80 1 -- 172.21.15.112:0/2973593213 shutdown_connections 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.154+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2973593213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e95a5ab0 0x5558e95a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.154+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2973593213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e959a2f0 0x5558e9596920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.154+0000 7f31c6532b80 1 --2- 172.21.15.112:0/2973593213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e959ac10 0x5558e95a2360 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.154+0000 7f31c6532b80 1 -- 172.21.15.112:0/2973593213 >> 172.21.15.112:0/2973593213 conn(0x5558e94c9550 msgr2=0x5558e9596210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.154+0000 7f31c6532b80 1 -- 172.21.15.112:0/2973593213 shutdown_connections 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.154+0000 7f31c6532b80 1 -- 172.21.15.112:0/2973593213 wait complete. 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.155+0000 7f31c6532b80 1 -- start start 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.155+0000 7f31c6532b80 1 Processor -- start 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.155+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e959a2f0 0x5558e95c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.155+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 0x5558e95bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.155+0000 7f31c6532b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e95a5ab0 0x5558e95bba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.155+0000 7f31c6532b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558e95b6610 con 0x5558e959ac10 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.155+0000 7f31c6532b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5558e95b5f40 con 0x5558e95a5ab0 2024-04-25T18:48:04.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.155+0000 7f31c6532b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5558e95b6ce0 con 0x5558e959a2f0 2024-04-25T18:48:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.155+0000 7f31c6532b80 10 client.0 _ll_register_callbacks cb 0x5558e94866d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f31b9200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e959a2f0 0x5558e95c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f31b3e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 0x5558e95bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f31b9200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e959a2f0 0x5558e95c00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:36684/0 (socket says 172.21.15.112:36684) 2024-04-25T18:48:04.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f31b9200700 1 -- 172.21.15.112:0/895815224 learned_addr learned my addr 172.21.15.112:0/895815224 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f31b9c00700 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e95a5ab0 0x5558e95bba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f31b3e00700 1 -- 172.21.15.112:0/895815224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e959a2f0 msgr2=0x5558e95c00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f31b3e00700 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e959a2f0 0x5558e95c00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f31b3e00700 1 -- 172.21.15.112:0/895815224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e95a5ab0 msgr2=0x5558e95bba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f31b3e00700 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e95a5ab0 0x5558e95bba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f31b3e00700 1 -- 172.21.15.112:0/895815224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558e961b450 con 0x5558e959ac10 2024-04-25T18:48:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f31b3e00700 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 0x5558e95bb440 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f31a0004fd0 tx=0x7f31a000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6080ba09c5bac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.156+0000 7f319aa00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:48:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.157+0000 7f319aa00700 1 -- 172.21.15.112:0/895815224 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0065070 con 0x5558e959ac10 2024-04-25T18:48:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.157+0000 7f319aa00700 1 -- 172.21.15.112:0/895815224 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f31a0087050 con 0x5558e959ac10 2024-04-25T18:48:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.157+0000 7f31c6532b80 1 -- 172.21.15.112:0/895815224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31a009ab60 con 0x5558e959ac10 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.157+0000 7f319aa00700 1 -- 172.21.15.112:0/895815224 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f31a0082050 con 0x5558e959ac10 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.157+0000 7f31c6532b80 10 client.15108 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.157+0000 7f31c6532b80 1 -- 172.21.15.112:0/895815224 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5558e961c5b0 con 0x5558e959ac10 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.157+0000 7f319aa00700 1 -- 172.21.15.112:0/895815224 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a009ab60 con 0x5558e959ac10 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.157+0000 7f319aa00700 1 -- 172.21.15.112:0/895815224 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f31a009db90 con 0x5558e959ac10 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.157+0000 7f31c6532b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.158+0000 7f31c6532b80 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.158+0000 7f31c6532b80 20 client.15108 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.158+0000 7f31c6532b80 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x5558e95b5320 0x5558e961f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.158+0000 7f31c6532b80 4 client.15108 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.158+0000 7f31c6532b80 1 -- 172.21.15.112:0/895815224 --> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] -- command(tid 0: {"prefix": "status"}) -- 0x5558e94f56d0 con 0x5558e95b5320 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.158+0000 7f31b9200700 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x5558e95b5320 0x5558e961f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f319aa00700 10 client.15108 ms_handle_connect on v2:172.21.15.171:6838/1642228003 2024-04-25T18:48:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f31b9200700 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x5558e95b5320 0x5558e961f620 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=533b513f69bdac5 server_cookie=2afbb98a3cb5ef47 in_seq=0 out_seq=0 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f319aa00700 1 -- 172.21.15.112:0/895815224 <== mds.2 v2:172.21.15.171:6838/1642228003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5558e94f56d0 con 0x5558e95b5320 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f319aa00700 10 client.15108 handle_command_reply: tid=0 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f31c6532b80 1 client.15108 shutdown 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f31c6532b80 1 -- 172.21.15.112:0/895815224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 msgr2=0x5558e95bb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f31c6532b80 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 0x5558e95bb440 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f31a0004fd0 tx=0x7f31a000cd70 comp rx=0 tx=0).stop 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f31c6532b80 1 -- 172.21.15.112:0/895815224 shutdown_connections 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f31c6532b80 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x5558e95b5320 0x5558e961f620 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f31c6532b80 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558e95a5ab0 0x5558e95bba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f31c6532b80 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5558e959a2f0 0x5558e95c00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f31c6532b80 1 --2- 172.21.15.112:0/895815224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558e959ac10 0x5558e95bb440 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f319aa00700 0 client.15108 ms_handle_reset on v2:172.21.15.171:6838/1642228003 2024-04-25T18:48:04.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.159+0000 7f31c6532b80 1 -- 172.21.15.112:0/895815224 >> 172.21.15.112:0/895815224 conn(0x5558e94c9550 msgr2=0x5558e95a2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.160+0000 7f31c6532b80 1 -- 172.21.15.112:0/895815224 shutdown_connections 2024-04-25T18:48:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.160+0000 7f31c6532b80 1 -- 172.21.15.112:0/895815224 wait complete. 2024-04-25T18:48:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.160+0000 7f31c6532b80 20 client.15108 trim_cache size 0 max 16384 2024-04-25T18:48:04.164 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:48:04.164 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:48:04.164 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-04-25T18:48:04.164 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24493, 2024-04-25T18:48:04.165 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T18:48:04.165 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T18:48:04.165 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T18:48:04.165 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 265.93886536500003, 2024-04-25T18:48:04.165 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 26, 2024-04-25T18:48:04.165 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 95, 2024-04-25T18:48:04.165 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 95, 2024-04-25T18:48:04.165 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 320.274086592 2024-04-25T18:48:04.165 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:48:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 -- 172.21.15.112:0/1213852352 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f319c0a3f60 msgr2=0x7f319c0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f319c0a3f60 0x7f319c0a6420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f31a8006010 tx=0x7f31a800b280 comp rx=0 tx=0).stop 2024-04-25T18:48:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 -- 172.21.15.112:0/1213852352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31b4103450 msgr2=0x7f31b4075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31b4103450 0x7f31b4075280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f31a000e9f0 tx=0x7f31a000ed00 comp rx=0 tx=0).stop 2024-04-25T18:48:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 -- 172.21.15.112:0/1213852352 shutdown_connections 2024-04-25T18:48:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f319c0a3f60 0x7f319c0a6420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31b4108a80 0x7f31b40757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31b4102a90 0x7f31b4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 --2- 172.21.15.112:0/1213852352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31b4103450 0x7f31b4075280 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 -- 172.21.15.112:0/1213852352 >> 172.21.15.112:0/1213852352 conn(0x7f31b4079bf0 msgr2=0x7f31b4101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 -- 172.21.15.112:0/1213852352 shutdown_connections 2024-04-25T18:48:04.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:04.161+0000 7f31bbc00700 1 -- 172.21.15.112:0/1213852352 wait complete. 2024-04-25T18:48:04.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:04 smithi171 ceph-mon[33266]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T18:48:04.701 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:04 smithi112 ceph-mon[27864]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T18:48:04.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:04 smithi161 ceph-mon[32609]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T18:48:05.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.a status 2024-04-25T18:48:05.725 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:48:06.435 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:06 smithi112 ceph-mon[27864]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-25T18:48:06.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:06 smithi171 ceph-mon[33266]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-25T18:48:06.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:06 smithi161 ceph-mon[32609]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-25T18:48:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.953+0000 7f2c47000640 1 -- 172.21.15.112:0/3172788837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c40100fd0 msgr2=0x7f2c401013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.953+0000 7f2c47000640 1 --2- 172.21.15.112:0/3172788837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c40100fd0 0x7f2c401013b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009930 tx=0x7f2c3002a4d0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.954+0000 7f2c47000640 1 -- 172.21.15.112:0/3172788837 shutdown_connections 2024-04-25T18:48:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.954+0000 7f2c47000640 1 --2- 172.21.15.112:0/3172788837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c401063d0 0x7f2c40106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.954+0000 7f2c47000640 1 --2- 172.21.15.112:0/3172788837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c401018f0 0x7f2c40105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.954+0000 7f2c47000640 1 --2- 172.21.15.112:0/3172788837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c40100fd0 0x7f2c401013b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.954+0000 7f2c47000640 1 -- 172.21.15.112:0/3172788837 >> 172.21.15.112:0/3172788837 conn(0x7f2c40077f60 msgr2=0x7f2c40078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.954+0000 7f2c47000640 1 -- 172.21.15.112:0/3172788837 shutdown_connections 2024-04-25T18:48:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.954+0000 7f2c47000640 1 -- 172.21.15.112:0/3172788837 wait complete. 2024-04-25T18:48:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.955+0000 7f2c47000640 1 Processor -- start 2024-04-25T18:48:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.955+0000 7f2c47000640 1 -- start start 2024-04-25T18:48:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.955+0000 7f2c47000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c40100fd0 0x7f2c401a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.955+0000 7f2c47000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c401018f0 0x7f2c401a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.955+0000 7f2c47000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c401063d0 0x7f2c4019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.955+0000 7f2c47000640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40118820 con 0x7f2c401018f0 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.955+0000 7f2c47000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401186a0 con 0x7f2c40100fd0 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.955+0000 7f2c47000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401189a0 con 0x7f2c401063d0 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.955+0000 7f2c3fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c40100fd0 0x7f2c401a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.956+0000 7f2c3fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c40100fd0 0x7f2c401a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:37144/0 (socket says 172.21.15.112:37144) 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.956+0000 7f2c3fe00640 1 -- 172.21.15.112:0/3843810734 learned_addr learned my addr 172.21.15.112:0/3843810734 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.956+0000 7f2c45000640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c401063d0 0x7f2c4019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.956+0000 7f2c3f400640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c401018f0 0x7f2c401a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.956+0000 7f2c3fe00640 1 -- 172.21.15.112:0/3843810734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c401063d0 msgr2=0x7f2c4019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.956+0000 7f2c3fe00640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c401063d0 0x7f2c4019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.956+0000 7f2c3fe00640 1 -- 172.21.15.112:0/3843810734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c401018f0 msgr2=0x7f2c401a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.956+0000 7f2c3fe00640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c401018f0 0x7f2c401a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.956+0000 7f2c3fe00640 1 -- 172.21.15.112:0/3843810734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4019f280 con 0x7f2c40100fd0 2024-04-25T18:48:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.956+0000 7f2c3f400640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c401018f0 0x7f2c401a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:48:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.956+0000 7f2c3fe00640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c40100fd0 0x7f2c401a4950 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c30005ab0 tx=0x7f2c30003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.957+0000 7f2c3cc00640 1 -- 172.21.15.112:0/3843810734 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3003f070 con 0x7f2c40100fd0 2024-04-25T18:48:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.957+0000 7f2c3cc00640 1 -- 172.21.15.112:0/3843810734 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c30031b50 con 0x7f2c40100fd0 2024-04-25T18:48:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.957+0000 7f2c3cc00640 1 -- 172.21.15.112:0/3843810734 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c30035030 con 0x7f2c40100fd0 2024-04-25T18:48:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.957+0000 7f2c47000640 1 -- 172.21.15.112:0/3843810734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4019f510 con 0x7f2c40100fd0 2024-04-25T18:48:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.957+0000 7f2c47000640 1 -- 172.21.15.112:0/3843810734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c401ab810 con 0x7f2c40100fd0 2024-04-25T18:48:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.961+0000 7f2c49fe4740 1 Processor -- start 2024-04-25T18:48:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.961+0000 7f2c49fe4740 1 -- start start 2024-04-25T18:48:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.961+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c3f820 0x55e389c3fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.961+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c40140 0x55e389c4a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.961+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c4aa00 0x55e389c4cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.961+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e389ba5900 con 0x55e389c40140 2024-04-25T18:48:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.961+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e389b4bef0 con 0x55e389c4aa00 2024-04-25T18:48:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.961+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e389ba18c0 con 0x55e389c3f820 2024-04-25T18:48:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c45000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c3f820 0x55e389c3fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c45000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c3f820 0x55e389c3fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:36704/0 (socket says 172.21.15.112:36704) 2024-04-25T18:48:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c45000640 1 -- 172.21.15.112:0/1142361374 learned_addr learned my addr 172.21.15.112:0/1142361374 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c3f400640 1 --2- 172.21.15.112:0/1142361374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c4aa00 0x55e389c4cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c3cc00640 1 -- 172.21.15.112:0/3843810734 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f2c30042070 con 0x7f2c40100fd0 2024-04-25T18:48:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c3fe00640 1 --2- 172.21.15.112:0/1142361374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c40140 0x55e389c4a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c45000640 1 -- 172.21.15.112:0/1142361374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c4aa00 msgr2=0x55e389c4cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c45000640 1 --2- 172.21.15.112:0/1142361374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c4aa00 0x55e389c4cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c45000640 1 -- 172.21.15.112:0/1142361374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c40140 msgr2=0x55e389c4a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c45000640 1 --2- 172.21.15.112:0/1142361374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c40140 0x55e389c4a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c45000640 1 -- 172.21.15.112:0/1142361374 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3898b8210 con 0x55e389c3f820 2024-04-25T18:48:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c3f400640 1 --2- 172.21.15.112:0/1142361374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c4aa00 0x55e389c4cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:48:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.962+0000 7f2c3fe00640 1 --2- 172.21.15.112:0/1142361374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c40140 0x55e389c4a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:48:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.963+0000 7f2c45000640 1 --2- 172.21.15.112:0/1142361374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c3f820 0x55e389c3fc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009f50 tx=0x7f2c380118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d24e90717ea33d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.963+0000 7f2c35600640 1 -- 172.21.15.112:0/1142361374 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38004d20 con 0x55e389c3f820 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.963+0000 7f2c35600640 1 -- 172.21.15.112:0/1142361374 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c38004050 con 0x55e389c3f820 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.963+0000 7f2c35600640 1 -- 172.21.15.112:0/1142361374 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38005260 con 0x55e389c3f820 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.964+0000 7f2c3cc00640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a4070 0x7f2c240a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.964+0000 7f2c3cc00640 1 -- 172.21.15.112:0/3843810734 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2c300f0cf0 con 0x7f2c40100fd0 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.964+0000 7f2c3fe00640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a4070 0x7f2c240a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.964+0000 7f2c49fe4740 1 -- 172.21.15.112:0/1142361374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c3f820 msgr2=0x55e389c3fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.964+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/1142361374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c3f820 0x55e389c3fc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009f50 tx=0x7f2c380118d0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.964+0000 7f2c49fe4740 1 -- 172.21.15.112:0/1142361374 shutdown_connections 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.964+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/1142361374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c4aa00 0x55e389c4cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.964+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/1142361374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c40140 0x55e389c4a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.964+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/1142361374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c3f820 0x55e389c3fc00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.964+0000 7f2c49fe4740 1 -- 172.21.15.112:0/1142361374 >> 172.21.15.112:0/1142361374 conn(0x55e389c3bd30 msgr2=0x55e389c3c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.965+0000 7f2c49fe4740 1 -- 172.21.15.112:0/1142361374 shutdown_connections 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.965+0000 7f2c49fe4740 1 -- 172.21.15.112:0/1142361374 wait complete. 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.965+0000 7f2c3fe00640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a4070 0x7f2c240a6530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c300ee260 tx=0x7f2c30031f50 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.965+0000 7f2c49fe4740 1 -- start start 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.966+0000 7f2c49fe4740 1 Processor -- start 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.966+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c3f820 0x55e389c47950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.966+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 0x55e389c47e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.966+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c4aa00 0x55e389c483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.966+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e389c4d630 con 0x55e389c3f820 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.966+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e389b5f310 con 0x55e389c4aa00 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.966+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e389c4d7b0 con 0x55e389c40140 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.966+0000 7f2c49fe4740 10 client.0 _ll_register_callbacks cb 0x55e389b2dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.967+0000 7f2c3f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 0x55e389c47e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.967+0000 7f2c3f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 0x55e389c47e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:36708/0 (socket says 172.21.15.112:36708) 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.967+0000 7f2c3f400640 1 -- 172.21.15.112:0/4066493323 learned_addr learned my addr 172.21.15.112:0/4066493323 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.967+0000 7f2c45000640 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c3f820 0x55e389c47950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.967+0000 7f2c3f400640 1 -- 172.21.15.112:0/4066493323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c4aa00 msgr2=0x55e389c483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.967+0000 7f2c3f400640 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c4aa00 0x55e389c483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.967+0000 7f2c3f400640 1 -- 172.21.15.112:0/4066493323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c3f820 msgr2=0x55e389c47950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.967+0000 7f2c3f400640 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c3f820 0x55e389c47950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.967+0000 7f2c3f400640 1 -- 172.21.15.112:0/4066493323 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e389984620 con 0x55e389c40140 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.967+0000 7f2c3f400640 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 0x55e389c47e90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c013790 tx=0x7f2c2c013c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c9628df9a53d652 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.967+0000 7f2c2a000640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.968+0000 7f2c2a000640 1 -- 172.21.15.112:0/4066493323 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c011dd0 con 0x55e389c40140 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.968+0000 7f2c49fe4740 1 -- 172.21.15.112:0/4066493323 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c38004050 con 0x55e389c40140 2024-04-25T18:48:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.968+0000 7f2c2a000640 1 -- 172.21.15.112:0/4066493323 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2c2c005020 con 0x55e389c40140 2024-04-25T18:48:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.968+0000 7f2c2a000640 1 -- 172.21.15.112:0/4066493323 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c2c005220 con 0x55e389c40140 2024-04-25T18:48:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.968+0000 7f2c49fe4740 10 client.25252 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.968+0000 7f2c49fe4740 1 -- 172.21.15.112:0/4066493323 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e389c59860 con 0x55e389c40140 2024-04-25T18:48:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.968+0000 7f2c2a000640 1 -- 172.21.15.112:0/4066493323 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c38004050 con 0x55e389c40140 2024-04-25T18:48:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.969+0000 7f2c2a000640 1 -- 172.21.15.112:0/4066493323 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2c2c01a030 con 0x55e389c40140 2024-04-25T18:48:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.969+0000 7f2c49fe4740 10 client.25252 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.969+0000 7f2c49fe4740 10 client.25252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:48:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.969+0000 7f2c49fe4740 20 client.25252 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.969+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e389c5ad90 0x55e389c5d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.969+0000 7f2c49fe4740 4 client.25252 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:48:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.969+0000 7f2c49fe4740 1 -- 172.21.15.112:0/4066493323 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e389bb98b0 con 0x55e389c5ad90 2024-04-25T18:48:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.969+0000 7f2c45000640 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e389c5ad90 0x55e389c5d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.970+0000 7f2c45000640 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e389c5ad90 0x55e389c5d180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6146471a14b4878a server_cookie=4a4b53050542e140 in_seq=0 out_seq=0 2024-04-25T18:48:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.970+0000 7f2c2a000640 10 client.25252 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.971+0000 7f2c2a000640 1 -- 172.21.15.112:0/4066493323 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e389bb98b0 con 0x55e389c5ad90 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.971+0000 7f2c2a000640 10 client.25252 handle_command_reply: tid=0 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.971+0000 7f2c49fe4740 1 client.25252 shutdown 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.971+0000 7f2c49fe4740 1 -- 172.21.15.112:0/4066493323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 msgr2=0x55e389c47e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.971+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 0x55e389c47e90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c013790 tx=0x7f2c2c013c50 comp rx=0 tx=0).stop 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.971+0000 7f2c49fe4740 1 -- 172.21.15.112:0/4066493323 shutdown_connections 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.971+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c4aa00 0x55e389c483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.971+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 0x55e389c47e90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.971+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e389c5ad90 0x55e389c5d180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.971+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/4066493323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c3f820 0x55e389c47950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.971+0000 7f2c49fe4740 1 -- 172.21.15.112:0/4066493323 >> 172.21.15.112:0/4066493323 conn(0x55e389c3bd30 msgr2=0x55e389c41ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.972+0000 7f2c2a000640 0 client.25252 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.972+0000 7f2c49fe4740 1 -- 172.21.15.112:0/4066493323 shutdown_connections 2024-04-25T18:48:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.972+0000 7f2c49fe4740 1 -- 172.21.15.112:0/4066493323 wait complete. 2024-04-25T18:48:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.972+0000 7f2c49fe4740 20 client.25252 trim_cache size 0 max 16384 2024-04-25T18:48:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.998+0000 7f2c49fe4740 1 Processor -- start 2024-04-25T18:48:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.999+0000 7f2c49fe4740 1 -- start start 2024-04-25T18:48:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.999+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c3f820 0x55e389c564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.999+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 0x55e389c62a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.999+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c4aa00 0x55e389c62fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.999+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401b6ae0 con 0x55e389c4aa00 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.999+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401b43b0 con 0x55e389c3f820 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.999+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401b2200 con 0x55e389c40140 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.999+0000 7f2c45000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c3f820 0x55e389c564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:07.999+0000 7f2c3f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 0x55e389c62a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.000+0000 7f2c45000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c3f820 0x55e389c564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:37188/0 (socket says 172.21.15.112:37188) 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.000+0000 7f2c45000640 1 -- 172.21.15.112:0/467523899 learned_addr learned my addr 172.21.15.112:0/467523899 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.000+0000 7f2c45000640 1 --2- 172.21.15.112:0/467523899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c4aa00 0x55e389c62fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.000+0000 7f2c3f400640 1 -- 172.21.15.112:0/467523899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c3f820 msgr2=0x55e389c564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.000+0000 7f2c3f400640 1 --2- 172.21.15.112:0/467523899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c3f820 0x55e389c564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.000+0000 7f2c3f400640 1 -- 172.21.15.112:0/467523899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c4aa00 msgr2=0x55e389c62fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.000+0000 7f2c3f400640 1 --2- 172.21.15.112:0/467523899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c4aa00 0x55e389c62fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.000+0000 7f2c3f400640 1 -- 172.21.15.112:0/467523899 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e389c45ec0 con 0x55e389c40140 2024-04-25T18:48:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.000+0000 7f2c3f400640 1 --2- 172.21.15.112:0/467523899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 0x55e389c62a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0244b0 tx=0x7f2c2c024720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5b232c435810876 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.001+0000 7f2c34c00640 1 -- 172.21.15.112:0/467523899 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c000e60 con 0x55e389c40140 2024-04-25T18:48:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.001+0000 7f2c34c00640 1 -- 172.21.15.112:0/467523899 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2c011dd0 con 0x55e389c40140 2024-04-25T18:48:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.001+0000 7f2c34c00640 1 -- 172.21.15.112:0/467523899 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c004990 con 0x55e389c40140 2024-04-25T18:48:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.001+0000 7f2c49fe4740 1 -- 172.21.15.112:0/467523899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 msgr2=0x55e389c62a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.001+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/467523899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 0x55e389c62a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0244b0 tx=0x7f2c2c024720 comp rx=0 tx=0).stop 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.002+0000 7f2c49fe4740 1 -- 172.21.15.112:0/467523899 shutdown_connections 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.002+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/467523899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c4aa00 0x55e389c62fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.002+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/467523899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c40140 0x55e389c62a90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.002+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/467523899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c3f820 0x55e389c564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.002+0000 7f2c49fe4740 1 -- 172.21.15.112:0/467523899 >> 172.21.15.112:0/467523899 conn(0x55e389c3bd30 msgr2=0x55e389c3c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.002+0000 7f2c49fe4740 1 -- 172.21.15.112:0/467523899 shutdown_connections 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.002+0000 7f2c49fe4740 1 -- 172.21.15.112:0/467523899 wait complete. 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.002+0000 7f2c49fe4740 1 -- start start 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.002+0000 7f2c49fe4740 1 Processor -- start 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c3f820 0x55e389c52e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c40140 0x55e389c4f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c49fe4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c4aa00 0x55e389c4ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401b2870 con 0x55e389c3f820 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401b3ce0 con 0x55e389c40140 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c49fe4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401b2f40 con 0x55e389c4aa00 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c49fe4740 10 client.0 _ll_register_callbacks cb 0x7f2c2c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c3f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c40140 0x55e389c4f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c45000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c3f820 0x55e389c52e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c3f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c40140 0x55e389c4f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:37190/0 (socket says 172.21.15.112:37190) 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c3f400640 1 -- 172.21.15.112:0/3874363824 learned_addr learned my addr 172.21.15.112:0/3874363824 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.003+0000 7f2c45000640 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c4aa00 0x55e389c4ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.004+0000 7f2c3f400640 1 -- 172.21.15.112:0/3874363824 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c4aa00 msgr2=0x55e389c4ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.004+0000 7f2c3f400640 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c4aa00 0x55e389c4ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.004+0000 7f2c3f400640 1 -- 172.21.15.112:0/3874363824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c3f820 msgr2=0x55e389c52e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.004+0000 7f2c3f400640 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c3f820 0x55e389c52e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.004+0000 7f2c3f400640 1 -- 172.21.15.112:0/3874363824 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e389c50710 con 0x55e389c40140 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.004+0000 7f2c3f400640 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c40140 0x55e389c4f9d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0245d0 tx=0x7f2c2c024b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30ffd53914108309 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.004+0000 7f2c2a000640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.004+0000 7f2c2a000640 1 -- 172.21.15.112:0/3874363824 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0117b0 con 0x55e389c40140 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.004+0000 7f2c2a000640 1 -- 172.21.15.112:0/3874363824 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2c2c0048e0 con 0x55e389c40140 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.004+0000 7f2c49fe4740 1 -- 172.21.15.112:0/3874363824 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c2c011dd0 con 0x55e389c40140 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.005+0000 7f2c2a000640 1 -- 172.21.15.112:0/3874363824 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c2c047050 con 0x55e389c40140 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.005+0000 7f2c49fe4740 10 client.25256 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.005+0000 7f2c49fe4740 1 -- 172.21.15.112:0/3874363824 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e389cc02e0 con 0x55e389c40140 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.005+0000 7f2c2a000640 1 -- 172.21.15.112:0/3874363824 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2c011dd0 con 0x55e389c40140 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.005+0000 7f2c2a000640 1 -- 172.21.15.112:0/3874363824 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2c2c02d070 con 0x55e389c40140 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.005+0000 7f2c49fe4740 10 client.25256 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.005+0000 7f2c49fe4740 10 client.25256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:48:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.005+0000 7f2c49fe4740 20 client.25256 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.005+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e389cc0990 0x55e389cc2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.005+0000 7f2c49fe4740 4 client.25256 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:48:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.005+0000 7f2c49fe4740 1 -- 172.21.15.112:0/3874363824 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e389bb3320 con 0x55e389cc0990 2024-04-25T18:48:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.006+0000 7f2c45000640 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e389cc0990 0x55e389cc2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.006+0000 7f2c2a000640 10 client.25256 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.006+0000 7f2c45000640 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e389cc0990 0x55e389cc2db0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c254f109a3e8e112 server_cookie=87c62f34297397d8 in_seq=0 out_seq=0 2024-04-25T18:48:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c2a000640 1 -- 172.21.15.112:0/3874363824 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e389bb3320 con 0x55e389cc0990 2024-04-25T18:48:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c2a000640 10 client.25256 handle_command_reply: tid=0 2024-04-25T18:48:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c49fe4740 1 client.25256 shutdown 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c49fe4740 1 -- 172.21.15.112:0/3874363824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c40140 msgr2=0x55e389c4f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c40140 0x55e389c4f9d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0245d0 tx=0x7f2c2c024b40 comp rx=0 tx=0).stop 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c49fe4740 1 -- 172.21.15.112:0/3874363824 shutdown_connections 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e389c4aa00 0x55e389c4ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e389c40140 0x55e389c4f9d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e389cc0990 0x55e389cc2db0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c49fe4740 1 --2- 172.21.15.112:0/3874363824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e389c3f820 0x55e389c52e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c2a000640 0 client.25256 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.016+0000 7f2c49fe4740 1 -- 172.21.15.112:0/3874363824 >> 172.21.15.112:0/3874363824 conn(0x55e389c3bd30 msgr2=0x55e389c42c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.017+0000 7f2c49fe4740 1 -- 172.21.15.112:0/3874363824 shutdown_connections 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.017+0000 7f2c49fe4740 1 -- 172.21.15.112:0/3874363824 wait complete. 2024-04-25T18:48:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.017+0000 7f2c49fe4740 20 client.25256 trim_cache size 0 max 16384 2024-04-25T18:48:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:48:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:48:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:48:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:48:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.017+0000 7f2c47000640 1 -- 172.21.15.112:0/3843810734 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a4070 msgr2=0x7f2c240a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.017+0000 7f2c47000640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a4070 0x7f2c240a6530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c300ee260 tx=0x7f2c30031f50 comp rx=0 tx=0).stop 2024-04-25T18:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.017+0000 7f2c47000640 1 -- 172.21.15.112:0/3843810734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c40100fd0 msgr2=0x7f2c401a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.017+0000 7f2c47000640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c40100fd0 0x7f2c401a4950 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c30005ab0 tx=0x7f2c30003b90 comp rx=0 tx=0).stop 2024-04-25T18:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.018+0000 7f2c47000640 1 -- 172.21.15.112:0/3843810734 shutdown_connections 2024-04-25T18:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.018+0000 7f2c47000640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a4070 0x7f2c240a6530 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.018+0000 7f2c47000640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c401063d0 0x7f2c4019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.018+0000 7f2c47000640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c401018f0 0x7f2c401a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.018+0000 7f2c47000640 1 --2- 172.21.15.112:0/3843810734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c40100fd0 0x7f2c401a4950 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.018+0000 7f2c47000640 1 -- 172.21.15.112:0/3843810734 >> 172.21.15.112:0/3843810734 conn(0x7f2c40077f60 msgr2=0x7f2c40103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.018+0000 7f2c47000640 1 -- 172.21.15.112:0/3843810734 shutdown_connections 2024-04-25T18:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:08.018+0000 7f2c47000640 1 -- 172.21.15.112:0/3843810734 wait complete. 2024-04-25T18:48:08.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.114+0000 7fdcc0400700 1 -- 172.21.15.112:0/2208644392 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcb806b0f0 msgr2=0x7fdcb810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.114+0000 7fdcc0400700 1 --2- 172.21.15.112:0/2208644392 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcb806b0f0 0x7fdcb810f900 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009f90 tx=0x7fdcb000b330 comp rx=0 tx=0).stop 2024-04-25T18:48:08.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.114+0000 7fdcc0400700 1 -- 172.21.15.112:0/2208644392 shutdown_connections 2024-04-25T18:48:08.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.114+0000 7fdcc0400700 1 --2- 172.21.15.112:0/2208644392 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcb806b0f0 0x7fdcb810f900 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.114+0000 7fdcc0400700 1 --2- 172.21.15.112:0/2208644392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb8108a30 0x7fdcb8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.114+0000 7fdcc0400700 1 --2- 172.21.15.112:0/2208644392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcb806a750 0x7fdcb806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.114+0000 7fdcc0400700 1 -- 172.21.15.112:0/2208644392 >> 172.21.15.112:0/2208644392 conn(0x7fdcb80788f0 msgr2=0x7fdcb8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:08.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.114+0000 7fdcc0400700 1 -- 172.21.15.112:0/2208644392 shutdown_connections 2024-04-25T18:48:08.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.114+0000 7fdcc0400700 1 -- 172.21.15.112:0/2208644392 wait complete. 2024-04-25T18:48:08.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.116+0000 7fdcc0400700 1 Processor -- start 2024-04-25T18:48:08.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.116+0000 7fdcc0400700 1 -- start start 2024-04-25T18:48:08.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.116+0000 7fdcc0400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcb806a750 0x7fdcb81990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.116+0000 7fdcc0400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb806b0f0 0x7fdcb8199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcc0400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcb8108a30 0x7fdcb819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcc0400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8112bd0 con 0x7fdcb806a750 2024-04-25T18:48:08.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcc0400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8112a50 con 0x7fdcb806b0f0 2024-04-25T18:48:08.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcc0400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8112d50 con 0x7fdcb8108a30 2024-04-25T18:48:08.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcbda00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcb806a750 0x7fdcb81990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcbd000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb806b0f0 0x7fdcb8199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcbd000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb806b0f0 0x7fdcb8199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:37208/0 (socket says 172.21.15.112:37208) 2024-04-25T18:48:08.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcbd000700 1 -- 172.21.15.112:0/4027305044 learned_addr learned my addr 172.21.15.112:0/4027305044 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:08.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcbe400700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcb8108a30 0x7fdcb819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcbe400700 1 -- 172.21.15.112:0/4027305044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb806b0f0 msgr2=0x7fdcb8199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcbe400700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb806b0f0 0x7fdcb8199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcbe400700 1 -- 172.21.15.112:0/4027305044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcb806a750 msgr2=0x7fdcb81990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcbe400700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcb806a750 0x7fdcb81990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.117+0000 7fdcbe400700 1 -- 172.21.15.112:0/4027305044 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb819e080 con 0x7fdcb8108a30 2024-04-25T18:48:08.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.118+0000 7fdcbe400700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcb8108a30 0x7fdcb819d940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009790 tx=0x7fdcb000ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:08.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.118+0000 7fdcb6000700 1 -- 172.21.15.112:0/4027305044 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0021070 con 0x7fdcb8108a30 2024-04-25T18:48:08.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.118+0000 7fdcc0400700 1 -- 172.21.15.112:0/4027305044 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb819e310 con 0x7fdcb8108a30 2024-04-25T18:48:08.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.118+0000 7fdcc0400700 1 -- 172.21.15.112:0/4027305044 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb8101940 con 0x7fdcb8108a30 2024-04-25T18:48:08.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.119+0000 7fdcb6000700 1 -- 172.21.15.112:0/4027305044 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcb0007420 con 0x7fdcb8108a30 2024-04-25T18:48:08.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.119+0000 7fdcb6000700 1 -- 172.21.15.112:0/4027305044 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb001d900 con 0x7fdcb8108a30 2024-04-25T18:48:08.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.124+0000 7fdccad36b80 1 Processor -- start 2024-04-25T18:48:08.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.124+0000 7fdcb6000700 1 -- 172.21.15.112:0/4027305044 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fdcb0024070 con 0x7fdcb8108a30 2024-04-25T18:48:08.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.124+0000 7fdccad36b80 1 -- start start 2024-04-25T18:48:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.124+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 0x5613fae256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.125+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae25c10 0x5613fae30570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.125+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae30ab0 0x5613fae32ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.125+0000 7fdccad36b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5613fad78e40 con 0x5613fae252f0 2024-04-25T18:48:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.125+0000 7fdccad36b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5613fad1f6e0 con 0x5613fae30ab0 2024-04-25T18:48:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.125+0000 7fdccad36b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5613fad74e00 con 0x5613fae25c10 2024-04-25T18:48:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.125+0000 7fdcbd000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 0x5613fae256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.125+0000 7fdcbd000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 0x5613fae256d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34586/0 (socket says 172.21.15.112:34586) 2024-04-25T18:48:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.125+0000 7fdcbd000700 1 -- 172.21.15.112:0/3085280288 learned_addr learned my addr 172.21.15.112:0/3085280288 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:08.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.125+0000 7fdcbd000700 1 --2- 172.21.15.112:0/3085280288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae30ab0 0x5613fae32ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.125+0000 7fdcbda00700 1 --2- 172.21.15.112:0/3085280288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae25c10 0x5613fae30570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.126+0000 7fdcbd000700 1 -- 172.21.15.112:0/3085280288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae25c10 msgr2=0x5613fae30570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.126+0000 7fdcbd000700 1 --2- 172.21.15.112:0/3085280288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae25c10 0x5613fae30570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.126+0000 7fdcbd000700 1 -- 172.21.15.112:0/3085280288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae30ab0 msgr2=0x5613fae32ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.126+0000 7fdcbd000700 1 --2- 172.21.15.112:0/3085280288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae30ab0 0x5613fae32ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.126+0000 7fdcbd000700 1 -- 172.21.15.112:0/3085280288 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613fad15630 con 0x5613fae252f0 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.126+0000 7fdcbd000700 1 --2- 172.21.15.112:0/3085280288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 0x5613fae256d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdca4012e40 tx=0x7fdca4012430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac7376f0ef73393 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.126+0000 7fdcb6000700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdca00a3f60 0x7fdca00a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.126+0000 7fdcb6000700 1 -- 172.21.15.112:0/4027305044 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fdcb00d1510 con 0x7fdcb8108a30 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.127+0000 7fdcbda00700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdca00a3f60 0x7fdca00a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.127+0000 7fdcaaa00700 1 -- 172.21.15.112:0/3085280288 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4004050 con 0x5613fae252f0 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.127+0000 7fdcaaa00700 1 -- 172.21.15.112:0/3085280288 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdca4003680 con 0x5613fae252f0 2024-04-25T18:48:08.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.127+0000 7fdcaaa00700 1 -- 172.21.15.112:0/3085280288 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca400f6a0 con 0x5613fae252f0 2024-04-25T18:48:08.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.128+0000 7fdcbda00700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdca00a3f60 0x7fdca00a6420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdcac004510 tx=0x7fdcac018040 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:08.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.128+0000 7fdccad36b80 1 -- 172.21.15.112:0/3085280288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 msgr2=0x5613fae256d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.128+0000 7fdccad36b80 1 --2- 172.21.15.112:0/3085280288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 0x5613fae256d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdca4012e40 tx=0x7fdca4012430 comp rx=0 tx=0).stop 2024-04-25T18:48:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.128+0000 7fdccad36b80 1 -- 172.21.15.112:0/3085280288 shutdown_connections 2024-04-25T18:48:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.128+0000 7fdccad36b80 1 --2- 172.21.15.112:0/3085280288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae30ab0 0x5613fae32ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.128+0000 7fdccad36b80 1 --2- 172.21.15.112:0/3085280288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 0x5613fae256d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.128+0000 7fdccad36b80 1 --2- 172.21.15.112:0/3085280288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae25c10 0x5613fae30570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.128+0000 7fdccad36b80 1 -- 172.21.15.112:0/3085280288 >> 172.21.15.112:0/3085280288 conn(0x5613fad54550 msgr2=0x5613fae21790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.128+0000 7fdccad36b80 1 -- 172.21.15.112:0/3085280288 shutdown_connections 2024-04-25T18:48:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.128+0000 7fdccad36b80 1 -- 172.21.15.112:0/3085280288 wait complete. 2024-04-25T18:48:08.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.129+0000 7fdccad36b80 1 -- start start 2024-04-25T18:48:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.129+0000 7fdccad36b80 1 Processor -- start 2024-04-25T18:48:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 0x5613fae30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 0x5613fae2d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 0x5613fae2da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdccad36b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5613fae335a0 con 0x5613fae30ab0 2024-04-25T18:48:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdccad36b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5613fad32810 con 0x5613fae25c10 2024-04-25T18:48:08.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdccad36b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5613fae33720 con 0x5613fae252f0 2024-04-25T18:48:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdccad36b80 10 client.0 _ll_register_callbacks cb 0x5613fad116d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdcbe400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 0x5613fae2da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdcbda00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 0x5613fae30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdcbe400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 0x5613fae2da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34600/0 (socket says 172.21.15.112:34600) 2024-04-25T18:48:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdcbda00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 0x5613fae30200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:36762/0 (socket says 172.21.15.112:36762) 2024-04-25T18:48:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.130+0000 7fdcbe400700 1 -- 172.21.15.112:0/1117060195 learned_addr learned my addr 172.21.15.112:0/1117060195 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:08.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.131+0000 7fdcbd000700 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 0x5613fae2d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.131+0000 7fdcbda00700 1 -- 172.21.15.112:0/1117060195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 msgr2=0x5613fae2d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.131+0000 7fdcbda00700 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 0x5613fae2d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.131+0000 7fdcbda00700 1 -- 172.21.15.112:0/1117060195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 msgr2=0x5613fae2da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.131+0000 7fdcbda00700 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 0x5613fae2da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.131+0000 7fdcbda00700 1 -- 172.21.15.112:0/1117060195 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613facc6880 con 0x5613fae252f0 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.131+0000 7fdcbe400700 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 0x5613fae2da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.131+0000 7fdcbda00700 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 0x5613fae30200 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0046e0 tx=0x7fdcac0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be7b0dc2079f524d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.131+0000 7fdc9f400700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.132+0000 7fdc9f400700 1 -- 172.21.15.112:0/1117060195 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac0229f0 con 0x5613fae252f0 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.132+0000 7fdc9f400700 1 -- 172.21.15.112:0/1117060195 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fdcac022b90 con 0x5613fae252f0 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.132+0000 7fdccad36b80 1 -- 172.21.15.112:0/1117060195 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdca4003680 con 0x5613fae252f0 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.132+0000 7fdc9f400700 1 -- 172.21.15.112:0/1117060195 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdcac03a050 con 0x5613fae252f0 2024-04-25T18:48:08.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.132+0000 7fdccad36b80 10 client.25270 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:08.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.132+0000 7fdccad36b80 1 -- 172.21.15.112:0/1117060195 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5613fad0faa0 con 0x5613fae252f0 2024-04-25T18:48:08.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.132+0000 7fdc9f400700 1 -- 172.21.15.112:0/1117060195 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdca4003680 con 0x5613fae252f0 2024-04-25T18:48:08.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.133+0000 7fdc9f400700 1 -- 172.21.15.112:0/1117060195 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdcac0213e0 con 0x5613fae252f0 2024-04-25T18:48:08.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.133+0000 7fdccad36b80 10 client.25270 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:08.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.133+0000 7fdccad36b80 10 client.25270 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T18:48:08.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.133+0000 7fdccad36b80 20 client.25270 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:08.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.133+0000 7fdccad36b80 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5613fae404b0 0x5613fae41920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.133+0000 7fdccad36b80 4 client.25270 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:48:08.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.133+0000 7fdccad36b80 1 -- 172.21.15.112:0/1117060195 --> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613fad86630 con 0x5613fae404b0 2024-04-25T18:48:08.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.133+0000 7fdcbd000700 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5613fae404b0 0x5613fae41920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.134+0000 7fdcbd000700 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5613fae404b0 0x5613fae41920 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7977aed0db0f4041 server_cookie=4ca32e3ba7d524e8 in_seq=0 out_seq=0 2024-04-25T18:48:08.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.134+0000 7fdc9f400700 10 client.25270 ms_handle_connect on v2:172.21.15.112:6840/1028859850 2024-04-25T18:48:08.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.135+0000 7fdc9f400700 1 -- 172.21.15.112:0/1117060195 <== mds.1 v2:172.21.15.112:6840/1028859850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613fad86630 con 0x5613fae404b0 2024-04-25T18:48:08.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.135+0000 7fdc9f400700 10 client.25270 handle_command_reply: tid=0 2024-04-25T18:48:08.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.135+0000 7fdccad36b80 1 client.25270 shutdown 2024-04-25T18:48:08.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdccad36b80 1 -- 172.21.15.112:0/1117060195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 msgr2=0x5613fae30200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdccad36b80 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 0x5613fae30200 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0046e0 tx=0x7fdcac0047c0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdccad36b80 1 -- 172.21.15.112:0/1117060195 shutdown_connections 2024-04-25T18:48:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdccad36b80 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 0x5613fae2da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdccad36b80 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 0x5613fae30200 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdccad36b80 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5613fae404b0 0x5613fae41920 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdccad36b80 1 --2- 172.21.15.112:0/1117060195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 0x5613fae2d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdccad36b80 1 -- 172.21.15.112:0/1117060195 >> 172.21.15.112:0/1117060195 conn(0x5613fad54550 msgr2=0x5613fae20950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdc9f400700 0 client.25270 ms_handle_reset on v2:172.21.15.112:6840/1028859850 2024-04-25T18:48:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdccad36b80 1 -- 172.21.15.112:0/1117060195 shutdown_connections 2024-04-25T18:48:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdccad36b80 1 -- 172.21.15.112:0/1117060195 wait complete. 2024-04-25T18:48:08.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.136+0000 7fdccad36b80 20 client.25270 trim_cache size 0 max 16384 2024-04-25T18:48:08.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.161+0000 7fdccad36b80 1 Processor -- start 2024-04-25T18:48:08.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.161+0000 7fdccad36b80 1 -- start start 2024-04-25T18:48:08.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.161+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 0x5613fae2db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.161+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 0x5613fae2e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.161+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 0x5613fae2c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.161+0000 7fdccad36b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5613fae3fc60 con 0x5613fae30ab0 2024-04-25T18:48:08.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.161+0000 7fdccad36b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5613fae40330 con 0x5613fae25c10 2024-04-25T18:48:08.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.161+0000 7fdccad36b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5613fae40a00 con 0x5613fae252f0 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.162+0000 7fdcbd000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 0x5613fae2e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.162+0000 7fdcbd000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 0x5613fae2e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:37230/0 (socket says 172.21.15.112:37230) 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.162+0000 7fdcbd000700 1 -- 172.21.15.112:0/569049652 learned_addr learned my addr 172.21.15.112:0/569049652 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.162+0000 7fdcbda00700 1 --2- 172.21.15.112:0/569049652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 0x5613fae2db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.162+0000 7fdcbe400700 1 --2- 172.21.15.112:0/569049652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 0x5613fae2c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.162+0000 7fdcbd000700 1 -- 172.21.15.112:0/569049652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 msgr2=0x5613fae2db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.162+0000 7fdcbd000700 1 --2- 172.21.15.112:0/569049652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 0x5613fae2db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.162+0000 7fdcbd000700 1 -- 172.21.15.112:0/569049652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 msgr2=0x5613fae2c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.162+0000 7fdcbd000700 1 --2- 172.21.15.112:0/569049652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 0x5613fae2c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.162+0000 7fdcbd000700 1 -- 172.21.15.112:0/569049652 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613fae22d00 con 0x5613fae25c10 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.162+0000 7fdcbd000700 1 --2- 172.21.15.112:0/569049652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 0x5613fae2e160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009db0 tx=0x7fdca4014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5ee5faef1e0ecfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.163+0000 7fdcaa000700 1 -- 172.21.15.112:0/569049652 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4003700 con 0x5613fae25c10 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.163+0000 7fdcaa000700 1 -- 172.21.15.112:0/569049652 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdca4003e40 con 0x5613fae25c10 2024-04-25T18:48:08.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.163+0000 7fdcaa000700 1 -- 172.21.15.112:0/569049652 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca40053b0 con 0x5613fae25c10 2024-04-25T18:48:08.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.164+0000 7fdccad36b80 1 -- 172.21.15.112:0/569049652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 msgr2=0x5613fae2e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.164+0000 7fdccad36b80 1 --2- 172.21.15.112:0/569049652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 0x5613fae2e160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009db0 tx=0x7fdca4014040 comp rx=0 tx=0).stop 2024-04-25T18:48:08.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.164+0000 7fdccad36b80 1 -- 172.21.15.112:0/569049652 shutdown_connections 2024-04-25T18:48:08.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.164+0000 7fdccad36b80 1 --2- 172.21.15.112:0/569049652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae30ab0 0x5613fae2c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.164+0000 7fdccad36b80 1 --2- 172.21.15.112:0/569049652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae252f0 0x5613fae2db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.164+0000 7fdccad36b80 1 --2- 172.21.15.112:0/569049652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae25c10 0x5613fae2e160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.164+0000 7fdccad36b80 1 -- 172.21.15.112:0/569049652 >> 172.21.15.112:0/569049652 conn(0x5613fad54550 msgr2=0x5613fae284c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:08.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.164+0000 7fdccad36b80 1 -- 172.21.15.112:0/569049652 shutdown_connections 2024-04-25T18:48:08.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.164+0000 7fdccad36b80 1 -- 172.21.15.112:0/569049652 wait complete. 2024-04-25T18:48:08.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.165+0000 7fdccad36b80 1 -- start start 2024-04-25T18:48:08.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.165+0000 7fdccad36b80 1 Processor -- start 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.165+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 0x5613fae4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.165+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae25c10 0x5613fae2a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.165+0000 7fdccad36b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae30ab0 0x5613fae2a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.165+0000 7fdccad36b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5613fae417a0 con 0x5613fae252f0 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.165+0000 7fdccad36b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5613fae410d0 con 0x5613fae30ab0 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.165+0000 7fdccad36b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5613fae41e70 con 0x5613fae25c10 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.165+0000 7fdccad36b80 10 client.0 _ll_register_callbacks cb 0x5613fad116d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdcbda00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 0x5613fae4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdcbe400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae30ab0 0x5613fae2a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdcbda00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 0x5613fae4b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34614/0 (socket says 172.21.15.112:34614) 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdcbda00700 1 -- 172.21.15.112:0/3959924704 learned_addr learned my addr 172.21.15.112:0/3959924704 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:08.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdcbd000700 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae25c10 0x5613fae2a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdcbe400700 1 -- 172.21.15.112:0/3959924704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae25c10 msgr2=0x5613fae2a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdcbe400700 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae25c10 0x5613fae2a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdcbe400700 1 -- 172.21.15.112:0/3959924704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 msgr2=0x5613fae4b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdcbe400700 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 0x5613fae4b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdcbe400700 1 -- 172.21.15.112:0/3959924704 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613fae46d60 con 0x5613fae30ab0 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdcbe400700 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae30ab0 0x5613fae2a780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdcb009d000 tx=0x7fdcb009d150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2f81c711248c005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.166+0000 7fdc9f400700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.167+0000 7fdc9f400700 1 -- 172.21.15.112:0/3959924704 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb000cea0 con 0x5613fae30ab0 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.167+0000 7fdc9f400700 1 -- 172.21.15.112:0/3959924704 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fdcb001d450 con 0x5613fae30ab0 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.167+0000 7fdccad36b80 1 -- 172.21.15.112:0/3959924704 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5613fad806d0 con 0x5613fae30ab0 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.167+0000 7fdc9f400700 1 -- 172.21.15.112:0/3959924704 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdcb002c2b0 con 0x5613fae30ab0 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.167+0000 7fdccad36b80 10 client.25274 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:08.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.167+0000 7fdccad36b80 1 -- 172.21.15.112:0/3959924704 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5613faea7730 con 0x5613fae30ab0 2024-04-25T18:48:08.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.167+0000 7fdc9f400700 1 -- 172.21.15.112:0/3959924704 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5613fad806d0 con 0x5613fae30ab0 2024-04-25T18:48:08.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.168+0000 7fdc9f400700 1 -- 172.21.15.112:0/3959924704 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdcb0011070 con 0x5613fae30ab0 2024-04-25T18:48:08.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.168+0000 7fdccad36b80 10 client.25274 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:08.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.168+0000 7fdccad36b80 10 client.25274 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T18:48:08.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.168+0000 7fdccad36b80 20 client.25274 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:08.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.168+0000 7fdccad36b80 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5613faeac340 0x5613faeab050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:08.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.168+0000 7fdccad36b80 4 client.25274 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.168+0000 7fdccad36b80 1 -- 172.21.15.112:0/3959924704 --> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] -- command(tid 0: {"prefix": "status"}) -- 0x5613fabf9200 con 0x5613faeac340 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.168+0000 7fdcbda00700 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5613faeac340 0x5613faeab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdcbda00700 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5613faeac340 0x5613faeab050 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fd9c55c24f1c490 server_cookie=f1425f92341b1fcf in_seq=0 out_seq=0 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdc9f400700 10 client.25274 ms_handle_connect on v2:172.21.15.112:6840/1028859850 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdc9f400700 1 -- 172.21.15.112:0/3959924704 <== mds.1 v2:172.21.15.112:6840/1028859850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5613fabf9200 con 0x5613faeac340 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdc9f400700 10 client.25274 handle_command_reply: tid=0 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdccad36b80 1 client.25274 shutdown 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdccad36b80 1 -- 172.21.15.112:0/3959924704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae30ab0 msgr2=0x5613fae2a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdccad36b80 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae30ab0 0x5613fae2a780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdcb009d000 tx=0x7fdcb009d150 comp rx=0 tx=0).stop 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdccad36b80 1 -- 172.21.15.112:0/3959924704 shutdown_connections 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdccad36b80 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x5613faeac340 0x5613faeab050 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdccad36b80 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5613fae30ab0 0x5613fae2a780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdccad36b80 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5613fae252f0 0x5613fae4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdccad36b80 1 --2- 172.21.15.112:0/3959924704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613fae25c10 0x5613fae2a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdc9f400700 0 client.25274 ms_handle_reset on v2:172.21.15.112:6840/1028859850 2024-04-25T18:48:08.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.169+0000 7fdccad36b80 1 -- 172.21.15.112:0/3959924704 >> 172.21.15.112:0/3959924704 conn(0x5613fad54550 msgr2=0x5613fae27210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:08.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.170+0000 7fdccad36b80 1 -- 172.21.15.112:0/3959924704 shutdown_connections 2024-04-25T18:48:08.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.170+0000 7fdccad36b80 1 -- 172.21.15.112:0/3959924704 wait complete. 2024-04-25T18:48:08.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.170+0000 7fdccad36b80 20 client.25274 trim_cache size 0 max 16384 2024-04-25T18:48:08.174 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:48:08.174 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:48:08.174 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-04-25T18:48:08.174 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24511, 2024-04-25T18:48:08.174 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T18:48:08.174 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T18:48:08.175 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T18:48:08.175 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 270.960882035, 2024-04-25T18:48:08.175 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 26, 2024-04-25T18:48:08.175 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 95, 2024-04-25T18:48:08.175 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 95, 2024-04-25T18:48:08.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 314.54512432500002 2024-04-25T18:48:08.175 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 -- 172.21.15.112:0/4027305044 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdca00a3f60 msgr2=0x7fdca00a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdca00a3f60 0x7fdca00a6420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdcac004510 tx=0x7fdcac018040 comp rx=0 tx=0).stop 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 -- 172.21.15.112:0/4027305044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcb8108a30 msgr2=0x7fdcb819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcb8108a30 0x7fdcb819d940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009790 tx=0x7fdcb000ccb0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 -- 172.21.15.112:0/4027305044 shutdown_connections 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdca00a3f60 0x7fdca00a6420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcb8108a30 0x7fdcb819d940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcb806a750 0x7fdcb81990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 --2- 172.21.15.112:0/4027305044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcb806b0f0 0x7fdcb8199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 -- 172.21.15.112:0/4027305044 >> 172.21.15.112:0/4027305044 conn(0x7fdcb80788f0 msgr2=0x7fdcb80ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 -- 172.21.15.112:0/4027305044 shutdown_connections 2024-04-25T18:48:08.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:08.171+0000 7fdcc0400700 1 -- 172.21.15.112:0/4027305044 wait complete. 2024-04-25T18:48:08.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:08 smithi112 ceph-mon[27864]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T18:48:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:08 smithi171 ceph-mon[33266]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T18:48:08.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:08 smithi161 ceph-mon[32609]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T18:48:09.256 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T18:48:09.720 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:48:10.419 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:10 smithi112 ceph-mon[27864]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T18:48:10.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:10 smithi171 ceph-mon[33266]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T18:48:10.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:10 smithi161 ceph-mon[32609]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T18:48:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.173+0000 7fcaaec00700 1 -- 172.21.15.112:0/1682197713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa8108970 msgr2=0x7fcaa8108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.173+0000 7fcaaec00700 1 --2- 172.21.15.112:0/1682197713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa8108970 0x7fcaa8108d50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca98009990 tx=0x7fca98009ca0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.173+0000 7fcaaec00700 1 -- 172.21.15.112:0/1682197713 shutdown_connections 2024-04-25T18:48:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.173+0000 7fcaaec00700 1 --2- 172.21.15.112:0/1682197713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa8103330 0x7fcaa810f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.173+0000 7fcaaec00700 1 --2- 172.21.15.112:0/1682197713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa8108970 0x7fcaa8108d50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.173+0000 7fcaaec00700 1 --2- 172.21.15.112:0/1682197713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8102970 0x7fcaa8102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.173+0000 7fcaaec00700 1 -- 172.21.15.112:0/1682197713 >> 172.21.15.112:0/1682197713 conn(0x7fcaa80fe4c0 msgr2=0x7fcaa81008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.173+0000 7fcaaec00700 1 -- 172.21.15.112:0/1682197713 shutdown_connections 2024-04-25T18:48:12.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.173+0000 7fcaaec00700 1 -- 172.21.15.112:0/1682197713 wait complete. 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.174+0000 7fcaaec00700 1 Processor -- start 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.174+0000 7fcaaec00700 1 -- start start 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.174+0000 7fcaaec00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8102970 0x7fcaa8199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.174+0000 7fcaaec00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa8103330 0x7fcaa81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.174+0000 7fcaaec00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa8108970 0x7fcaa819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.174+0000 7fcaaec00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8112bb0 con 0x7fcaa8108970 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.175+0000 7fcaaec00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8112a30 con 0x7fcaa8103330 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.175+0000 7fcaaec00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8112d30 con 0x7fcaa8102970 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.175+0000 7fcaae200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa8108970 0x7fcaa819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.175+0000 7fcaad800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8102970 0x7fcaa8199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.175+0000 7fcaad800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8102970 0x7fcaa8199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:47950/0 (socket says 172.21.15.112:47950) 2024-04-25T18:48:12.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.175+0000 7fcaad800700 1 -- 172.21.15.112:0/1021469850 learned_addr learned my addr 172.21.15.112:0/1021469850 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:12.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.175+0000 7fcaad800700 1 -- 172.21.15.112:0/1021469850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa8103330 msgr2=0x7fcaa81995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:12.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.175+0000 7fcaad800700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa8103330 0x7fcaa81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.175+0000 7fcaad800700 1 -- 172.21.15.112:0/1021469850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa8108970 msgr2=0x7fcaa819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.175+0000 7fcaad800700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa8108970 0x7fcaa819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.175+0000 7fcaad800700 1 -- 172.21.15.112:0/1021469850 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa819e090 con 0x7fcaa8102970 2024-04-25T18:48:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.176+0000 7fcaad800700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8102970 0x7fcaa8199080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca980047b0 tx=0x7fca98009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.176+0000 7fcaa6000700 1 -- 172.21.15.112:0/1021469850 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98027070 con 0x7fcaa8102970 2024-04-25T18:48:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.176+0000 7fcaaec00700 1 -- 172.21.15.112:0/1021469850 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa819e320 con 0x7fcaa8102970 2024-04-25T18:48:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.176+0000 7fcaa6000700 1 -- 172.21.15.112:0/1021469850 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca98022470 con 0x7fcaa8102970 2024-04-25T18:48:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.176+0000 7fcaa6000700 1 -- 172.21.15.112:0/1021469850 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98017650 con 0x7fcaa8102970 2024-04-25T18:48:12.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.176+0000 7fcaaec00700 1 -- 172.21.15.112:0/1021469850 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa81a5c00 con 0x7fcaa8102970 2024-04-25T18:48:12.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.180+0000 7fcaa6000700 1 -- 172.21.15.112:0/1021469850 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fca9802a070 con 0x7fcaa8102970 2024-04-25T18:48:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.183+0000 7fcaa6000700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fca8c0a3f60 0x7fca8c0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.183+0000 7fcaa6000700 1 -- 172.21.15.112:0/1021469850 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fca980d1910 con 0x7fcaa8102970 2024-04-25T18:48:12.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.183+0000 7fcaace00700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fca8c0a3f60 0x7fca8c0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.183+0000 7fcababc9b80 1 Processor -- start 2024-04-25T18:48:12.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.183+0000 7fcababc9b80 1 -- start start 2024-04-25T18:48:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234c3e0 0x560ee234c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234cd00 0x560ee235bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 0x560ee235e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcababc9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560ee229fe40 con 0x560ee234c3e0 2024-04-25T18:48:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcababc9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ee22466e0 con 0x560ee234cd00 2024-04-25T18:48:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcababc9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560ee229be00 con 0x560ee235c050 2024-04-25T18:48:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcaace00700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fca8c0a3f60 0x7fca8c0a6420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca94005950 tx=0x7fca940058e0 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcaad800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234c3e0 0x560ee234c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcaace00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234cd00 0x560ee235bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcaad800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234c3e0 0x560ee234c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37852/0 (socket says 172.21.15.112:37852) 2024-04-25T18:48:12.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcaad800700 1 -- 172.21.15.112:0/3444892177 learned_addr learned my addr 172.21.15.112:0/3444892177 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.184+0000 7fcaae200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 0x560ee235e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.185+0000 7fcaad800700 1 -- 172.21.15.112:0/3444892177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 msgr2=0x560ee235e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.185+0000 7fcaad800700 1 --2- 172.21.15.112:0/3444892177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 0x560ee235e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.185+0000 7fcaad800700 1 -- 172.21.15.112:0/3444892177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234cd00 msgr2=0x560ee235bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.185+0000 7fcaad800700 1 --2- 172.21.15.112:0/3444892177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234cd00 0x560ee235bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.185+0000 7fcaad800700 1 -- 172.21.15.112:0/3444892177 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee223c630 con 0x560ee234c3e0 2024-04-25T18:48:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.185+0000 7fcaace00700 1 --2- 172.21.15.112:0/3444892177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234cd00 0x560ee235bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.185+0000 7fcaae200700 1 --2- 172.21.15.112:0/3444892177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 0x560ee235e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:12.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.185+0000 7fcaad800700 1 --2- 172.21.15.112:0/3444892177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234c3e0 0x560ee234c7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fca9809a1f0 tx=0x7fca98099bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691c3a4bd3ff3062 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.185+0000 7fca9ea00700 1 -- 172.21.15.112:0/3444892177 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9809df20 con 0x560ee234c3e0 2024-04-25T18:48:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.186+0000 7fca9ea00700 1 -- 172.21.15.112:0/3444892177 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca9809d820 con 0x560ee234c3e0 2024-04-25T18:48:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.186+0000 7fca9ea00700 1 -- 172.21.15.112:0/3444892177 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9809da80 con 0x560ee234c3e0 2024-04-25T18:48:12.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.186+0000 7fcababc9b80 1 -- 172.21.15.112:0/3444892177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234c3e0 msgr2=0x560ee234c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.186+0000 7fcababc9b80 1 --2- 172.21.15.112:0/3444892177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234c3e0 0x560ee234c7c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fca9809a1f0 tx=0x7fca98099bf0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.187+0000 7fcababc9b80 1 -- 172.21.15.112:0/3444892177 shutdown_connections 2024-04-25T18:48:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.187+0000 7fcababc9b80 1 --2- 172.21.15.112:0/3444892177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 0x560ee235e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.187+0000 7fcababc9b80 1 --2- 172.21.15.112:0/3444892177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234c3e0 0x560ee234c7c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.187+0000 7fcababc9b80 1 --2- 172.21.15.112:0/3444892177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234cd00 0x560ee235bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.187+0000 7fcababc9b80 1 -- 172.21.15.112:0/3444892177 >> 172.21.15.112:0/3444892177 conn(0x560ee227b550 msgr2=0x560ee2348a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.187+0000 7fcababc9b80 1 -- 172.21.15.112:0/3444892177 shutdown_connections 2024-04-25T18:48:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.187+0000 7fcababc9b80 1 -- 172.21.15.112:0/3444892177 wait complete. 2024-04-25T18:48:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.187+0000 7fcababc9b80 1 -- start start 2024-04-25T18:48:12.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.188+0000 7fcababc9b80 1 Processor -- start 2024-04-25T18:48:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.188+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 0x560ee2356fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.188+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee234cd00 0x560ee2357500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.188+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 0x560ee2357a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.188+0000 7fcababc9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560ee234f6a0 con 0x560ee235c050 2024-04-25T18:48:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.188+0000 7fcababc9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ee2259810 con 0x560ee234c3e0 2024-04-25T18:48:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.188+0000 7fcababc9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560ee234f820 con 0x560ee234cd00 2024-04-25T18:48:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.188+0000 7fcababc9b80 10 client.0 _ll_register_callbacks cb 0x560ee22386d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.189+0000 7fcaad800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 0x560ee2356fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.189+0000 7fcaad800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 0x560ee2356fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:59462/0 (socket says 172.21.15.112:59462) 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.189+0000 7fcaad800700 1 -- 172.21.15.112:0/134042649 learned_addr learned my addr 172.21.15.112:0/134042649 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.189+0000 7fcaace00700 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee234cd00 0x560ee2357500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.189+0000 7fcaae200700 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 0x560ee2357a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.189+0000 7fcaad800700 1 -- 172.21.15.112:0/134042649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee234cd00 msgr2=0x560ee2357500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.189+0000 7fcaad800700 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee234cd00 0x560ee2357500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.189+0000 7fcaad800700 1 -- 172.21.15.112:0/134042649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 msgr2=0x560ee2357a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.189+0000 7fcaad800700 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 0x560ee2357a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.189+0000 7fcaad800700 1 -- 172.21.15.112:0/134042649 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee21ed880 con 0x560ee234c3e0 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.190+0000 7fcaad800700 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 0x560ee2356fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca98006010 tx=0x7fca980058c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e48bc6a517119743 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.190+0000 7fca93400700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.190+0000 7fca93400700 1 -- 172.21.15.112:0/134042649 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980bb060 con 0x560ee234c3e0 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.190+0000 7fca93400700 1 -- 172.21.15.112:0/134042649 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fca9809dc80 con 0x560ee234c3e0 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.190+0000 7fcababc9b80 1 -- 172.21.15.112:0/134042649 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9809d820 con 0x560ee234c3e0 2024-04-25T18:48:12.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.190+0000 7fca93400700 1 -- 172.21.15.112:0/134042649 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca980a6400 con 0x560ee234c3e0 2024-04-25T18:48:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.190+0000 7fcababc9b80 10 client.25283 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.190+0000 7fcababc9b80 1 -- 172.21.15.112:0/134042649 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ee2236aa0 con 0x560ee234c3e0 2024-04-25T18:48:12.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.190+0000 7fca93400700 1 -- 172.21.15.112:0/134042649 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca9809d820 con 0x560ee234c3e0 2024-04-25T18:48:12.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.191+0000 7fca93400700 1 -- 172.21.15.112:0/134042649 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fca980681a0 con 0x560ee234c3e0 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.191+0000 7fcababc9b80 10 client.25283 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.191+0000 7fcababc9b80 10 client.25283 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.191+0000 7fcababc9b80 20 client.25283 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.191+0000 7fcababc9b80 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560ee2366f40 0x560ee23683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.191+0000 7fcababc9b80 4 client.25283 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.191+0000 7fcababc9b80 1 -- 172.21.15.112:0/134042649 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ee22ad630 con 0x560ee2366f40 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.191+0000 7fcaace00700 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560ee2366f40 0x560ee23683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.192+0000 7fcaace00700 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560ee2366f40 0x560ee23683b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b921c981cde490ab server_cookie=2e5a2c490abadaf2 in_seq=0 out_seq=0 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.192+0000 7fca93400700 10 client.25283 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.193+0000 7fca93400700 1 -- 172.21.15.112:0/134042649 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ee22ad630 con 0x560ee2366f40 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.193+0000 7fca93400700 10 client.25283 handle_command_reply: tid=0 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.193+0000 7fcababc9b80 1 client.25283 shutdown 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.193+0000 7fcababc9b80 1 -- 172.21.15.112:0/134042649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 msgr2=0x560ee2356fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:12.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.193+0000 7fcababc9b80 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 0x560ee2356fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca98006010 tx=0x7fca980058c0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.193+0000 7fcababc9b80 1 -- 172.21.15.112:0/134042649 shutdown_connections 2024-04-25T18:48:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.193+0000 7fcababc9b80 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560ee2366f40 0x560ee23683b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.193+0000 7fcababc9b80 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 0x560ee2357a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.193+0000 7fcababc9b80 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 0x560ee2356fc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.193+0000 7fcababc9b80 1 --2- 172.21.15.112:0/134042649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee234cd00 0x560ee2357500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.193+0000 7fcababc9b80 1 -- 172.21.15.112:0/134042649 >> 172.21.15.112:0/134042649 conn(0x560ee227b550 msgr2=0x560ee2348680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.194+0000 7fca93400700 0 client.25283 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.194+0000 7fcababc9b80 1 -- 172.21.15.112:0/134042649 shutdown_connections 2024-04-25T18:48:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.194+0000 7fcababc9b80 1 -- 172.21.15.112:0/134042649 wait complete. 2024-04-25T18:48:12.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.194+0000 7fcababc9b80 20 client.25283 trim_cache size 0 max 16384 2024-04-25T18:48:12.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.227+0000 7fcababc9b80 1 Processor -- start 2024-04-25T18:48:12.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.227+0000 7fcababc9b80 1 -- start start 2024-04-25T18:48:12.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.227+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 0x560ee234f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.227+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234cd00 0x560ee2371810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.227+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 0x560ee2371d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.227+0000 7fcababc9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560ee23666f0 con 0x560ee234cd00 2024-04-25T18:48:12.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.227+0000 7fcababc9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ee2366dc0 con 0x560ee234c3e0 2024-04-25T18:48:12.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.227+0000 7fcababc9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560ee2367490 con 0x560ee235c050 2024-04-25T18:48:12.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.228+0000 7fcaae200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 0x560ee2371d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.228+0000 7fcaad800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 0x560ee234f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.228+0000 7fcaae200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 0x560ee2371d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:47988/0 (socket says 172.21.15.112:47988) 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.228+0000 7fcaae200700 1 -- 172.21.15.112:0/3021608793 learned_addr learned my addr 172.21.15.112:0/3021608793 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.228+0000 7fcaace00700 1 --2- 172.21.15.112:0/3021608793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234cd00 0x560ee2371810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.228+0000 7fcaad800700 1 -- 172.21.15.112:0/3021608793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 msgr2=0x560ee2371d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.228+0000 7fcaad800700 1 --2- 172.21.15.112:0/3021608793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 0x560ee2371d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.228+0000 7fcaad800700 1 -- 172.21.15.112:0/3021608793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234cd00 msgr2=0x560ee2371810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.228+0000 7fcaad800700 1 --2- 172.21.15.112:0/3021608793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234cd00 0x560ee2371810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.228+0000 7fcaad800700 1 -- 172.21.15.112:0/3021608793 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee2349560 con 0x560ee234c3e0 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.228+0000 7fcaad800700 1 --2- 172.21.15.112:0/3021608793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 0x560ee234f3a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca9809bad0 tx=0x7fca9809bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=146a36d940c57df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.229+0000 7fca9e000700 1 -- 172.21.15.112:0/3021608793 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980bb070 con 0x560ee234c3e0 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.229+0000 7fca9e000700 1 -- 172.21.15.112:0/3021608793 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca9809d820 con 0x560ee234c3e0 2024-04-25T18:48:12.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.229+0000 7fca9e000700 1 -- 172.21.15.112:0/3021608793 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9809da80 con 0x560ee234c3e0 2024-04-25T18:48:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.229+0000 7fcababc9b80 1 -- 172.21.15.112:0/3021608793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 msgr2=0x560ee234f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.229+0000 7fcababc9b80 1 --2- 172.21.15.112:0/3021608793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 0x560ee234f3a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca9809bad0 tx=0x7fca9809bd30 comp rx=0 tx=0).stop 2024-04-25T18:48:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.229+0000 7fcababc9b80 1 -- 172.21.15.112:0/3021608793 shutdown_connections 2024-04-25T18:48:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.229+0000 7fcababc9b80 1 --2- 172.21.15.112:0/3021608793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee235c050 0x560ee2371d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.229+0000 7fcababc9b80 1 --2- 172.21.15.112:0/3021608793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234c3e0 0x560ee234f3a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.229+0000 7fcababc9b80 1 --2- 172.21.15.112:0/3021608793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee234cd00 0x560ee2371810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.229+0000 7fcababc9b80 1 -- 172.21.15.112:0/3021608793 >> 172.21.15.112:0/3021608793 conn(0x560ee227b550 msgr2=0x560ee2347b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.230+0000 7fcababc9b80 1 -- 172.21.15.112:0/3021608793 shutdown_connections 2024-04-25T18:48:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.230+0000 7fcababc9b80 1 -- 172.21.15.112:0/3021608793 wait complete. 2024-04-25T18:48:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.230+0000 7fcababc9b80 1 -- start start 2024-04-25T18:48:12.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.230+0000 7fcababc9b80 1 Processor -- start 2024-04-25T18:48:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.231+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee234c3e0 0x560ee2353270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.231+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234cd00 0x560ee23537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.231+0000 7fcababc9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 0x560ee2353cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.231+0000 7fcababc9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560ee2368230 con 0x560ee235c050 2024-04-25T18:48:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.231+0000 7fcababc9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ee2367b60 con 0x560ee234cd00 2024-04-25T18:48:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.231+0000 7fcababc9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560ee2368900 con 0x560ee234c3e0 2024-04-25T18:48:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.231+0000 7fcababc9b80 10 client.0 _ll_register_callbacks cb 0x560ee22386d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.231+0000 7fcaae200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 0x560ee2353cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.231+0000 7fcaad800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee234c3e0 0x560ee2353270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.231+0000 7fcaae200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 0x560ee2353cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37888/0 (socket says 172.21.15.112:37888) 2024-04-25T18:48:12.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.231+0000 7fcaae200700 1 -- 172.21.15.112:0/628940433 learned_addr learned my addr 172.21.15.112:0/628940433 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fcaad800700 1 -- 172.21.15.112:0/628940433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234cd00 msgr2=0x560ee23537b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T18:48:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fcaad800700 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234cd00 0x560ee23537b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fcaad800700 1 -- 172.21.15.112:0/628940433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 msgr2=0x560ee2353cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fcaad800700 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 0x560ee2353cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fcaad800700 1 -- 172.21.15.112:0/628940433 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee2362480 con 0x560ee234c3e0 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fcaae200700 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 0x560ee2353cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fcaad800700 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee234c3e0 0x560ee2353270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fca9809a1f0 tx=0x7fca98099bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=582dfac2049d2320 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fca93400700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fca93400700 1 -- 172.21.15.112:0/628940433 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980bb070 con 0x560ee234c3e0 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fcababc9b80 1 -- 172.21.15.112:0/628940433 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9809d820 con 0x560ee234c3e0 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fca93400700 1 -- 172.21.15.112:0/628940433 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fca9809da80 con 0x560ee234c3e0 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fca93400700 1 -- 172.21.15.112:0/628940433 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca9809df20 con 0x560ee234c3e0 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fcababc9b80 10 client.25294 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.232+0000 7fcababc9b80 1 -- 172.21.15.112:0/628940433 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ee2362480 con 0x560ee234c3e0 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.233+0000 7fca93400700 1 -- 172.21.15.112:0/628940433 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca9809d820 con 0x560ee234c3e0 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.233+0000 7fca93400700 1 -- 172.21.15.112:0/628940433 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fca9800fad0 con 0x560ee234c3e0 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.233+0000 7fcababc9b80 10 client.25294 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.233+0000 7fcababc9b80 10 client.25294 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T18:48:12.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.233+0000 7fcababc9b80 20 client.25294 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.233+0000 7fcababc9b80 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560ee23d26b0 0x560ee23d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.233+0000 7fcababc9b80 4 client.25294 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:48:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.233+0000 7fcababc9b80 1 -- 172.21.15.112:0/628940433 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "status"}) -- 0x560ee22a76d0 con 0x560ee23d26b0 2024-04-25T18:48:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.234+0000 7fcaace00700 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560ee23d26b0 0x560ee23d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.234+0000 7fcaace00700 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560ee23d26b0 0x560ee23d13c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6627a13e4ef906f3 server_cookie=fc46a7bb94458541 in_seq=0 out_seq=0 2024-04-25T18:48:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.234+0000 7fca93400700 10 client.25294 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.234+0000 7fca93400700 1 -- 172.21.15.112:0/628940433 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560ee22a76d0 con 0x560ee23d26b0 2024-04-25T18:48:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.234+0000 7fca93400700 10 client.25294 handle_command_reply: tid=0 2024-04-25T18:48:12.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fcababc9b80 1 client.25294 shutdown 2024-04-25T18:48:12.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fcababc9b80 1 -- 172.21.15.112:0/628940433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee234c3e0 msgr2=0x560ee2353270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:12.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fcababc9b80 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee234c3e0 0x560ee2353270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fca9809a1f0 tx=0x7fca98099bf0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fcababc9b80 1 -- 172.21.15.112:0/628940433 shutdown_connections 2024-04-25T18:48:12.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fcababc9b80 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560ee23d26b0 0x560ee23d13c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fcababc9b80 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ee235c050 0x560ee2353cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fcababc9b80 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560ee234c3e0 0x560ee2353270 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fcababc9b80 1 --2- 172.21.15.112:0/628940433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ee234cd00 0x560ee23537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fcababc9b80 1 -- 172.21.15.112:0/628940433 >> 172.21.15.112:0/628940433 conn(0x560ee227b550 msgr2=0x560ee23558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fca93400700 0 client.25294 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fcababc9b80 1 -- 172.21.15.112:0/628940433 shutdown_connections 2024-04-25T18:48:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.235+0000 7fcababc9b80 1 -- 172.21.15.112:0/628940433 wait complete. 2024-04-25T18:48:12.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.236+0000 7fcababc9b80 20 client.25294 trim_cache size 0 max 16384 2024-04-25T18:48:12.240 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:48:12.240 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "2fe18a52-0332-11ef-bc93-c7b262605968", 2024-04-25T18:48:12.240 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-25T18:48:12.240 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24512, 2024-04-25T18:48:12.241 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T18:48:12.241 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T18:48:12.241 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T18:48:12.241 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 291.95107323100001, 2024-04-25T18:48:12.241 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 26, 2024-04-25T18:48:12.241 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 95, 2024-04-25T18:48:12.241 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 95, 2024-04-25T18:48:12.241 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 315.83459941199999 2024-04-25T18:48:12.241 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.236+0000 7fcaaec00700 1 -- 172.21.15.112:0/1021469850 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fca8c0a3f60 msgr2=0x7fca8c0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.236+0000 7fcaaec00700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fca8c0a3f60 0x7fca8c0a6420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca94005950 tx=0x7fca940058e0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.236+0000 7fcaaec00700 1 -- 172.21.15.112:0/1021469850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8102970 msgr2=0x7fcaa8199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.236+0000 7fcaaec00700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8102970 0x7fcaa8199080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca980047b0 tx=0x7fca98009550 comp rx=0 tx=0).stop 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.236+0000 7fcaaec00700 1 -- 172.21.15.112:0/1021469850 shutdown_connections 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.236+0000 7fcaaec00700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fca8c0a3f60 0x7fca8c0a6420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.236+0000 7fcaaec00700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa8108970 0x7fcaa819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.236+0000 7fcaaec00700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8102970 0x7fcaa8199080 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.236+0000 7fcaaec00700 1 --2- 172.21.15.112:0/1021469850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa8103330 0x7fcaa81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.236+0000 7fcaaec00700 1 -- 172.21.15.112:0/1021469850 >> 172.21.15.112:0/1021469850 conn(0x7fcaa80fe4c0 msgr2=0x7fcaa80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.237+0000 7fcaaec00700 1 -- 172.21.15.112:0/1021469850 shutdown_connections 2024-04-25T18:48:12.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:12.237+0000 7fcaaec00700 1 -- 172.21.15.112:0/1021469850 wait complete. 2024-04-25T18:48:12.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:12 smithi171 ceph-mon[33266]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-25T18:48:12.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:12 smithi161 ceph-mon[32609]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-25T18:48:12.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:12 smithi112 ceph-mon[27864]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-25T18:48:13.378 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-25T18:48:14.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:14 smithi171 ceph-mon[33266]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T18:48:14.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:14 smithi161 ceph-mon[32609]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T18:48:14.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:14 smithi112 ceph-mon[27864]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T18:48:16.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:16 smithi171 ceph-mon[33266]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T18:48:16.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:16 smithi161 ceph-mon[32609]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T18:48:16.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:16 smithi112 ceph-mon[27864]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T18:48:18.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:18 smithi171 ceph-mon[33266]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T18:48:18.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:18 smithi161 ceph-mon[32609]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T18:48:18.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:18 smithi112 ceph-mon[27864]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T18:48:20.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:20 smithi171 ceph-mon[33266]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T18:48:20.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:20 smithi161 ceph-mon[32609]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T18:48:20.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:20 smithi112 ceph-mon[27864]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T18:48:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.204+0000 7f4677e00640 1 -- 172.21.15.112:0/309285011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4678074320 msgr2=0x7f4678074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.204+0000 7f4677e00640 1 --2- 172.21.15.112:0/309285011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4678074320 0x7f4678074700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4668009900 tx=0x7f466802a480 comp rx=0 tx=0).stop 2024-04-25T18:48:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.205+0000 7f4677e00640 1 -- 172.21.15.112:0/309285011 shutdown_connections 2024-04-25T18:48:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.205+0000 7f4677e00640 1 --2- 172.21.15.112:0/309285011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f467810e610 0x7f4678110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.205+0000 7f4677e00640 1 --2- 172.21.15.112:0/309285011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4678074c40 0x7f467810e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.205+0000 7f4677e00640 1 --2- 172.21.15.112:0/309285011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4678074320 0x7f4678074700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.205+0000 7f4677e00640 1 -- 172.21.15.112:0/309285011 >> 172.21.15.112:0/309285011 conn(0x7f46780fd8a0 msgr2=0x7f46780ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.205+0000 7f4677e00640 1 -- 172.21.15.112:0/309285011 shutdown_connections 2024-04-25T18:48:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.205+0000 7f4677e00640 1 -- 172.21.15.112:0/309285011 wait complete. 2024-04-25T18:48:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.206+0000 7f4677e00640 1 Processor -- start 2024-04-25T18:48:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.207+0000 7f4677e00640 1 -- start start 2024-04-25T18:48:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.207+0000 7f4677e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4678074320 0x7f46781a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.207+0000 7f4677e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4678074c40 0x7f46781a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.207+0000 7f4677e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f467810e610 0x7f467819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.207+0000 7f4677e00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46781142a0 con 0x7f4678074320 2024-04-25T18:48:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.207+0000 7f4677e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4678114120 con 0x7f4678074c40 2024-04-25T18:48:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.207+0000 7f4677e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4678114420 con 0x7f467810e610 2024-04-25T18:48:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.207+0000 7f4676a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4678074320 0x7f46781a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.208+0000 7f4676a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4678074320 0x7f46781a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39526/0 (socket says 172.21.15.112:39526) 2024-04-25T18:48:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.208+0000 7f4676a00640 1 -- 172.21.15.112:0/3883240761 learned_addr learned my addr 172.21.15.112:0/3883240761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.208+0000 7f4677400640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f467810e610 0x7f467819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.208+0000 7f4676000640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4678074c40 0x7f46781a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.208+0000 7f4676000640 1 -- 172.21.15.112:0/3883240761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f467810e610 msgr2=0x7f467819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.208+0000 7f4676000640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f467810e610 0x7f467819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.208+0000 7f4676000640 1 -- 172.21.15.112:0/3883240761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4678074320 msgr2=0x7f46781a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.208+0000 7f4676000640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4678074320 0x7f46781a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.208+0000 7f4676000640 1 -- 172.21.15.112:0/3883240761 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467819aef0 con 0x7f4678074c40 2024-04-25T18:48:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.208+0000 7f4676a00640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4678074320 0x7f46781a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.208+0000 7f4677400640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f467810e610 0x7f467819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:48:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.209+0000 7f4676000640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4678074c40 0x7f46781a0ac0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f466400ea20 tx=0x7f466400eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.209+0000 7f466f400640 1 -- 172.21.15.112:0/3883240761 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466400ccc0 con 0x7f4678074c40 2024-04-25T18:48:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.209+0000 7f466f400640 1 -- 172.21.15.112:0/3883240761 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f466400ce60 con 0x7f4678074c40 2024-04-25T18:48:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.209+0000 7f466f400640 1 -- 172.21.15.112:0/3883240761 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664004a00 con 0x7f4678074c40 2024-04-25T18:48:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.210+0000 7f4677e00640 1 -- 172.21.15.112:0/3883240761 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467819b1e0 con 0x7f4678074c40 2024-04-25T18:48:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.210+0000 7f4677e00640 1 -- 172.21.15.112:0/3883240761 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467806b970 con 0x7f4678074c40 2024-04-25T18:48:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.214+0000 7f4680a80740 1 Processor -- start 2024-04-25T18:48:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.214+0000 7f4680a80740 1 -- start start 2024-04-25T18:48:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.215+0000 7f4680a80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93469b0 0x55d1d9346d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.215+0000 7f4680a80740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 0x55d1d9351650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.215+0000 7f4680a80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d9351b90 0x55d1d9353f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.215+0000 7f4680a80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d1d92ac900 con 0x55d1d9351b90 2024-04-25T18:48:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.215+0000 7f4680a80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d1d9252ef0 con 0x55d1d93469b0 2024-04-25T18:48:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.215+0000 7f4680a80740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d1d92a88c0 con 0x55d1d93472d0 2024-04-25T18:48:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.215+0000 7f4676a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 0x55d1d9351650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.215+0000 7f4677400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93469b0 0x55d1d9346d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.215+0000 7f4676a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 0x55d1d9351650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:49954/0 (socket says 172.21.15.112:49954) 2024-04-25T18:48:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.215+0000 7f4676a00640 1 -- 172.21.15.112:0/3080550251 learned_addr learned my addr 172.21.15.112:0/3080550251 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.216+0000 7f4676a00640 1 -- 172.21.15.112:0/3080550251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93469b0 msgr2=0x55d1d9346d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.216+0000 7f4676a00640 1 --2- 172.21.15.112:0/3080550251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93469b0 0x55d1d9346d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.216+0000 7f4676a00640 1 -- 172.21.15.112:0/3080550251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d9351b90 msgr2=0x55d1d9353f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T18:48:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.216+0000 7f4676a00640 1 --2- 172.21.15.112:0/3080550251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d9351b90 0x55d1d9353f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.216+0000 7f4676a00640 1 -- 172.21.15.112:0/3080550251 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d8fbf210 con 0x55d1d93472d0 2024-04-25T18:48:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.216+0000 7f466f400640 1 -- 172.21.15.112:0/3883240761 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f4664004ba0 con 0x7f4678074c40 2024-04-25T18:48:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.216+0000 7f4676a00640 1 --2- 172.21.15.112:0/3080550251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 0x55d1d9351650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4668006010 tx=0x7f46680044b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ff618940bef938 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.217+0000 7f4663e00640 1 -- 172.21.15.112:0/3080550251 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668040070 con 0x55d1d93472d0 2024-04-25T18:48:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.217+0000 7f4663e00640 1 -- 172.21.15.112:0/3080550251 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46680047a0 con 0x55d1d93472d0 2024-04-25T18:48:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.217+0000 7f4663e00640 1 -- 172.21.15.112:0/3080550251 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668035040 con 0x55d1d93472d0 2024-04-25T18:48:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.218+0000 7f4680a80740 1 -- 172.21.15.112:0/3080550251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 msgr2=0x55d1d9351650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.218+0000 7f4680a80740 1 --2- 172.21.15.112:0/3080550251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 0x55d1d9351650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4668006010 tx=0x7f46680044b0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.218+0000 7f4680a80740 1 -- 172.21.15.112:0/3080550251 shutdown_connections 2024-04-25T18:48:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.218+0000 7f4680a80740 1 --2- 172.21.15.112:0/3080550251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d9351b90 0x55d1d9353f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.218+0000 7f4680a80740 1 --2- 172.21.15.112:0/3080550251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 0x55d1d9351650 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.218+0000 7f4680a80740 1 --2- 172.21.15.112:0/3080550251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93469b0 0x55d1d9346d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.218+0000 7f4680a80740 1 -- 172.21.15.112:0/3080550251 >> 172.21.15.112:0/3080550251 conn(0x55d1d9342ec0 msgr2=0x55d1d93432a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.218+0000 7f4680a80740 1 -- 172.21.15.112:0/3080550251 shutdown_connections 2024-04-25T18:48:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.218+0000 7f4680a80740 1 -- 172.21.15.112:0/3080550251 wait complete. 2024-04-25T18:48:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.218+0000 7f466f400640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f465c0a3e50 0x7f465c0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.219+0000 7f466f400640 1 -- 172.21.15.112:0/3883240761 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f46640d0710 con 0x7f4678074c40 2024-04-25T18:48:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.219+0000 7f4680a80740 1 -- start start 2024-04-25T18:48:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.219+0000 7f4680a80740 1 Processor -- start 2024-04-25T18:48:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.219+0000 7f4676a00640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f465c0a3e50 0x7f465c0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.220+0000 7f4680a80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 0x55d1d934eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.220+0000 7f4680a80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93472d0 0x55d1d934f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.220+0000 7f4680a80740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d9351b90 0x55d1d934f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.220+0000 7f4680a80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d1d9354760 con 0x55d1d93469b0 2024-04-25T18:48:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.220+0000 7f4680a80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d1d9266310 con 0x55d1d93472d0 2024-04-25T18:48:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.220+0000 7f4680a80740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d1d93548e0 con 0x55d1d9351b90 2024-04-25T18:48:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.220+0000 7f4680a80740 10 client.0 _ll_register_callbacks cb 0x55d1d9234cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:21.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4676000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93472d0 0x55d1d934f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4677400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d9351b90 0x55d1d934f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4676000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93472d0 0x55d1d934f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:40278/0 (socket says 172.21.15.112:40278) 2024-04-25T18:48:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4676000640 1 -- 172.21.15.112:0/3555391589 learned_addr learned my addr 172.21.15.112:0/3555391589 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4676000640 1 -- 172.21.15.112:0/3555391589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d9351b90 msgr2=0x55d1d934f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4676000640 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d9351b90 0x55d1d934f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4676000640 1 -- 172.21.15.112:0/3555391589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 msgr2=0x55d1d934eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4676a00640 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 0x55d1d934eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4676000640 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 0x55d1d934eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4676000640 1 -- 172.21.15.112:0/3555391589 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d908b620 con 0x55d1d93472d0 2024-04-25T18:48:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4676a00640 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 0x55d1d934eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:48:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.221+0000 7f4676a00640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f465c0a3e50 0x7f465c0a6310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46680096c0 tx=0x7f4668009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.222+0000 7f4676000640 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93472d0 0x55d1d934f080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f466400c6a0 tx=0x7f466409bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c07e036b203013de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.222+0000 7f4660c00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.222+0000 7f4660c00640 1 -- 172.21.15.112:0/3555391589 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46640b4020 con 0x55d1d93472d0 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.222+0000 7f4660c00640 1 -- 172.21.15.112:0/3555391589 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f466409bac0 con 0x55d1d93472d0 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.222+0000 7f4680a80740 1 -- 172.21.15.112:0/3555391589 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46680047a0 con 0x55d1d93472d0 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.222+0000 7f4660c00640 1 -- 172.21.15.112:0/3555391589 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46640ba050 con 0x55d1d93472d0 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.222+0000 7f4680a80740 10 client.25298 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.222+0000 7f4680a80740 1 -- 172.21.15.112:0/3555391589 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d1d93607d0 con 0x55d1d93472d0 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.223+0000 7f4660c00640 1 -- 172.21.15.112:0/3555391589 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46680047a0 con 0x55d1d93472d0 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.223+0000 7f4660c00640 1 -- 172.21.15.112:0/3555391589 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f466401f060 con 0x55d1d93472d0 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.223+0000 7f4680a80740 10 client.25298 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.223+0000 7f4680a80740 10 client.25298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:48:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.223+0000 7f4680a80740 20 client.25298 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:21.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.223+0000 7f4680a80740 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55d1d9361c90 0x55d1d9364080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.223+0000 7f4680a80740 4 client.25298 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:48:21.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.223+0000 7f4680a80740 1 -- 172.21.15.112:0/3555391589 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1d92c08b0 con 0x55d1d9361c90 2024-04-25T18:48:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.223+0000 7f4676a00640 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55d1d9361c90 0x55d1d9364080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.224+0000 7f4676a00640 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55d1d9361c90 0x55d1d9364080 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd41dacde83e326 server_cookie=a63d88970ab1785c in_seq=0 out_seq=0 2024-04-25T18:48:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.224+0000 7f4660c00640 10 client.25298 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.225+0000 7f4660c00640 1 -- 172.21.15.112:0/3555391589 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1d92c08b0 con 0x55d1d9361c90 2024-04-25T18:48:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.225+0000 7f4660c00640 10 client.25298 handle_command_reply: tid=0 2024-04-25T18:48:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.225+0000 7f4680a80740 1 client.25298 shutdown 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.225+0000 7f4680a80740 1 -- 172.21.15.112:0/3555391589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93472d0 msgr2=0x55d1d934f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.225+0000 7f4680a80740 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93472d0 0x55d1d934f080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f466400c6a0 tx=0x7f466409bfb0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.226+0000 7f4680a80740 1 -- 172.21.15.112:0/3555391589 shutdown_connections 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.226+0000 7f4680a80740 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d9351b90 0x55d1d934f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.226+0000 7f4680a80740 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d93472d0 0x55d1d934f080 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.226+0000 7f4680a80740 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55d1d9361c90 0x55d1d9364080 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.226+0000 7f4680a80740 1 --2- 172.21.15.112:0/3555391589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 0x55d1d934eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.226+0000 7f4660c00640 0 client.25298 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.226+0000 7f4680a80740 1 -- 172.21.15.112:0/3555391589 >> 172.21.15.112:0/3555391589 conn(0x55d1d9342ec0 msgr2=0x55d1d9348bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.226+0000 7f4680a80740 1 -- 172.21.15.112:0/3555391589 shutdown_connections 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.226+0000 7f4680a80740 1 -- 172.21.15.112:0/3555391589 wait complete. 2024-04-25T18:48:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.226+0000 7f4680a80740 20 client.25298 trim_cache size 0 max 16384 2024-04-25T18:48:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.256+0000 7f4680a80740 1 Processor -- start 2024-04-25T18:48:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.256+0000 7f4680a80740 1 -- start start 2024-04-25T18:48:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.256+0000 7f4680a80740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93469b0 0x55d1d9349370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.256+0000 7f4680a80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93472d0 0x55d1d934f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.257+0000 7f4680a80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d9351b90 0x55d1d9350a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.257+0000 7f4680a80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46781bad60 con 0x55d1d93472d0 2024-04-25T18:48:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.257+0000 7f4680a80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f46781b8630 con 0x55d1d9351b90 2024-04-25T18:48:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.257+0000 7f4680a80740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f46780722d0 con 0x55d1d93469b0 2024-04-25T18:48:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.257+0000 7f4676000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93472d0 0x55d1d934f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.257+0000 7f4677400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d9351b90 0x55d1d9350a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.257+0000 7f4676000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93472d0 0x55d1d934f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39540/0 (socket says 172.21.15.112:39540) 2024-04-25T18:48:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.257+0000 7f4676000640 1 -- 172.21.15.112:0/4013733253 learned_addr learned my addr 172.21.15.112:0/4013733253 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.257+0000 7f4676a00640 1 --2- 172.21.15.112:0/4013733253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93469b0 0x55d1d9349370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.258+0000 7f4676a00640 1 -- 172.21.15.112:0/4013733253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d9351b90 msgr2=0x55d1d9350a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.258+0000 7f4676a00640 1 --2- 172.21.15.112:0/4013733253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d9351b90 0x55d1d9350a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.258+0000 7f4676a00640 1 -- 172.21.15.112:0/4013733253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93472d0 msgr2=0x55d1d934f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.258+0000 7f4676a00640 1 --2- 172.21.15.112:0/4013733253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93472d0 0x55d1d934f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.258+0000 7f4676a00640 1 -- 172.21.15.112:0/4013733253 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d93643f0 con 0x55d1d93469b0 2024-04-25T18:48:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.258+0000 7f4677400640 1 --2- 172.21.15.112:0/4013733253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d9351b90 0x55d1d9350a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.258+0000 7f4676a00640 1 --2- 172.21.15.112:0/4013733253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93469b0 0x55d1d9349370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4668005ab0 tx=0x7f4668000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bb372cf75a00e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.259+0000 7f4663400640 1 -- 172.21.15.112:0/4013733253 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668040070 con 0x55d1d93469b0 2024-04-25T18:48:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.259+0000 7f4663400640 1 -- 172.21.15.112:0/4013733253 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4668003be0 con 0x55d1d93469b0 2024-04-25T18:48:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.259+0000 7f4663400640 1 -- 172.21.15.112:0/4013733253 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668035040 con 0x55d1d93469b0 2024-04-25T18:48:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.260+0000 7f4680a80740 1 -- 172.21.15.112:0/4013733253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93469b0 msgr2=0x55d1d9349370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.260+0000 7f4680a80740 1 --2- 172.21.15.112:0/4013733253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93469b0 0x55d1d9349370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4668005ab0 tx=0x7f4668000bc0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.260+0000 7f4680a80740 1 -- 172.21.15.112:0/4013733253 shutdown_connections 2024-04-25T18:48:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.260+0000 7f4680a80740 1 --2- 172.21.15.112:0/4013733253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d9351b90 0x55d1d9350a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.260+0000 7f4680a80740 1 --2- 172.21.15.112:0/4013733253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93472d0 0x55d1d934f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.260+0000 7f4680a80740 1 --2- 172.21.15.112:0/4013733253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93469b0 0x55d1d9349370 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.260+0000 7f4680a80740 1 -- 172.21.15.112:0/4013733253 >> 172.21.15.112:0/4013733253 conn(0x55d1d9342ec0 msgr2=0x55d1d9350370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.260+0000 7f4680a80740 1 -- 172.21.15.112:0/4013733253 shutdown_connections 2024-04-25T18:48:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.260+0000 7f4680a80740 1 -- 172.21.15.112:0/4013733253 wait complete. 2024-04-25T18:48:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.260+0000 7f4680a80740 1 -- start start 2024-04-25T18:48:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.261+0000 7f4680a80740 1 Processor -- start 2024-04-25T18:48:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.261+0000 7f4680a80740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 0x55d1d93bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.261+0000 7f4680a80740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 0x55d1d93bf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.261+0000 7f4680a80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d9351b90 0x55d1d9356f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.261+0000 7f4680a80740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4678072940 con 0x55d1d93469b0 2024-04-25T18:48:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4680a80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4678073db0 con 0x55d1d9351b90 2024-04-25T18:48:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4680a80740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4678073010 con 0x55d1d93472d0 2024-04-25T18:48:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4680a80740 10 client.0 _ll_register_callbacks cb 0x7f4664098710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4676a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 0x55d1d93bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4676000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 0x55d1d93bf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4676a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 0x55d1d93bf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39550/0 (socket says 172.21.15.112:39550) 2024-04-25T18:48:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4676a00640 1 -- 172.21.15.112:0/2976183572 learned_addr learned my addr 172.21.15.112:0/2976183572 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4677400640 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d9351b90 0x55d1d9356f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4676a00640 1 -- 172.21.15.112:0/2976183572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 msgr2=0x55d1d93bf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4676a00640 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 0x55d1d93bf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4676a00640 1 -- 172.21.15.112:0/2976183572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d9351b90 msgr2=0x55d1d9356f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4676a00640 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d9351b90 0x55d1d9356f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.262+0000 7f4676a00640 1 -- 172.21.15.112:0/2976183572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d93576a0 con 0x55d1d93469b0 2024-04-25T18:48:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.263+0000 7f4676a00640 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 0x55d1d93bf490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4668004820 tx=0x7f466802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6bd81526c34bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.263+0000 7f4660c00640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:48:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.263+0000 7f4660c00640 1 -- 172.21.15.112:0/2976183572 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466805e020 con 0x55d1d93469b0 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.263+0000 7f4660c00640 1 -- 172.21.15.112:0/2976183572 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f4668003730 con 0x55d1d93469b0 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.263+0000 7f4680a80740 1 -- 172.21.15.112:0/2976183572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4668003be0 con 0x55d1d93469b0 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.264+0000 7f4660c00640 1 -- 172.21.15.112:0/2976183572 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4668035040 con 0x55d1d93469b0 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.264+0000 7f4680a80740 10 client.15171 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.264+0000 7f4680a80740 1 -- 172.21.15.112:0/2976183572 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d1d93c7ca0 con 0x55d1d93469b0 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.264+0000 7f4660c00640 1 -- 172.21.15.112:0/2976183572 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4668003be0 con 0x55d1d93469b0 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.264+0000 7f4660c00640 1 -- 172.21.15.112:0/2976183572 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f466804c040 con 0x55d1d93469b0 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.264+0000 7f4680a80740 10 client.15171 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.264+0000 7f4680a80740 10 client.15171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.264+0000 7f4680a80740 20 client.15171 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.264+0000 7f4680a80740 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55d1d93c7f90 0x55d1d93ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.264+0000 7f4680a80740 4 client.15171 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:48:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.264+0000 7f4680a80740 1 -- 172.21.15.112:0/2976183572 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1d92ba320 con 0x55d1d93c7f90 2024-04-25T18:48:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.265+0000 7f4676000640 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55d1d93c7f90 0x55d1d93ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.265+0000 7f4676000640 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55d1d93c7f90 0x55d1d93ca380 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=624ec2f1388759bf server_cookie=c65aa247b740db1e in_seq=0 out_seq=0 2024-04-25T18:48:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.265+0000 7f4660c00640 10 client.15171 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.269+0000 7f4660c00640 1 -- 172.21.15.112:0/2976183572 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1d92ba320 con 0x55d1d93c7f90 2024-04-25T18:48:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.269+0000 7f4660c00640 10 client.15171 handle_command_reply: tid=0 2024-04-25T18:48:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.269+0000 7f4680a80740 1 client.15171 shutdown 2024-04-25T18:48:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.270+0000 7f4680a80740 1 -- 172.21.15.112:0/2976183572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 msgr2=0x55d1d93bf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:21.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.270+0000 7f4680a80740 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 0x55d1d93bf490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4668004820 tx=0x7f466802a480 comp rx=0 tx=0).stop 2024-04-25T18:48:21.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.270+0000 7f4680a80740 1 -- 172.21.15.112:0/2976183572 shutdown_connections 2024-04-25T18:48:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.270+0000 7f4680a80740 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1d9351b90 0x55d1d9356f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.270+0000 7f4680a80740 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d1d93472d0 0x55d1d93bf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.270+0000 7f4680a80740 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55d1d93c7f90 0x55d1d93ca380 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.270+0000 7f4680a80740 1 --2- 172.21.15.112:0/2976183572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d1d93469b0 0x55d1d93bf490 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.270+0000 7f4680a80740 1 -- 172.21.15.112:0/2976183572 >> 172.21.15.112:0/2976183572 conn(0x55d1d9342ec0 msgr2=0x55d1d9350370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.270+0000 7f4660c00640 0 client.15171 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.271+0000 7f4680a80740 1 -- 172.21.15.112:0/2976183572 shutdown_connections 2024-04-25T18:48:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.271+0000 7f4680a80740 1 -- 172.21.15.112:0/2976183572 wait complete. 2024-04-25T18:48:21.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.271+0000 7f4680a80740 20 client.15171 trim_cache size 0 max 16384 2024-04-25T18:48:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:48:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:48:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:48:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:48:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 -- 172.21.15.112:0/3883240761 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f465c0a3e50 msgr2=0x7f465c0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f465c0a3e50 0x7f465c0a6310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46680096c0 tx=0x7f4668009260 comp rx=0 tx=0).stop 2024-04-25T18:48:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 -- 172.21.15.112:0/3883240761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4678074c40 msgr2=0x7f46781a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4678074c40 0x7f46781a0ac0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f466400ea20 tx=0x7f466400eee0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 -- 172.21.15.112:0/3883240761 shutdown_connections 2024-04-25T18:48:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f465c0a3e50 0x7f465c0a6310 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f467810e610 0x7f467819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4678074c40 0x7f46781a0ac0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 --2- 172.21.15.112:0/3883240761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4678074320 0x7f46781a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 -- 172.21.15.112:0/3883240761 >> 172.21.15.112:0/3883240761 conn(0x7f46780fd8a0 msgr2=0x7f46780fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 -- 172.21.15.112:0/3883240761 shutdown_connections 2024-04-25T18:48:21.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:21.272+0000 7f4677e00640 1 -- 172.21.15.112:0/3883240761 wait complete. 2024-04-25T18:48:22.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:22 smithi171 ceph-mon[33266]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T18:48:22.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:22 smithi161 ceph-mon[32609]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T18:48:22.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:22 smithi112 ceph-mon[27864]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T18:48:24.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:24 smithi171 ceph-mon[33266]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T18:48:24.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:24 smithi161 ceph-mon[32609]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T18:48:24.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:24 smithi112 ceph-mon[27864]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-25T18:48:26.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:26 smithi171 ceph-mon[33266]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:48:26.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:26 smithi161 ceph-mon[32609]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:48:26.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:26 smithi112 ceph-mon[27864]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:48:28.379 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-25T18:48:28.380 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.8 2024-04-25T18:48:28.380 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:48:28.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:28 smithi171 ceph-mon[33266]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T18:48:28.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:28 smithi161 ceph-mon[32609]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T18:48:28.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:28 smithi112 ceph-mon[27864]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T18:48:28.833 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:48:30.678 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:30 smithi112 ceph-mon[27864]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:48:30.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:30 smithi171 ceph-mon[33266]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:48:30.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:30 smithi161 ceph-mon[32609]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:48:31.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.252+0000 7f88f5400700 1 -- 172.21.15.112:0/179890611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88f0105480 msgr2=0x7f88f0105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:31.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.252+0000 7f88f5400700 1 --2- 172.21.15.112:0/179890611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88f0105480 0x7f88f0105910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f88e800b0a0 tx=0x7f88e800b3b0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.252+0000 7f88f5400700 1 -- 172.21.15.112:0/179890611 shutdown_connections 2024-04-25T18:48:31.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.252+0000 7f88f5400700 1 --2- 172.21.15.112:0/179890611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88f0105480 0x7f88f0105910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.252+0000 7f88f5400700 1 --2- 172.21.15.112:0/179890611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88f0107970 0x7f88f006a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.252+0000 7f88f5400700 1 --2- 172.21.15.112:0/179890611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88f006ac90 0x7f88f006b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.252+0000 7f88f5400700 1 -- 172.21.15.112:0/179890611 >> 172.21.15.112:0/179890611 conn(0x7f88f00788f0 msgr2=0x7f88f0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:31.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.253+0000 7f88f5400700 1 -- 172.21.15.112:0/179890611 shutdown_connections 2024-04-25T18:48:31.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.253+0000 7f88f5400700 1 -- 172.21.15.112:0/179890611 wait complete. 2024-04-25T18:48:31.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.254+0000 7f88f5400700 1 Processor -- start 2024-04-25T18:48:31.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.254+0000 7f88f5400700 1 -- start start 2024-04-25T18:48:31.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.255+0000 7f88f5400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88f006ac90 0x7f88f019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:31.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.255+0000 7f88f5400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88f0105480 0x7f88f019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:31.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.255+0000 7f88f5400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88f0107970 0x7f88f01a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:31.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.255+0000 7f88f5400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88f0119230 con 0x7f88f0105480 2024-04-25T18:48:31.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.255+0000 7f88f5400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01190b0 con 0x7f88f006ac90 2024-04-25T18:48:31.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.255+0000 7f88f5400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01193b0 con 0x7f88f0107970 2024-04-25T18:48:31.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.255+0000 7f88ee800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88f006ac90 0x7f88f019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.255+0000 7f88ee800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88f006ac90 0x7f88f019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:38192/0 (socket says 172.21.15.112:38192) 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.255+0000 7f88ee800700 1 -- 172.21.15.112:0/1134866019 learned_addr learned my addr 172.21.15.112:0/1134866019 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.255+0000 7f88ef200700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88f0107970 0x7f88f01a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.256+0000 7f88ee800700 1 -- 172.21.15.112:0/1134866019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88f0107970 msgr2=0x7f88f01a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.256+0000 7f88ee800700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88f0107970 0x7f88f01a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.256+0000 7f88ee800700 1 -- 172.21.15.112:0/1134866019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88f0105480 msgr2=0x7f88f019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.256+0000 7f88ee800700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88f0105480 0x7f88f019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.256+0000 7f88ee800700 1 -- 172.21.15.112:0/1134866019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f01a4730 con 0x7f88f006ac90 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.256+0000 7f88ee800700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88f006ac90 0x7f88f019f720 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88e000e9c0 tx=0x7f88e000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.256+0000 7f88e7400700 1 -- 172.21.15.112:0/1134866019 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e000cbd0 con 0x7f88f006ac90 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.256+0000 7f88e7400700 1 -- 172.21.15.112:0/1134866019 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e000cd70 con 0x7f88f006ac90 2024-04-25T18:48:31.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.256+0000 7f88e7400700 1 -- 172.21.15.112:0/1134866019 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0005610 con 0x7f88f006ac90 2024-04-25T18:48:31.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.257+0000 7f88f5400700 1 -- 172.21.15.112:0/1134866019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f01a4a20 con 0x7f88f006ac90 2024-04-25T18:48:31.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.257+0000 7f88f5400700 1 -- 172.21.15.112:0/1134866019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f01ac2a0 con 0x7f88f006ac90 2024-04-25T18:48:31.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.258+0000 7f88f5400700 1 -- 172.21.15.112:0/1134866019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f0069730 con 0x7f88f006ac90 2024-04-25T18:48:31.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.260+0000 7f88e7400700 1 -- 172.21.15.112:0/1134866019 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f88e0004750 con 0x7f88f006ac90 2024-04-25T18:48:31.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.262+0000 7f88e7400700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88d40a3fe0 0x7f88d40a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.263+0000 7f88e7400700 1 -- 172.21.15.112:0/1134866019 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f88e00d0f30 con 0x7f88f006ac90 2024-04-25T18:48:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.263+0000 7f88ede00700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88d40a3fe0 0x7f88d40a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:31.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.264+0000 7f88ede00700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88d40a3fe0 0x7f88d40a64a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88dc005950 tx=0x7f88dc0058e0 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:31.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.264+0000 7f88e7400700 1 -- 172.21.15.112:0/1134866019 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88e0053ff0 con 0x7f88f006ac90 2024-04-25T18:48:31.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.535+0000 7f88f5400700 1 -- 172.21.15.112:0/1134866019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88f01a0a80 con 0x7f88f006ac90 2024-04-25T18:48:31.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.536+0000 7f88e7400700 1 -- 172.21.15.112:0/1134866019 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7f88e009d390 con 0x7f88f006ac90 2024-04-25T18:48:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:48:31.538 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:47:25.174463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24493,24511,24512]},"id":1}]} 2024-04-25T18:48:31.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.539+0000 7f88f5400700 1 -- 172.21.15.112:0/1134866019 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88d40a3fe0 msgr2=0x7f88d40a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:31.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.539+0000 7f88f5400700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88d40a3fe0 0x7f88d40a64a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88dc005950 tx=0x7f88dc0058e0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.539+0000 7f88f5400700 1 -- 172.21.15.112:0/1134866019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88f006ac90 msgr2=0x7f88f019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:31.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.539+0000 7f88f5400700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88f006ac90 0x7f88f019f720 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88e000e9c0 tx=0x7f88e000ecd0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.540+0000 7f88f5400700 1 -- 172.21.15.112:0/1134866019 shutdown_connections 2024-04-25T18:48:31.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.540+0000 7f88f5400700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88d40a3fe0 0x7f88d40a64a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.540+0000 7f88f5400700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88f0107970 0x7f88f01a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.540+0000 7f88f5400700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88f006ac90 0x7f88f019f720 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.540+0000 7f88f5400700 1 --2- 172.21.15.112:0/1134866019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88f0105480 0x7f88f019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.540+0000 7f88f5400700 1 -- 172.21.15.112:0/1134866019 >> 172.21.15.112:0/1134866019 conn(0x7f88f00788f0 msgr2=0x7f88f0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:31.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.540+0000 7f88f5400700 1 -- 172.21.15.112:0/1134866019 shutdown_connections 2024-04-25T18:48:31.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:31.540+0000 7f88f5400700 1 -- 172.21.15.112:0/1134866019 wait complete. 2024-04-25T18:48:31.544 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-04-25T18:48:32.564 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24512 quiesce db / --timeout 60 --expiration 80.8 --await-for 42.0' 2024-04-25T18:48:32.564 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24512 quiesce db / --timeout 60 --expiration 80.8 --await-for 42.0 2024-04-25T18:48:32.618 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:32 smithi112 ceph-mon[27864]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:48:32.618 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:32 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1134866019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:48:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:32 smithi171 ceph-mon[33266]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:48:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:32 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1134866019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:48:32.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:32 smithi161 ceph-mon[32609]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:48:32.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:32 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1134866019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:48:33.014 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:48:33.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:33 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:33.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:33 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:33.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:33 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:34.359 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:34 smithi112 ceph-mon[27864]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T18:48:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.438+0000 7f9449400640 1 -- 172.21.15.112:0/3744572179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9444100b10 msgr2=0x7f9444100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.438+0000 7f9449400640 1 --2- 172.21.15.112:0/3744572179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9444100b10 0x7f9444100ef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9434009900 tx=0x7f943402a410 comp rx=0 tx=0).stop 2024-04-25T18:48:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.439+0000 7f9449400640 1 -- 172.21.15.112:0/3744572179 shutdown_connections 2024-04-25T18:48:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.439+0000 7f9449400640 1 --2- 172.21.15.112:0/3744572179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f944410e6c0 0x7f9444110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.439+0000 7f9449400640 1 --2- 172.21.15.112:0/3744572179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94441014c0 0x7f944410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.439+0000 7f9449400640 1 --2- 172.21.15.112:0/3744572179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9444100b10 0x7f9444100ef0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.439+0000 7f9449400640 1 -- 172.21.15.112:0/3744572179 >> 172.21.15.112:0/3744572179 conn(0x7f94440fc770 msgr2=0x7f94440feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.439+0000 7f9449400640 1 -- 172.21.15.112:0/3744572179 shutdown_connections 2024-04-25T18:48:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.439+0000 7f9449400640 1 -- 172.21.15.112:0/3744572179 wait complete. 2024-04-25T18:48:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.440+0000 7f9449400640 1 Processor -- start 2024-04-25T18:48:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.440+0000 7f9449400640 1 -- start start 2024-04-25T18:48:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.440+0000 7f9449400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9444100b10 0x7f94441a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.440+0000 7f9449400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94441014c0 0x7f94441a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.440+0000 7f9449400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944410e6c0 0x7f944419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.440+0000 7f9449400640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9444114250 con 0x7f944410e6c0 2024-04-25T18:48:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.440+0000 7f9449400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94441140d0 con 0x7f9444100b10 2024-04-25T18:48:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.440+0000 7f9449400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94441143d0 con 0x7f94441014c0 2024-04-25T18:48:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.441+0000 7f9448a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944410e6c0 0x7f944419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.441+0000 7f9448a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944410e6c0 0x7f944419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54264/0 (socket says 172.21.15.112:54264) 2024-04-25T18:48:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.441+0000 7f9443e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9444100b10 0x7f94441a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.441+0000 7f9448a00640 1 -- 172.21.15.112:0/3760265471 learned_addr learned my addr 172.21.15.112:0/3760265471 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.441+0000 7f9443400640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94441014c0 0x7f94441a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.442+0000 7f9448a00640 1 -- 172.21.15.112:0/3760265471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94441014c0 msgr2=0x7f94441a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.442+0000 7f9448a00640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94441014c0 0x7f94441a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.442+0000 7f9448a00640 1 -- 172.21.15.112:0/3760265471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9444100b10 msgr2=0x7f94441a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.442+0000 7f9448a00640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9444100b10 0x7f94441a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.442+0000 7f9448a00640 1 -- 172.21.15.112:0/3760265471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944419ad70 con 0x7f944410e6c0 2024-04-25T18:48:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.442+0000 7f9443400640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94441014c0 0x7f94441a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.442+0000 7f9448a00640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944410e6c0 0x7f944419a5f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f943c00eb10 tx=0x7f943c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.443+0000 7f9440c00640 1 -- 172.21.15.112:0/3760265471 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c004b20 con 0x7f944410e6c0 2024-04-25T18:48:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.443+0000 7f9440c00640 1 -- 172.21.15.112:0/3760265471 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c004cc0 con 0x7f944410e6c0 2024-04-25T18:48:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.443+0000 7f9449400640 1 -- 172.21.15.112:0/3760265471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944419b060 con 0x7f944410e6c0 2024-04-25T18:48:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.443+0000 7f9440c00640 1 -- 172.21.15.112:0/3760265471 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c018760 con 0x7f944410e6c0 2024-04-25T18:48:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.443+0000 7f9449400640 1 -- 172.21.15.112:0/3760265471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94441a7300 con 0x7f944410e6c0 2024-04-25T18:48:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.446+0000 7f944d883740 1 Processor -- start 2024-04-25T18:48:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.449+0000 7f9440c00640 1 -- 172.21.15.112:0/3760265471 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f943c018900 con 0x7f944410e6c0 2024-04-25T18:48:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.449+0000 7f944d883740 1 -- start start 2024-04-25T18:48:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.449+0000 7f944d883740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026ca9b0 0x5584026cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.449+0000 7f944d883740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026cb2d0 0x5584026d5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.450+0000 7f944d883740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026d5b90 0x5584026d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.450+0000 7f944d883740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558402630900 con 0x5584026ca9b0 2024-04-25T18:48:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.450+0000 7f944d883740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5584025d6ef0 con 0x5584026cb2d0 2024-04-25T18:48:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.450+0000 7f944d883740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55840262c8c0 con 0x5584026d5b90 2024-04-25T18:48:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.450+0000 7f9443400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026ca9b0 0x5584026cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.450+0000 7f9443400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026ca9b0 0x5584026cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54278/0 (socket says 172.21.15.112:54278) 2024-04-25T18:48:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.451+0000 7f9443400640 1 -- 172.21.15.112:0/380587701 learned_addr learned my addr 172.21.15.112:0/380587701 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.451+0000 7f9440c00640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f94280a4090 0x7f94280a6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.451+0000 7f9440c00640 1 -- 172.21.15.112:0/3760265471 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f943c0d0f40 con 0x7f944410e6c0 2024-04-25T18:48:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.452+0000 7f9443400640 1 --2- 172.21.15.112:0/380587701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026d5b90 0x5584026d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.452+0000 7f9443400640 1 -- 172.21.15.112:0/380587701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026d5b90 msgr2=0x5584026d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.452+0000 7f9443400640 1 --2- 172.21.15.112:0/380587701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026d5b90 0x5584026d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.452+0000 7f9443400640 1 -- 172.21.15.112:0/380587701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026cb2d0 msgr2=0x5584026d5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.452+0000 7f9443400640 1 --2- 172.21.15.112:0/380587701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026cb2d0 0x5584026d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.452+0000 7f9443400640 1 -- 172.21.15.112:0/380587701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558402343210 con 0x5584026ca9b0 2024-04-25T18:48:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.452+0000 7f9448a00640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f94280a4090 0x7f94280a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.453+0000 7f9443400640 1 --2- 172.21.15.112:0/380587701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026ca9b0 0x5584026cad90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9430013860 tx=0x7f943000ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df1c0885d69ee83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.453+0000 7f9439200640 1 -- 172.21.15.112:0/380587701 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430019030 con 0x5584026ca9b0 2024-04-25T18:48:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.453+0000 7f9439200640 1 -- 172.21.15.112:0/380587701 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94300132e0 con 0x5584026ca9b0 2024-04-25T18:48:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.453+0000 7f9439200640 1 -- 172.21.15.112:0/380587701 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430010b70 con 0x5584026ca9b0 2024-04-25T18:48:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.453+0000 7f9448a00640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f94280a4090 0x7f94280a6550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f943c0ce4b0 tx=0x7f943c005b10 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.454+0000 7f944d883740 1 -- 172.21.15.112:0/380587701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026ca9b0 msgr2=0x5584026cad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.454+0000 7f944d883740 1 --2- 172.21.15.112:0/380587701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026ca9b0 0x5584026cad90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9430013860 tx=0x7f943000ac20 comp rx=0 tx=0).stop 2024-04-25T18:48:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.455+0000 7f944d883740 1 -- 172.21.15.112:0/380587701 shutdown_connections 2024-04-25T18:48:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.455+0000 7f944d883740 1 --2- 172.21.15.112:0/380587701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026d5b90 0x5584026d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.455+0000 7f944d883740 1 --2- 172.21.15.112:0/380587701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026cb2d0 0x5584026d5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.455+0000 7f944d883740 1 --2- 172.21.15.112:0/380587701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026ca9b0 0x5584026cad90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.455+0000 7f944d883740 1 -- 172.21.15.112:0/380587701 >> 172.21.15.112:0/380587701 conn(0x5584026c6ec0 msgr2=0x5584026c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.455+0000 7f944d883740 1 -- 172.21.15.112:0/380587701 shutdown_connections 2024-04-25T18:48:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.455+0000 7f944d883740 1 -- 172.21.15.112:0/380587701 wait complete. 2024-04-25T18:48:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.456+0000 7f944d883740 1 -- start start 2024-04-25T18:48:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.456+0000 7f944d883740 1 Processor -- start 2024-04-25T18:48:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.457+0000 7f944d883740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026d2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.457+0000 7f944d883740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 0x5584026d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.457+0000 7f944d883740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026d5b90 0x5584026d3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.457+0000 7f944d883740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5584026d87c0 con 0x5584026cb2d0 2024-04-25T18:48:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.457+0000 7f944d883740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5584025ea310 con 0x5584026d5b90 2024-04-25T18:48:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.457+0000 7f944d883740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5584026d8940 con 0x5584026ca9b0 2024-04-25T18:48:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.457+0000 7f944d883740 10 client.0 _ll_register_callbacks cb 0x5584025b8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.458+0000 7f9443400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026d5b90 0x5584026d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.458+0000 7f9443400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026d5b90 0x5584026d3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:38234/0 (socket says 172.21.15.112:38234) 2024-04-25T18:48:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.458+0000 7f9443400640 1 -- 172.21.15.112:0/3467547828 learned_addr learned my addr 172.21.15.112:0/3467547828 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.458+0000 7f9443400640 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026d2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.459+0000 7f9443e00640 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 0x5584026d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.459+0000 7f9443e00640 1 -- 172.21.15.112:0/3467547828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 msgr2=0x5584026d2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.459+0000 7f9443e00640 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026d2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.459+0000 7f9443e00640 1 -- 172.21.15.112:0/3467547828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026d5b90 msgr2=0x5584026d3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.459+0000 7f9443e00640 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026d5b90 0x5584026d3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.459+0000 7f9443e00640 1 -- 172.21.15.112:0/3467547828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55840240f620 con 0x5584026cb2d0 2024-04-25T18:48:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.460+0000 7f9443e00640 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 0x5584026d3030 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f943402aa00 tx=0x7f9434004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ab640681baf8c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.460+0000 7f942d600640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:48:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.461+0000 7f942d600640 1 -- 172.21.15.112:0/3467547828 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434035040 con 0x5584026cb2d0 2024-04-25T18:48:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.461+0000 7f942d600640 1 -- 172.21.15.112:0/3467547828 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f9434005040 con 0x5584026cb2d0 2024-04-25T18:48:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.461+0000 7f944d883740 1 -- 172.21.15.112:0/3467547828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5584026448b0 con 0x5584026cb2d0 2024-04-25T18:48:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.461+0000 7f942d600640 1 -- 172.21.15.112:0/3467547828 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9434005240 con 0x5584026cb2d0 2024-04-25T18:48:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.461+0000 7f944d883740 10 client.15186 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.461+0000 7f944d883740 1 -- 172.21.15.112:0/3467547828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584026e4ac0 con 0x5584026cb2d0 2024-04-25T18:48:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.462+0000 7f942d600640 1 -- 172.21.15.112:0/3467547828 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5584026448b0 con 0x5584026cb2d0 2024-04-25T18:48:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.463+0000 7f942d600640 1 -- 172.21.15.112:0/3467547828 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9434049070 con 0x5584026cb2d0 2024-04-25T18:48:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.463+0000 7f944d883740 10 client.15186 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.463+0000 7f944d883740 10 client.15186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:48:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.463+0000 7f944d883740 20 client.15186 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.463+0000 7f944d883740 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5584026e5f80 0x5584026e8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.463+0000 7f944d883740 4 client.15186 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:48:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.463+0000 7f944d883740 1 -- 172.21.15.112:0/3467547828 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55840263e320 con 0x5584026e5f80 2024-04-25T18:48:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.463+0000 7f9443400640 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5584026e5f80 0x5584026e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.464+0000 7f9443400640 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5584026e5f80 0x5584026e8370 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0cd994250de1745 server_cookie=526f13feab585e38 in_seq=0 out_seq=0 2024-04-25T18:48:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.464+0000 7f942d600640 10 client.15186 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.465+0000 7f942d600640 1 -- 172.21.15.112:0/3467547828 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55840263e320 con 0x5584026e5f80 2024-04-25T18:48:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.465+0000 7f942d600640 10 client.15186 handle_command_reply: tid=0 2024-04-25T18:48:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.465+0000 7f944d883740 1 client.15186 shutdown 2024-04-25T18:48:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.465+0000 7f944d883740 1 -- 172.21.15.112:0/3467547828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 msgr2=0x5584026d3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.465+0000 7f944d883740 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 0x5584026d3030 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f943402aa00 tx=0x7f9434004270 comp rx=0 tx=0).stop 2024-04-25T18:48:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.466+0000 7f944d883740 1 -- 172.21.15.112:0/3467547828 shutdown_connections 2024-04-25T18:48:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.466+0000 7f944d883740 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026d5b90 0x5584026d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.466+0000 7f944d883740 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5584026e5f80 0x5584026e8370 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.466+0000 7f944d883740 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 0x5584026d3030 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.466+0000 7f944d883740 1 --2- 172.21.15.112:0/3467547828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026d2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.466+0000 7f942d600640 0 client.15186 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.466+0000 7f944d883740 1 -- 172.21.15.112:0/3467547828 >> 172.21.15.112:0/3467547828 conn(0x5584026c6ec0 msgr2=0x5584026ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.466+0000 7f944d883740 1 -- 172.21.15.112:0/3467547828 shutdown_connections 2024-04-25T18:48:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.466+0000 7f944d883740 1 -- 172.21.15.112:0/3467547828 wait complete. 2024-04-25T18:48:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.466+0000 7f944d883740 20 client.15186 trim_cache size 0 max 16384 2024-04-25T18:48:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.498+0000 7f944d883740 1 Processor -- start 2024-04-25T18:48:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.498+0000 7f944d883740 1 -- start start 2024-04-25T18:48:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.498+0000 7f944d883740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026e16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.498+0000 7f944d883740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026cb2d0 0x5584026d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.498+0000 7f944d883740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026d5b90 0x5584026d4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.498+0000 7f944d883740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94441ae180 con 0x5584026d5b90 2024-04-25T18:48:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.498+0000 7f944d883740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94441aba50 con 0x5584026cb2d0 2024-04-25T18:48:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.498+0000 7f944d883740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94441a98a0 con 0x5584026ca9b0 2024-04-25T18:48:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.499+0000 7f9443400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026d5b90 0x5584026d4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.499+0000 7f9443400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026e16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.499+0000 7f9443400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026d5b90 0x5584026d4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54306/0 (socket says 172.21.15.112:54306) 2024-04-25T18:48:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.499+0000 7f9443400640 1 -- 172.21.15.112:0/3201831803 learned_addr learned my addr 172.21.15.112:0/3201831803 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.499+0000 7f9443e00640 1 --2- 172.21.15.112:0/3201831803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026cb2d0 0x5584026d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.500+0000 7f9443e00640 1 -- 172.21.15.112:0/3201831803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 msgr2=0x5584026e16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.500+0000 7f9443e00640 1 --2- 172.21.15.112:0/3201831803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026e16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.500+0000 7f9443e00640 1 -- 172.21.15.112:0/3201831803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026d5b90 msgr2=0x5584026d4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.500+0000 7f9443e00640 1 --2- 172.21.15.112:0/3201831803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026d5b90 0x5584026d4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.500+0000 7f9443e00640 1 -- 172.21.15.112:0/3201831803 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584026cbc60 con 0x5584026cb2d0 2024-04-25T18:48:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.500+0000 7f9443400640 1 --2- 172.21.15.112:0/3201831803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026e16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:48:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.500+0000 7f9443e00640 1 --2- 172.21.15.112:0/3201831803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026cb2d0 0x5584026d3390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94340037d0 tx=0x7f943402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f39f436aeb91e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.500+0000 7f942fe00640 1 -- 172.21.15.112:0/3201831803 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434035040 con 0x5584026cb2d0 2024-04-25T18:48:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.501+0000 7f942fe00640 1 -- 172.21.15.112:0/3201831803 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94340042e0 con 0x5584026cb2d0 2024-04-25T18:48:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.501+0000 7f942fe00640 1 -- 172.21.15.112:0/3201831803 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434004770 con 0x5584026cb2d0 2024-04-25T18:48:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.501+0000 7f944d883740 1 -- 172.21.15.112:0/3201831803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026cb2d0 msgr2=0x5584026d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.501+0000 7f944d883740 1 --2- 172.21.15.112:0/3201831803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026cb2d0 0x5584026d3390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94340037d0 tx=0x7f943402a410 comp rx=0 tx=0).stop 2024-04-25T18:48:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.502+0000 7f944d883740 1 -- 172.21.15.112:0/3201831803 shutdown_connections 2024-04-25T18:48:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.502+0000 7f944d883740 1 --2- 172.21.15.112:0/3201831803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026d5b90 0x5584026d4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.502+0000 7f944d883740 1 --2- 172.21.15.112:0/3201831803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026cb2d0 0x5584026d3390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.502+0000 7f944d883740 1 --2- 172.21.15.112:0/3201831803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026e16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.502+0000 7f944d883740 1 -- 172.21.15.112:0/3201831803 >> 172.21.15.112:0/3201831803 conn(0x5584026c6ec0 msgr2=0x5584026d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.502+0000 7f944d883740 1 -- 172.21.15.112:0/3201831803 shutdown_connections 2024-04-25T18:48:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.502+0000 7f944d883740 1 -- 172.21.15.112:0/3201831803 wait complete. 2024-04-25T18:48:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.502+0000 7f944d883740 1 -- start start 2024-04-25T18:48:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.503+0000 7f944d883740 1 Processor -- start 2024-04-25T18:48:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.503+0000 7f944d883740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026dddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.503+0000 7f944d883740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 0x5584026db1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.503+0000 7f944d883740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026d5b90 0x5584026db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.503+0000 7f944d883740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94441a9f10 con 0x5584026cb2d0 2024-04-25T18:48:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.503+0000 7f944d883740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94441ab380 con 0x5584026d5b90 2024-04-25T18:48:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.503+0000 7f944d883740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94441aa5e0 con 0x5584026ca9b0 2024-04-25T18:48:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.503+0000 7f944d883740 10 client.0 _ll_register_callbacks cb 0x7f9434042da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.504+0000 7f9443400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026dddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.504+0000 7f9443400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026d5b90 0x5584026db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.504+0000 7f9443400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026dddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:32818/0 (socket says 172.21.15.112:32818) 2024-04-25T18:48:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.504+0000 7f9443400640 1 -- 172.21.15.112:0/3358806018 learned_addr learned my addr 172.21.15.112:0/3358806018 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.504+0000 7f9443e00640 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 0x5584026db1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.504+0000 7f9443e00640 1 -- 172.21.15.112:0/3358806018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 msgr2=0x5584026dddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.504+0000 7f9443e00640 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026dddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.504+0000 7f9443e00640 1 -- 172.21.15.112:0/3358806018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026d5b90 msgr2=0x5584026db6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.504+0000 7f9443e00640 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026d5b90 0x5584026db6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.504+0000 7f9443e00640 1 -- 172.21.15.112:0/3358806018 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584026dbd30 con 0x5584026cb2d0 2024-04-25T18:48:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.505+0000 7f9443e00640 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 0x5584026db1b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5584026d0c50 tx=0x7f9434004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516167dc5e4f3550 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.505+0000 7f942d600640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:48:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.505+0000 7f942d600640 1 -- 172.21.15.112:0/3358806018 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434035040 con 0x5584026cb2d0 2024-04-25T18:48:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.505+0000 7f942d600640 1 -- 172.21.15.112:0/3358806018 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f9434005670 con 0x5584026cb2d0 2024-04-25T18:48:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.505+0000 7f944d883740 1 -- 172.21.15.112:0/3358806018 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94340042e0 con 0x5584026cb2d0 2024-04-25T18:48:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.506+0000 7f942d600640 1 -- 172.21.15.112:0/3358806018 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9434004770 con 0x5584026cb2d0 2024-04-25T18:48:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.506+0000 7f944d883740 10 client.15192 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.506+0000 7f944d883740 1 -- 172.21.15.112:0/3358806018 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55840274c320 con 0x5584026cb2d0 2024-04-25T18:48:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.506+0000 7f942d600640 1 -- 172.21.15.112:0/3358806018 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94340042e0 con 0x5584026cb2d0 2024-04-25T18:48:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.506+0000 7f942d600640 1 -- 172.21.15.112:0/3358806018 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9434049040 con 0x5584026cb2d0 2024-04-25T18:48:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.507+0000 7f944d883740 10 client.15192 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.507+0000 7f944d883740 10 client.15192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:48:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.507+0000 7f944d883740 20 client.15192 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.507+0000 7f944d883740 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55840274c8c0 0x55840274ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.507+0000 7f944d883740 4 client.15192 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:48:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.507+0000 7f944d883740 1 -- 172.21.15.112:0/3358806018 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558402638190 con 0x55840274c8c0 2024-04-25T18:48:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.507+0000 7f9443400640 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55840274c8c0 0x55840274ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.508+0000 7f9443400640 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55840274c8c0 0x55840274ece0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417b69b614c5fa60 server_cookie=7c3d638953afe9ee in_seq=0 out_seq=0 2024-04-25T18:48:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.508+0000 7f942d600640 10 client.15192 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.517+0000 7f942d600640 1 -- 172.21.15.112:0/3358806018 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558402638190 con 0x55840274c8c0 2024-04-25T18:48:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.517+0000 7f942d600640 10 client.15192 handle_command_reply: tid=0 2024-04-25T18:48:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.517+0000 7f944d883740 1 client.15192 shutdown 2024-04-25T18:48:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.519+0000 7f944d883740 1 -- 172.21.15.112:0/3358806018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 msgr2=0x5584026db1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.519+0000 7f944d883740 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 0x5584026db1b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5584026d0c50 tx=0x7f9434004490 comp rx=0 tx=0).stop 2024-04-25T18:48:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.519+0000 7f944d883740 1 -- 172.21.15.112:0/3358806018 shutdown_connections 2024-04-25T18:48:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.519+0000 7f944d883740 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584026d5b90 0x5584026db6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.519+0000 7f944d883740 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55840274c8c0 0x55840274ece0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.519+0000 7f942d600640 0 client.15192 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.519+0000 7f944d883740 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584026cb2d0 0x5584026db1b0 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5584026d0c50 tx=0x7f9434004490 comp rx=0 tx=0).stop 2024-04-25T18:48:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.519+0000 7f944d883740 1 --2- 172.21.15.112:0/3358806018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5584026ca9b0 0x5584026dddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.519+0000 7f944d883740 1 -- 172.21.15.112:0/3358806018 >> 172.21.15.112:0/3358806018 conn(0x5584026c6ec0 msgr2=0x5584026d4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.521+0000 7f944d883740 1 -- 172.21.15.112:0/3358806018 shutdown_connections 2024-04-25T18:48:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.521+0000 7f944d883740 1 -- 172.21.15.112:0/3358806018 wait complete. 2024-04-25T18:48:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.521+0000 7f944d883740 20 client.15192 trim_cache size 0 max 16384 2024-04-25T18:48:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:48:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:48:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:48:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:48:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.522+0000 7f9449400640 1 -- 172.21.15.112:0/3760265471 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f94280a4090 msgr2=0x7f94280a6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.522+0000 7f9449400640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f94280a4090 0x7f94280a6550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f943c0ce4b0 tx=0x7f943c005b10 comp rx=0 tx=0).stop 2024-04-25T18:48:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.522+0000 7f9449400640 1 -- 172.21.15.112:0/3760265471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944410e6c0 msgr2=0x7f944419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.522+0000 7f9449400640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944410e6c0 0x7f944419a5f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f943c00eb10 tx=0x7f943c00c2d0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.523+0000 7f9449400640 1 -- 172.21.15.112:0/3760265471 shutdown_connections 2024-04-25T18:48:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.523+0000 7f9449400640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f94280a4090 0x7f94280a6550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.523+0000 7f9449400640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f944410e6c0 0x7f944419a5f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.523+0000 7f9449400640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94441014c0 0x7f94441a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.523+0000 7f9449400640 1 --2- 172.21.15.112:0/3760265471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9444100b10 0x7f94441a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.523+0000 7f9449400640 1 -- 172.21.15.112:0/3760265471 >> 172.21.15.112:0/3760265471 conn(0x7f94440fc770 msgr2=0x7f94440feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.523+0000 7f9449400640 1 -- 172.21.15.112:0/3760265471 shutdown_connections 2024-04-25T18:48:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:34.523+0000 7f9449400640 1 -- 172.21.15.112:0/3760265471 wait complete. 2024-04-25T18:48:34.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:34 smithi171 ceph-mon[33266]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T18:48:34.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:34 smithi161 ceph-mon[32609]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T18:48:35.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.360+0000 7f6d76000700 1 -- 172.21.15.112:0/1073653997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d701079f0 msgr2=0x7f6d701054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:35.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.360+0000 7f6d76000700 1 --2- 172.21.15.112:0/1073653997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d701079f0 0x7f6d701054a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009990 tx=0x7f6d60009ca0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.360+0000 7f6d76000700 1 -- 172.21.15.112:0/1073653997 shutdown_connections 2024-04-25T18:48:35.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.360+0000 7f6d76000700 1 --2- 172.21.15.112:0/1073653997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d7006a740 0x7f6d7006abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.360+0000 7f6d76000700 1 --2- 172.21.15.112:0/1073653997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d701079f0 0x7f6d701054a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.360+0000 7f6d76000700 1 --2- 172.21.15.112:0/1073653997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d701059e0 0x7f6d70105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.360+0000 7f6d76000700 1 -- 172.21.15.112:0/1073653997 >> 172.21.15.112:0/1073653997 conn(0x7f6d700788e0 msgr2=0x7f6d70078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:35.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.360+0000 7f6d76000700 1 -- 172.21.15.112:0/1073653997 shutdown_connections 2024-04-25T18:48:35.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.360+0000 7f6d76000700 1 -- 172.21.15.112:0/1073653997 wait complete. 2024-04-25T18:48:35.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.361+0000 7f6d76000700 1 Processor -- start 2024-04-25T18:48:35.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.361+0000 7f6d76000700 1 -- start start 2024-04-25T18:48:35.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.362+0000 7f6d76000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d7006a740 0x7f6d700740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.362+0000 7f6d76000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d701059e0 0x7f6d70074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.362+0000 7f6d76000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d701079f0 0x7f6d7006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.362+0000 7f6d76000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d70119290 con 0x7f6d701059e0 2024-04-25T18:48:35.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.362+0000 7f6d76000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d70119110 con 0x7f6d701079f0 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.362+0000 7f6d76000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6d70119410 con 0x7f6d7006a740 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.362+0000 7f6d6f400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d7006a740 0x7f6d700740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.362+0000 7f6d6f400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d7006a740 0x7f6d700740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:32838/0 (socket says 172.21.15.112:32838) 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.362+0000 7f6d6f400700 1 -- 172.21.15.112:0/2281388428 learned_addr learned my addr 172.21.15.112:0/2281388428 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.363+0000 7f6d6ea00700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d701059e0 0x7f6d70074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.363+0000 7f6d6f400700 1 -- 172.21.15.112:0/2281388428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d701079f0 msgr2=0x7f6d7006d2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.363+0000 7f6d6f400700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d701079f0 0x7f6d7006d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.363+0000 7f6d6f400700 1 -- 172.21.15.112:0/2281388428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d701059e0 msgr2=0x7f6d70074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.363+0000 7f6d6f400700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d701059e0 0x7f6d70074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.363+0000 7f6d6f400700 1 -- 172.21.15.112:0/2281388428 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7006dbb0 con 0x7f6d7006a740 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.363+0000 7f6d6ea00700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d701059e0 0x7f6d70074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.363+0000 7f6d6f400700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d7006a740 0x7f6d700740e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d60005910 tx=0x7f6d60003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.363+0000 7f6d67e00700 1 -- 172.21.15.112:0/2281388428 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60027070 con 0x7f6d7006a740 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.364+0000 7f6d67e00700 1 -- 172.21.15.112:0/2281388428 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d60004140 con 0x7f6d7006a740 2024-04-25T18:48:35.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.364+0000 7f6d67e00700 1 -- 172.21.15.112:0/2281388428 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6000f460 con 0x7f6d7006a740 2024-04-25T18:48:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.365+0000 7f6d76000700 1 -- 172.21.15.112:0/2281388428 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7006de40 con 0x7f6d7006a740 2024-04-25T18:48:35.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.365+0000 7f6d76000700 1 -- 172.21.15.112:0/2281388428 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d70071de0 con 0x7f6d7006a740 2024-04-25T18:48:35.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.369+0000 7f6d67e00700 1 -- 172.21.15.112:0/2281388428 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f6d60022600 con 0x7f6d7006a740 2024-04-25T18:48:35.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.369+0000 7f6d80a53b80 1 Processor -- start 2024-04-25T18:48:35.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.370+0000 7f6d80a53b80 1 -- start start 2024-04-25T18:48:35.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.370+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 0x55ac548121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.370+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 0x55ac5481d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.370+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac5481d560 0x55ac5481f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.370+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ac54765020 con 0x55ac5481d560 2024-04-25T18:48:35.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.370+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ac5470b8c0 con 0x55ac54811dd0 2024-04-25T18:48:35.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.370+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ac54760fe0 con 0x55ac548126f0 2024-04-25T18:48:35.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.371+0000 7f6d6f400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 0x55ac5481d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.371+0000 7f6d6f400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 0x55ac5481d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:32852/0 (socket says 172.21.15.112:32852) 2024-04-25T18:48:35.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.371+0000 7f6d6fe00700 1 --2- 172.21.15.112:0/1643550175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 0x55ac548121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.371+0000 7f6d67e00700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d540a4080 0x7f6d540a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.371+0000 7f6d6f400700 1 -- 172.21.15.112:0/1643550175 learned_addr learned my addr 172.21.15.112:0/1643550175 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:35.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.372+0000 7f6d67e00700 1 -- 172.21.15.112:0/2281388428 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f6d600d19c0 con 0x7f6d7006a740 2024-04-25T18:48:35.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.372+0000 7f6d6fe00700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d540a4080 0x7f6d540a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.372+0000 7f6d6f400700 1 -- 172.21.15.112:0/1643550175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 msgr2=0x55ac548121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:35.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.372+0000 7f6d6f400700 1 --2- 172.21.15.112:0/1643550175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 0x55ac548121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.372+0000 7f6d6f400700 1 -- 172.21.15.112:0/1643550175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac5481d560 msgr2=0x55ac5481f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:35.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.372+0000 7f6d6ea00700 1 --2- 172.21.15.112:0/1643550175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac5481d560 0x55ac5481f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.373+0000 7f6d6f400700 1 --2- 172.21.15.112:0/1643550175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac5481d560 0x55ac5481f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.373+0000 7f6d6f400700 1 -- 172.21.15.112:0/1643550175 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac54701810 con 0x55ac548126f0 2024-04-25T18:48:35.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.373+0000 7f6d6ea00700 1 --2- 172.21.15.112:0/1643550175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac5481d560 0x55ac5481f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:48:35.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.373+0000 7f6d6fe00700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d540a4080 0x7f6d540a6540 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6d68010ac0 tx=0x7f6d68007e90 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:35.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.373+0000 7f6d6f400700 1 --2- 172.21.15.112:0/1643550175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 0x55ac5481d020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d600cefc0 tx=0x7f6d600229a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c990c539d8c8ce5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:35.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.374+0000 7f6d64c00700 1 -- 172.21.15.112:0/1643550175 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6009d2f0 con 0x55ac548126f0 2024-04-25T18:48:35.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.374+0000 7f6d64c00700 1 -- 172.21.15.112:0/1643550175 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d600a40b0 con 0x55ac548126f0 2024-04-25T18:48:35.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.374+0000 7f6d64c00700 1 -- 172.21.15.112:0/1643550175 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6009d110 con 0x55ac548126f0 2024-04-25T18:48:35.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.375+0000 7f6d80a53b80 1 -- 172.21.15.112:0/1643550175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 msgr2=0x55ac5481d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:35.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.375+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/1643550175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 0x55ac5481d020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d600cefc0 tx=0x7f6d600229a0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.375+0000 7f6d80a53b80 1 -- 172.21.15.112:0/1643550175 shutdown_connections 2024-04-25T18:48:35.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.375+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/1643550175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac5481d560 0x55ac5481f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.375+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/1643550175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 0x55ac548121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.375+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/1643550175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 0x55ac5481d020 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.375+0000 7f6d80a53b80 1 -- 172.21.15.112:0/1643550175 >> 172.21.15.112:0/1643550175 conn(0x55ac547410e0 msgr2=0x55ac5480e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:35.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.375+0000 7f6d80a53b80 1 -- 172.21.15.112:0/1643550175 shutdown_connections 2024-04-25T18:48:35.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.375+0000 7f6d80a53b80 1 -- 172.21.15.112:0/1643550175 wait complete. 2024-04-25T18:48:35.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.376+0000 7f6d80a53b80 1 -- start start 2024-04-25T18:48:35.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.376+0000 7f6d80a53b80 1 Processor -- start 2024-04-25T18:48:35.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.377+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac54811dd0 0x55ac5481ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.377+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac548126f0 0x55ac54819ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.377+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 0x55ac5481a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.377+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ac5481fe90 con 0x55ac548126f0 2024-04-25T18:48:35.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.377+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ac5471e9f0 con 0x55ac5481d560 2024-04-25T18:48:35.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.377+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ac54820010 con 0x55ac54811dd0 2024-04-25T18:48:35.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.377+0000 7f6d80a53b80 10 client.0 _ll_register_callbacks cb 0x55ac546fd8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:35.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.377+0000 7f6d6fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 0x55ac5481a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.377+0000 7f6d6fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 0x55ac5481a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:38286/0 (socket says 172.21.15.112:38286) 2024-04-25T18:48:35.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.377+0000 7f6d6fe00700 1 -- 172.21.15.112:0/4179225959 learned_addr learned my addr 172.21.15.112:0/4179225959 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:35.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.378+0000 7f6d6f400700 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac54811dd0 0x55ac5481ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.378+0000 7f6d6fe00700 1 -- 172.21.15.112:0/4179225959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac54811dd0 msgr2=0x55ac5481ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:35.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.378+0000 7f6d6fe00700 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac54811dd0 0x55ac5481ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.378+0000 7f6d6fe00700 1 -- 172.21.15.112:0/4179225959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac548126f0 msgr2=0x55ac54819ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:35.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.378+0000 7f6d6fe00700 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac548126f0 0x55ac54819ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.378+0000 7f6d6fe00700 1 -- 172.21.15.112:0/4179225959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac546b2a60 con 0x55ac5481d560 2024-04-25T18:48:35.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.378+0000 7f6d6fe00700 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 0x55ac5481a530 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d68018690 tx=0x7f6d68018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65514c9763b318d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.378+0000 7f6d59600700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:48:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.379+0000 7f6d59600700 1 -- 172.21.15.112:0/4179225959 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68010040 con 0x55ac5481d560 2024-04-25T18:48:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.379+0000 7f6d59600700 1 -- 172.21.15.112:0/4179225959 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f6d68003c10 con 0x55ac5481d560 2024-04-25T18:48:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.379+0000 7f6d80a53b80 1 -- 172.21.15.112:0/4179225959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d600a40b0 con 0x55ac5481d560 2024-04-25T18:48:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.379+0000 7f6d59600700 1 -- 172.21.15.112:0/4179225959 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6d68010280 con 0x55ac5481d560 2024-04-25T18:48:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.379+0000 7f6d80a53b80 10 client.25328 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.379+0000 7f6d80a53b80 1 -- 172.21.15.112:0/4179225959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ac546fbc80 con 0x55ac5481d560 2024-04-25T18:48:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.379+0000 7f6d59600700 1 -- 172.21.15.112:0/4179225959 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d68011040 con 0x55ac5481d560 2024-04-25T18:48:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.380+0000 7f6d59600700 1 -- 172.21.15.112:0/4179225959 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6d680044e0 con 0x55ac5481d560 2024-04-25T18:48:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.380+0000 7f6d80a53b80 10 client.25328 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.380+0000 7f6d80a53b80 10 client.25328 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-25T18:48:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.380+0000 7f6d80a53b80 20 client.25328 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:35.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.380+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55ac5482ca80 0x55ac5482eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.380+0000 7f6d80a53b80 4 client.25328 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:48:35.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.380+0000 7f6d80a53b80 1 -- 172.21.15.112:0/4179225959 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6d600a40b0 con 0x55ac5482ca80 2024-04-25T18:48:35.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.380+0000 7f6d6f400700 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55ac5482ca80 0x55ac5482eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.381+0000 7f6d59600700 10 client.25328 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:35.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.381+0000 7f6d6f400700 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55ac5482ca80 0x55ac5482eee0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e838810d8a62f6bd server_cookie=95dfc3da53e9d1f0 in_seq=0 out_seq=0 2024-04-25T18:48:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.382+0000 7f6d59600700 1 -- 172.21.15.112:0/4179225959 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f6d600a40b0 con 0x55ac5482ca80 2024-04-25T18:48:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.382+0000 7f6d59600700 10 client.25328 handle_command_reply: tid=0 2024-04-25T18:48:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.382+0000 7f6d80a53b80 1 client.25328 shutdown 2024-04-25T18:48:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.382+0000 7f6d80a53b80 1 -- 172.21.15.112:0/4179225959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 msgr2=0x55ac5481a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.382+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 0x55ac5481a530 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d68018690 tx=0x7f6d68018000 comp rx=0 tx=0).stop 2024-04-25T18:48:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.383+0000 7f6d80a53b80 1 -- 172.21.15.112:0/4179225959 shutdown_connections 2024-04-25T18:48:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.383+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55ac5482ca80 0x55ac5482eee0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.383+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 0x55ac5481a530 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.383+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac54811dd0 0x55ac5481ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.383+0000 7f6d59600700 0 client.25328 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:35.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.383+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/4179225959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac548126f0 0x55ac54819ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.383+0000 7f6d80a53b80 1 -- 172.21.15.112:0/4179225959 >> 172.21.15.112:0/4179225959 conn(0x55ac547410e0 msgr2=0x55ac5480ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:35.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.383+0000 7f6d80a53b80 1 -- 172.21.15.112:0/4179225959 shutdown_connections 2024-04-25T18:48:35.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.383+0000 7f6d80a53b80 1 -- 172.21.15.112:0/4179225959 wait complete. 2024-04-25T18:48:35.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.383+0000 7f6d80a53b80 20 client.25328 trim_cache size 0 max 16384 2024-04-25T18:48:35.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.404+0000 7f6d80a53b80 1 Processor -- start 2024-04-25T18:48:35.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.404+0000 7f6d80a53b80 1 -- start start 2024-04-25T18:48:35.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.404+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 0x55ac54828cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.404+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac548126f0 0x55ac548347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.404+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac5481d560 0x55ac54834cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.404+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701be7e0 con 0x55ac548126f0 2024-04-25T18:48:35.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.404+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701bc070 con 0x55ac54811dd0 2024-04-25T18:48:35.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.404+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701b9ec0 con 0x55ac5481d560 2024-04-25T18:48:35.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.405+0000 7f6d6f400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 0x55ac54828cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.405+0000 7f6d6fe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac5481d560 0x55ac54834cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.405+0000 7f6d6f400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 0x55ac54828cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:38302/0 (socket says 172.21.15.112:38302) 2024-04-25T18:48:35.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.405+0000 7f6d6f400700 1 -- 172.21.15.112:0/3663918558 learned_addr learned my addr 172.21.15.112:0/3663918558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:35.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.405+0000 7f6d6fe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac5481d560 0x55ac54834cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:32880/0 (socket says 172.21.15.112:32880) 2024-04-25T18:48:35.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.405+0000 7f6d6f400700 1 -- 172.21.15.112:0/3663918558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac5481d560 msgr2=0x55ac54834cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:35.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.405+0000 7f6d6f400700 1 --2- 172.21.15.112:0/3663918558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac5481d560 0x55ac54834cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.405+0000 7f6d6f400700 1 -- 172.21.15.112:0/3663918558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac548126f0 msgr2=0x55ac548347b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T18:48:35.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.406+0000 7f6d6ea00700 1 --2- 172.21.15.112:0/3663918558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac548126f0 0x55ac548347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.406+0000 7f6d6f400700 1 --2- 172.21.15.112:0/3663918558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac548126f0 0x55ac548347b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.406+0000 7f6d6f400700 1 -- 172.21.15.112:0/3663918558 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac5482f560 con 0x55ac54811dd0 2024-04-25T18:48:35.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.406+0000 7f6d6ea00700 1 --2- 172.21.15.112:0/3663918558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac548126f0 0x55ac548347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:48:35.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.406+0000 7f6d6f400700 1 --2- 172.21.15.112:0/3663918558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 0x55ac54828cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d600cf010 tx=0x7f6d600229a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5759d0a2b5bb1851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:35.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.407+0000 7f6d5be00700 1 -- 172.21.15.112:0/3663918558 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600a40b0 con 0x55ac54811dd0 2024-04-25T18:48:35.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.407+0000 7f6d5be00700 1 -- 172.21.15.112:0/3663918558 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d6009deb0 con 0x55ac54811dd0 2024-04-25T18:48:35.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.407+0000 7f6d5be00700 1 -- 172.21.15.112:0/3663918558 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600a1b60 con 0x55ac54811dd0 2024-04-25T18:48:35.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.407+0000 7f6d80a53b80 1 -- 172.21.15.112:0/3663918558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 msgr2=0x55ac54828cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:35.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.407+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/3663918558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 0x55ac54828cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d600cf010 tx=0x7f6d600229a0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.408+0000 7f6d80a53b80 1 -- 172.21.15.112:0/3663918558 shutdown_connections 2024-04-25T18:48:35.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.408+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/3663918558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac5481d560 0x55ac54834cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.408+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/3663918558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac54811dd0 0x55ac54828cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.408+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/3663918558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac548126f0 0x55ac548347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.408+0000 7f6d80a53b80 1 -- 172.21.15.112:0/3663918558 >> 172.21.15.112:0/3663918558 conn(0x55ac547410e0 msgr2=0x55ac54814700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:35.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.408+0000 7f6d80a53b80 1 -- 172.21.15.112:0/3663918558 shutdown_connections 2024-04-25T18:48:35.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.408+0000 7f6d80a53b80 1 -- 172.21.15.112:0/3663918558 wait complete. 2024-04-25T18:48:35.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.409+0000 7f6d80a53b80 1 -- start start 2024-04-25T18:48:35.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.409+0000 7f6d80a53b80 1 Processor -- start 2024-04-25T18:48:35.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.409+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac54811dd0 0x55ac54822140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 0x55ac54822680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d80a53b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 0x55ac54822bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701ba530 con 0x55ac54811dd0 2024-04-25T18:48:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701bb9a0 con 0x55ac5481d560 2024-04-25T18:48:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d80a53b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701bac00 con 0x55ac548126f0 2024-04-25T18:48:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d80a53b80 10 client.0 _ll_register_callbacks cb 0x55ac546fd8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d6f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac54811dd0 0x55ac54822140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d6f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac54811dd0 0x55ac54822140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54394/0 (socket says 172.21.15.112:54394) 2024-04-25T18:48:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d6fe00700 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 0x55ac54822bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d6f400700 1 -- 172.21.15.112:0/434420898 learned_addr learned my addr 172.21.15.112:0/434420898 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d6ea00700 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 0x55ac54822680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d6fe00700 1 -- 172.21.15.112:0/434420898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 msgr2=0x55ac54822680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:35.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d6fe00700 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 0x55ac54822680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.410+0000 7f6d6fe00700 1 -- 172.21.15.112:0/434420898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac54811dd0 msgr2=0x55ac54822140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:35.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.411+0000 7f6d6fe00700 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac54811dd0 0x55ac54822140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.411+0000 7f6d6fe00700 1 -- 172.21.15.112:0/434420898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac54893030 con 0x55ac5481d560 2024-04-25T18:48:35.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.411+0000 7f6d6f400700 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac54811dd0 0x55ac54822140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:35.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.411+0000 7f6d6fe00700 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 0x55ac54822bc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55ac54835230 tx=0x7f6d68007c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ca1387aa6d113 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:35.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.411+0000 7f6d59600700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:48:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.412+0000 7f6d59600700 1 -- 172.21.15.112:0/434420898 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68037060 con 0x55ac5481d560 2024-04-25T18:48:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.412+0000 7f6d59600700 1 -- 172.21.15.112:0/434420898 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f6d68004b40 con 0x55ac5481d560 2024-04-25T18:48:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.412+0000 7f6d80a53b80 1 -- 172.21.15.112:0/434420898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6009deb0 con 0x55ac5481d560 2024-04-25T18:48:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.412+0000 7f6d59600700 1 -- 172.21.15.112:0/434420898 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6d68007970 con 0x55ac5481d560 2024-04-25T18:48:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.412+0000 7f6d80a53b80 10 client.25334 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.412+0000 7f6d80a53b80 1 -- 172.21.15.112:0/434420898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ac54894140 con 0x55ac5481d560 2024-04-25T18:48:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.412+0000 7f6d59600700 1 -- 172.21.15.112:0/434420898 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d6009deb0 con 0x55ac5481d560 2024-04-25T18:48:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.413+0000 7f6d59600700 1 -- 172.21.15.112:0/434420898 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6d68031070 con 0x55ac5481d560 2024-04-25T18:48:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.413+0000 7f6d80a53b80 10 client.25334 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.413+0000 7f6d80a53b80 10 client.25334 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-25T18:48:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.413+0000 7f6d80a53b80 20 client.25334 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.413+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55ac548948c0 0x55ac54896ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.413+0000 7f6d80a53b80 4 client.25334 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.8, "await_for": 42.0}] 2024-04-25T18:48:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.413+0000 7f6d80a53b80 1 -- 172.21.15.112:0/434420898 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.8, "await_for": 42.0}) -- 0x55ac54772810 con 0x55ac548948c0 2024-04-25T18:48:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.413+0000 7f6d6f400700 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55ac548948c0 0x55ac54896ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:35.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.414+0000 7f6d6f400700 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55ac548948c0 0x55ac54896ce0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd5d60e0259cbbf server_cookie=95fc3f4cef5118a6 in_seq=0 out_seq=0 2024-04-25T18:48:35.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.414+0000 7f6d59600700 10 client.25334 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:35.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.425+0000 7f6d59600700 1 -- 172.21.15.112:0/434420898 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55ac54772810 con 0x55ac548948c0 2024-04-25T18:48:35.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.425+0000 7f6d59600700 10 client.25334 handle_command_reply: tid=0 2024-04-25T18:48:35.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.425+0000 7f6d80a53b80 1 client.25334 shutdown 2024-04-25T18:48:35.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.425+0000 7f6d80a53b80 1 -- 172.21.15.112:0/434420898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 msgr2=0x55ac54822bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:35.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.425+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 0x55ac54822bc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55ac54835230 tx=0x7f6d68007c00 comp rx=0 tx=0).stop 2024-04-25T18:48:35.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.425+0000 7f6d80a53b80 1 -- 172.21.15.112:0/434420898 shutdown_connections 2024-04-25T18:48:35.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.425+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55ac548948c0 0x55ac54896ce0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.425+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac5481d560 0x55ac54822bc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.425+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac54811dd0 0x55ac54822140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.425+0000 7f6d80a53b80 1 --2- 172.21.15.112:0/434420898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ac548126f0 0x55ac54822680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.425+0000 7f6d80a53b80 1 -- 172.21.15.112:0/434420898 >> 172.21.15.112:0/434420898 conn(0x55ac547410e0 msgr2=0x55ac5481b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:35.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.426+0000 7f6d59600700 0 client.25334 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:35.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.426+0000 7f6d80a53b80 1 -- 172.21.15.112:0/434420898 shutdown_connections 2024-04-25T18:48:35.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.426+0000 7f6d80a53b80 1 -- 172.21.15.112:0/434420898 wait complete. 2024-04-25T18:48:35.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.426+0000 7f6d80a53b80 20 client.25334 trim_cache size 0 max 16384 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 26, 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout: "leader": 24512, 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout: "set_version": 2, 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout: "sets": { 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout: "b8363855": { 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout: "version": 2, 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout: "age_ref": 0.0, 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout: "state": { 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "QUIESCED", 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 0.0 2024-04-25T18:48:35.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: "timeout": 60.0, 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: "expiration": 80.8, 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: "members": { 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: "file:/": { 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: "excluded": false, 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: "state": { 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "QUIESCED", 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 0.0 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:48:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:48:35.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.427+0000 7f6d76000700 1 -- 172.21.15.112:0/2281388428 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d540a4080 msgr2=0x7f6d540a6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:35.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.427+0000 7f6d76000700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d540a4080 0x7f6d540a6540 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6d68010ac0 tx=0x7f6d68007e90 comp rx=0 tx=0).stop 2024-04-25T18:48:35.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.427+0000 7f6d76000700 1 -- 172.21.15.112:0/2281388428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d7006a740 msgr2=0x7f6d700740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:35.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.427+0000 7f6d76000700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d7006a740 0x7f6d700740e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d60005910 tx=0x7f6d60003730 comp rx=0 tx=0).stop 2024-04-25T18:48:35.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.427+0000 7f6d76000700 1 -- 172.21.15.112:0/2281388428 shutdown_connections 2024-04-25T18:48:35.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.427+0000 7f6d76000700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d540a4080 0x7f6d540a6540 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.427+0000 7f6d76000700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d701079f0 0x7f6d7006d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.427+0000 7f6d76000700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d7006a740 0x7f6d700740e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.427+0000 7f6d76000700 1 --2- 172.21.15.112:0/2281388428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d701059e0 0x7f6d70074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:35.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.427+0000 7f6d76000700 1 -- 172.21.15.112:0/2281388428 >> 172.21.15.112:0/2281388428 conn(0x7f6d700788e0 msgr2=0x7f6d700ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:35.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.428+0000 7f6d76000700 1 -- 172.21.15.112:0/2281388428 shutdown_connections 2024-04-25T18:48:35.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:48:35.428+0000 7f6d76000700 1 -- 172.21.15.112:0/2281388428 wait complete. 2024-04-25T18:48:35.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:35 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:35.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:35 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:35.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:35 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:48:35.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:35 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:35.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:35 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:35.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:35 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:48:35.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:35 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:35.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:35 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:35.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:35 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:48:36.407 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b8363855', quiesce took 8.0 seconds. Will release after: 20.8 2024-04-25T18:48:36.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:36 smithi171 ceph-mon[33266]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T18:48:36.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:36 smithi161 ceph-mon[32609]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T18:48:36.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:36 smithi112 ceph-mon[27864]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T18:48:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:38 smithi171 ceph-mon[33266]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-25T18:48:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:38 smithi161 ceph-mon[32609]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-25T18:48:38.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:38 smithi112 ceph-mon[27864]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-25T18:48:40.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:40 smithi171 ceph-mon[33266]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T18:48:40.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:40 smithi161 ceph-mon[32609]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T18:48:40.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:40 smithi112 ceph-mon[27864]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T18:48:42.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:42 smithi171 ceph-mon[33266]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T18:48:42.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:42 smithi161 ceph-mon[32609]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T18:48:42.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:42 smithi112 ceph-mon[27864]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T18:48:44.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:44 smithi171 ceph-mon[33266]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-25T18:48:44.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:44 smithi161 ceph-mon[32609]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-25T18:48:44.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:44 smithi112 ceph-mon[27864]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-25T18:48:46.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:46 smithi171 ceph-mon[33266]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T18:48:46.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:46 smithi161 ceph-mon[32609]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T18:48:46.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:46 smithi112 ceph-mon[27864]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T18:48:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.658+0000 7fefbb200640 1 -- 172.21.15.112:0/84370081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefbc103890 msgr2=0x7fefbc10bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.658+0000 7fefbb200640 1 --2- 172.21.15.112:0/84370081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefbc103890 0x7fefbc10bd70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009900 tx=0x7fefa402a480 comp rx=0 tx=0).stop 2024-04-25T18:48:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.658+0000 7fefbb200640 1 -- 172.21.15.112:0/84370081 shutdown_connections 2024-04-25T18:48:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.658+0000 7fefbb200640 1 --2- 172.21.15.112:0/84370081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefbc10c340 0x7fefbc10e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.658+0000 7fefbb200640 1 --2- 172.21.15.112:0/84370081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefbc103890 0x7fefbc10bd70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.658+0000 7fefbb200640 1 --2- 172.21.15.112:0/84370081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefbc102f70 0x7fefbc103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.658+0000 7fefbb200640 1 -- 172.21.15.112:0/84370081 >> 172.21.15.112:0/84370081 conn(0x7fefbc0fc770 msgr2=0x7fefbc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.658+0000 7fefbb200640 1 -- 172.21.15.112:0/84370081 shutdown_connections 2024-04-25T18:48:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.658+0000 7fefbb200640 1 -- 172.21.15.112:0/84370081 wait complete. 2024-04-25T18:48:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.659+0000 7fefbb200640 1 Processor -- start 2024-04-25T18:48:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.659+0000 7fefbb200640 1 -- start start 2024-04-25T18:48:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.659+0000 7fefbb200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefbc102f70 0x7fefbc100bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.659+0000 7fefbb200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefbc103890 0x7fefbc101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.659+0000 7fefbb200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefbc10c340 0x7fefbc101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefbb200640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc111ff0 con 0x7fefbc102f70 2024-04-25T18:48:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefbb200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc111e70 con 0x7fefbc10c340 2024-04-25T18:48:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefbb200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc112170 con 0x7fefbc103890 2024-04-25T18:48:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefbc103890 0x7fefbc101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefbc103890 0x7fefbc101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:57140/0 (socket says 172.21.15.112:57140) 2024-04-25T18:48:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefb3e00640 1 -- 172.21.15.112:0/3182615286 learned_addr learned my addr 172.21.15.112:0/3182615286 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefba800640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefbc10c340 0x7fefbc101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefb9e00640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefbc102f70 0x7fefbc100bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefb3e00640 1 -- 172.21.15.112:0/3182615286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefbc10c340 msgr2=0x7fefbc101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefb3e00640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefbc10c340 0x7fefbc101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefb3e00640 1 -- 172.21.15.112:0/3182615286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefbc102f70 msgr2=0x7fefbc100bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefb3e00640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefbc102f70 0x7fefbc100bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.660+0000 7fefb3e00640 1 -- 172.21.15.112:0/3182615286 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc1a6a90 con 0x7fefbc103890 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.661+0000 7fefb3e00640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefbc103890 0x7fefbc101100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005ab0 tx=0x7fefa4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.661+0000 7fefb2a00640 1 -- 172.21.15.112:0/3182615286 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa403f070 con 0x7fefbc103890 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.661+0000 7fefbb200640 1 -- 172.21.15.112:0/3182615286 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefbc1a6d20 con 0x7fefbc103890 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.661+0000 7fefb2a00640 1 -- 172.21.15.112:0/3182615286 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefa4031b60 con 0x7fefbc103890 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.661+0000 7fefb2a00640 1 -- 172.21.15.112:0/3182615286 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4035030 con 0x7fefbc103890 2024-04-25T18:48:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.661+0000 7fefbb200640 1 -- 172.21.15.112:0/3182615286 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefbc1a7200 con 0x7fefbc103890 2024-04-25T18:48:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.664+0000 7fefc3e19740 1 Processor -- start 2024-04-25T18:48:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.664+0000 7fefc3e19740 1 -- start start 2024-04-25T18:48:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439b9b0 0x564f3439bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f3439c2d0 0x564f343a6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f343a6b90 0x564f343a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefc3e19740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f34301900 con 0x564f343a6b90 2024-04-25T18:48:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefc3e19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564f342a7ef0 con 0x564f3439c2d0 2024-04-25T18:48:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefc3e19740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564f342fd8c0 con 0x564f3439b9b0 2024-04-25T18:48:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefba800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439b9b0 0x564f3439bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefba800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439b9b0 0x564f3439bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:57152/0 (socket says 172.21.15.112:57152) 2024-04-25T18:48:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefba800640 1 -- 172.21.15.112:0/4018936802 learned_addr learned my addr 172.21.15.112:0/4018936802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefb9e00640 1 --2- 172.21.15.112:0/4018936802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f3439c2d0 0x564f343a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefba800640 1 -- 172.21.15.112:0/4018936802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f3439c2d0 msgr2=0x564f343a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefba800640 1 --2- 172.21.15.112:0/4018936802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f3439c2d0 0x564f343a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefba800640 1 -- 172.21.15.112:0/4018936802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f343a6b90 msgr2=0x564f343a8f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T18:48:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefba800640 1 --2- 172.21.15.112:0/4018936802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f343a6b90 0x564f343a8f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.665+0000 7fefba800640 1 -- 172.21.15.112:0/4018936802 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f34014210 con 0x564f3439b9b0 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.666+0000 7fefba800640 1 --2- 172.21.15.112:0/4018936802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439b9b0 0x564f3439bd90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009f50 tx=0x7fefb40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f12bd36b23f4f6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.666+0000 7fefb2a00640 1 -- 172.21.15.112:0/3182615286 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fefa4042070 con 0x7fefbc103890 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.666+0000 7fefab400640 1 -- 172.21.15.112:0/4018936802 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4004d20 con 0x564f3439b9b0 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.666+0000 7fefab400640 1 -- 172.21.15.112:0/4018936802 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefb4004050 con 0x564f3439b9b0 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.666+0000 7fefab400640 1 -- 172.21.15.112:0/4018936802 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4005280 con 0x564f3439b9b0 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.667+0000 7fefc3e19740 1 -- 172.21.15.112:0/4018936802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439b9b0 msgr2=0x564f3439bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.667+0000 7fefc3e19740 1 --2- 172.21.15.112:0/4018936802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439b9b0 0x564f3439bd90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009f50 tx=0x7fefb40118d0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.667+0000 7fefc3e19740 1 -- 172.21.15.112:0/4018936802 shutdown_connections 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.667+0000 7fefc3e19740 1 --2- 172.21.15.112:0/4018936802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f343a6b90 0x564f343a8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.667+0000 7fefc3e19740 1 --2- 172.21.15.112:0/4018936802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f3439c2d0 0x564f343a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.667+0000 7fefc3e19740 1 --2- 172.21.15.112:0/4018936802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439b9b0 0x564f3439bd90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.667+0000 7fefc3e19740 1 -- 172.21.15.112:0/4018936802 >> 172.21.15.112:0/4018936802 conn(0x564f34397ec0 msgr2=0x564f343982a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.667+0000 7fefc3e19740 1 -- 172.21.15.112:0/4018936802 shutdown_connections 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.667+0000 7fefc3e19740 1 -- 172.21.15.112:0/4018936802 wait complete. 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.667+0000 7fefb2a00640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fefa00a3de0 0x7fefa00a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.668+0000 7fefc3e19740 1 -- start start 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.668+0000 7fefb2a00640 1 -- 172.21.15.112:0/3182615286 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fefa40f0dd0 con 0x7fefbc103890 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.668+0000 7fefba800640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fefa00a3de0 0x7fefa00a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.668+0000 7fefc3e19740 1 Processor -- start 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.668+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.668+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 0x564f343a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.668+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f343a6b90 0x564f343a4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.668+0000 7fefc3e19740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564f343a97c0 con 0x564f3439b9b0 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.668+0000 7fefc3e19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564f342bb310 con 0x564f343a6b90 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.668+0000 7fefc3e19740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564f343a9940 con 0x564f3439c2d0 2024-04-25T18:48:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.668+0000 7fefc3e19740 10 client.0 _ll_register_callbacks cb 0x564f34289cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefb9e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefb9e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343a3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35814/0 (socket says 172.21.15.112:35814) 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefb9e00640 1 -- 172.21.15.112:0/1814683638 learned_addr learned my addr 172.21.15.112:0/1814683638 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefb3e00640 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 0x564f343a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefba800640 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f343a6b90 0x564f343a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefba800640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fefa00a3de0 0x7fefa00a62a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fefb401b040 tx=0x7fefb4000f40 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefb9e00640 1 -- 172.21.15.112:0/1814683638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 msgr2=0x564f343a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefb9e00640 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 0x564f343a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefb9e00640 1 -- 172.21.15.112:0/1814683638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f343a6b90 msgr2=0x564f343a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefb9e00640 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f343a6b90 0x564f343a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefb9e00640 1 -- 172.21.15.112:0/1814683638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f340e0620 con 0x564f3439b9b0 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fefb9e00640 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343a3ac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fefac012710 tx=0x7fefac012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2660c7ba9e6055 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.669+0000 7fef9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:48:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.670+0000 7fef9fe00640 1 -- 172.21.15.112:0/1814683638 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac00f920 con 0x564f3439b9b0 2024-04-25T18:48:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.670+0000 7fef9fe00640 1 -- 172.21.15.112:0/1814683638 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fefac00fac0 con 0x564f3439b9b0 2024-04-25T18:48:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.670+0000 7fefc3e19740 1 -- 172.21.15.112:0/1814683638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefb4004050 con 0x564f3439b9b0 2024-04-25T18:48:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.670+0000 7fef9fe00640 1 -- 172.21.15.112:0/1814683638 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fefac00fcc0 con 0x564f3439b9b0 2024-04-25T18:48:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.670+0000 7fefc3e19740 10 client.15204 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.670+0000 7fefc3e19740 1 -- 172.21.15.112:0/1814683638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f343b5a00 con 0x564f3439b9b0 2024-04-25T18:48:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.670+0000 7fef9fe00640 1 -- 172.21.15.112:0/1814683638 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefb4004050 con 0x564f3439b9b0 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.670+0000 7fef9fe00640 1 -- 172.21.15.112:0/1814683638 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fefac01c420 con 0x564f3439b9b0 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.671+0000 7fefc3e19740 10 client.15204 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.671+0000 7fefc3e19740 10 client.15204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.671+0000 7fefc3e19740 20 client.15204 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.671+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564f343b6ec0 0x564f343b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.671+0000 7fefc3e19740 4 client.15204 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.671+0000 7fefc3e19740 1 -- 172.21.15.112:0/1814683638 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f343158b0 con 0x564f343b6ec0 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.671+0000 7fefb3e00640 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564f343b6ec0 0x564f343b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.671+0000 7fefb3e00640 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564f343b6ec0 0x564f343b92d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caebaacac0885eab server_cookie=813ba9e1632f8e24 in_seq=0 out_seq=0 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.671+0000 7fef9fe00640 10 client.15204 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.672+0000 7fef9fe00640 1 -- 172.21.15.112:0/1814683638 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f343158b0 con 0x564f343b6ec0 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.672+0000 7fef9fe00640 10 client.15204 handle_command_reply: tid=0 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.672+0000 7fefc3e19740 1 client.15204 shutdown 2024-04-25T18:48:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.672+0000 7fefc3e19740 1 -- 172.21.15.112:0/1814683638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 msgr2=0x564f343a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.672+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343a3ac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fefac012710 tx=0x7fefac012bd0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.673+0000 7fefc3e19740 1 -- 172.21.15.112:0/1814683638 shutdown_connections 2024-04-25T18:48:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.673+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f343a6b90 0x564f343a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.673+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 0x564f343a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.673+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564f343b6ec0 0x564f343b92d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.673+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1814683638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343a3ac0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.673+0000 7fefc3e19740 1 -- 172.21.15.112:0/1814683638 >> 172.21.15.112:0/1814683638 conn(0x564f34397ec0 msgr2=0x564f3439dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.673+0000 7fef9fe00640 0 client.15204 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.673+0000 7fefc3e19740 1 -- 172.21.15.112:0/1814683638 shutdown_connections 2024-04-25T18:48:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.673+0000 7fefc3e19740 1 -- 172.21.15.112:0/1814683638 wait complete. 2024-04-25T18:48:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.673+0000 7fefc3e19740 20 client.15204 trim_cache size 0 max 16384 2024-04-25T18:48:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.689+0000 7fefc3e19740 1 Processor -- start 2024-04-25T18:48:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.689+0000 7fefc3e19740 1 -- start start 2024-04-25T18:48:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.689+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.689+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 0x564f343bebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.689+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f343a6b90 0x564f343bf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.689+0000 7fefc3e19740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc1b2530 con 0x564f3439b9b0 2024-04-25T18:48:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.689+0000 7fefc3e19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc1afe00 con 0x564f343a6b90 2024-04-25T18:48:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.689+0000 7fefc3e19740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc1adc50 con 0x564f3439c2d0 2024-04-25T18:48:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefb9e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 0x564f343bebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefb9e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35822/0 (socket says 172.21.15.112:35822) 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefb9e00640 1 -- 172.21.15.112:0/1576006767 learned_addr learned my addr 172.21.15.112:0/1576006767 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefb3e00640 1 --2- 172.21.15.112:0/1576006767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 0x564f343bebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:57174/0 (socket says 172.21.15.112:57174) 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefba800640 1 --2- 172.21.15.112:0/1576006767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f343a6b90 0x564f343bf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefb3e00640 1 -- 172.21.15.112:0/1576006767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f343a6b90 msgr2=0x564f343bf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefb3e00640 1 --2- 172.21.15.112:0/1576006767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f343a6b90 0x564f343bf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefb3e00640 1 -- 172.21.15.112:0/1576006767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 msgr2=0x564f343b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefb3e00640 1 --2- 172.21.15.112:0/1576006767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefb3e00640 1 -- 172.21.15.112:0/1576006767 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f3439c100 con 0x564f3439c2d0 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefb3e00640 1 --2- 172.21.15.112:0/1576006767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 0x564f343bebe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fefa40ee2e0 tx=0x7fefa403b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e933b594dee17d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.690+0000 7fefaaa00640 1 -- 172.21.15.112:0/1576006767 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa40bde60 con 0x564f3439c2d0 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.691+0000 7fefaaa00640 1 -- 172.21.15.112:0/1576006767 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefa4031d10 con 0x564f3439c2d0 2024-04-25T18:48:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.691+0000 7fefaaa00640 1 -- 172.21.15.112:0/1576006767 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4049040 con 0x564f3439c2d0 2024-04-25T18:48:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.691+0000 7fefc3e19740 1 -- 172.21.15.112:0/1576006767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 msgr2=0x564f343bebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.691+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1576006767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 0x564f343bebe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fefa40ee2e0 tx=0x7fefa403b800 comp rx=0 tx=0).stop 2024-04-25T18:48:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.691+0000 7fefc3e19740 1 -- 172.21.15.112:0/1576006767 shutdown_connections 2024-04-25T18:48:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.691+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1576006767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f343a6b90 0x564f343bf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.691+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1576006767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f3439c2d0 0x564f343bebe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.691+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1576006767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.691+0000 7fefc3e19740 1 -- 172.21.15.112:0/1576006767 >> 172.21.15.112:0/1576006767 conn(0x564f34397ec0 msgr2=0x564f3439edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.691+0000 7fefc3e19740 1 -- 172.21.15.112:0/1576006767 shutdown_connections 2024-04-25T18:48:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.691+0000 7fefc3e19740 1 -- 172.21.15.112:0/1576006767 wait complete. 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.692+0000 7fefc3e19740 1 -- start start 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.692+0000 7fefc3e19740 1 Processor -- start 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.692+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343aef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.692+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f3439c2d0 0x564f343ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.692+0000 7fefc3e19740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f343a6b90 0x564f343ac790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.692+0000 7fefc3e19740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc1ae2c0 con 0x564f3439b9b0 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.692+0000 7fefc3e19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc1af730 con 0x564f3439c2d0 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.692+0000 7fefc3e19740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc1ae990 con 0x564f343a6b90 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefc3e19740 10 client.0 _ll_register_callbacks cb 0x7fefac009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefb9e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefb9e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343aef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35832/0 (socket says 172.21.15.112:35832) 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefb9e00640 1 -- 172.21.15.112:0/1153273140 learned_addr learned my addr 172.21.15.112:0/1153273140 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefb3e00640 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f3439c2d0 0x564f343ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefba800640 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f343a6b90 0x564f343ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefb9e00640 1 -- 172.21.15.112:0/1153273140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f343a6b90 msgr2=0x564f343ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefb9e00640 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f343a6b90 0x564f343ac790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefb9e00640 1 -- 172.21.15.112:0/1153273140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f3439c2d0 msgr2=0x564f343ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefb9e00640 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f3439c2d0 0x564f343ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefb9e00640 1 -- 172.21.15.112:0/1153273140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f343acfc0 con 0x564f3439b9b0 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefba800640 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f343a6b90 0x564f343ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fefb9e00640 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343aef10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fefac000c00 tx=0x7fefac01c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d744b76ce3bd39cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.693+0000 7fef9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.694+0000 7fef9fe00640 1 -- 172.21.15.112:0/1153273140 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac00fe60 con 0x564f3439b9b0 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.694+0000 7fef9fe00640 1 -- 172.21.15.112:0/1153273140 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fefac01cd10 con 0x564f3439b9b0 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.694+0000 7fefc3e19740 1 -- 172.21.15.112:0/1153273140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa4031d10 con 0x564f3439b9b0 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.694+0000 7fef9fe00640 1 -- 172.21.15.112:0/1153273140 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fefac034050 con 0x564f3439b9b0 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.694+0000 7fefc3e19740 10 client.15210 fetch_fsmap learned FSMap version 26 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.694+0000 7fefc3e19740 1 -- 172.21.15.112:0/1153273140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f3441d410 con 0x564f3439b9b0 2024-04-25T18:48:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.694+0000 7fef9fe00640 1 -- 172.21.15.112:0/1153273140 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefa4031d10 con 0x564f3439b9b0 2024-04-25T18:48:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.695+0000 7fef9fe00640 1 -- 172.21.15.112:0/1153273140 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fefac0037d0 con 0x564f3439b9b0 2024-04-25T18:48:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.695+0000 7fefc3e19740 10 client.15210 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:48:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.695+0000 7fefc3e19740 10 client.15210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:48:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.695+0000 7fefc3e19740 20 client.15210 populate_metadata read hostname 'smithi112' 2024-04-25T18:48:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.695+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564f3441db30 0x564f3441ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.695+0000 7fefc3e19740 4 client.15210 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:48:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.695+0000 7fefc3e19740 1 -- 172.21.15.112:0/1153273140 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f3430f320 con 0x564f3441db30 2024-04-25T18:48:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.695+0000 7fefb3e00640 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564f3441db30 0x564f3441ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.696+0000 7fefb3e00640 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564f3441db30 0x564f3441ff20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ba3e6c81cafa88 server_cookie=25b639eba650ea8e in_seq=0 out_seq=0 2024-04-25T18:48:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.696+0000 7fef9fe00640 10 client.15210 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fef9fe00640 1 -- 172.21.15.112:0/1153273140 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f3430f320 con 0x564f3441db30 2024-04-25T18:48:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fef9fe00640 10 client.15210 handle_command_reply: tid=0 2024-04-25T18:48:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fefc3e19740 1 client.15210 shutdown 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fefc3e19740 1 -- 172.21.15.112:0/1153273140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 msgr2=0x564f343aef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343aef10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fefac000c00 tx=0x7fefac01c630 comp rx=0 tx=0).stop 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fefc3e19740 1 -- 172.21.15.112:0/1153273140 shutdown_connections 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f343a6b90 0x564f343ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564f3439c2d0 0x564f343ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564f3441db30 0x564f3441ff20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fefc3e19740 1 --2- 172.21.15.112:0/1153273140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564f3439b9b0 0x564f343aef10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fef9fe00640 0 client.15210 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fefc3e19740 1 -- 172.21.15.112:0/1153273140 >> 172.21.15.112:0/1153273140 conn(0x564f34397ec0 msgr2=0x564f343a3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.702+0000 7fefc3e19740 1 -- 172.21.15.112:0/1153273140 shutdown_connections 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefc3e19740 1 -- 172.21.15.112:0/1153273140 wait complete. 2024-04-25T18:48:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefc3e19740 20 client.15210 trim_cache size 0 max 16384 2024-04-25T18:48:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:48:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:48:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:48:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefbb200640 1 -- 172.21.15.112:0/3182615286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fefa00a3de0 msgr2=0x7fefa00a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefbb200640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fefa00a3de0 0x7fefa00a62a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fefb401b040 tx=0x7fefb4000f40 comp rx=0 tx=0).stop 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefbb200640 1 -- 172.21.15.112:0/3182615286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefbc103890 msgr2=0x7fefbc101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefbb200640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefbc103890 0x7fefbc101100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005ab0 tx=0x7fefa4003c10 comp rx=0 tx=0).stop 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefbb200640 1 -- 172.21.15.112:0/3182615286 shutdown_connections 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefbb200640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fefa00a3de0 0x7fefa00a62a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefbb200640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefbc10c340 0x7fefbc101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefbb200640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fefbc103890 0x7fefbc101100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefbb200640 1 --2- 172.21.15.112:0/3182615286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefbc102f70 0x7fefbc100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.703+0000 7fefbb200640 1 -- 172.21.15.112:0/3182615286 >> 172.21.15.112:0/3182615286 conn(0x7fefbc0fc770 msgr2=0x7fefbc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.704+0000 7fefbb200640 1 -- 172.21.15.112:0/3182615286 shutdown_connections 2024-04-25T18:48:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:48:47.704+0000 7fefbb200640 1 -- 172.21.15.112:0/3182615286 wait complete. 2024-04-25T18:48:48.321 INFO:tasks.workunit.client.0.smithi112.stderr:Updating files: 65% (8143/12492) Updating files: 66% (8245/12492) Updating files: 67% (8370/12492) Updating files: 68% (8495/12492) Updating files: 69% (8620/12492) Updating files: 70% (8745/12492) Updating files: 71% (8870/12492) Updating files: 72% (8995/12492) Updating files: 73% (9120/12492) Updating files: 74% (9245/12492) Updating files: 75% (9369/12492) Updating files: 76% (9494/12492) Updating files: 77% (9619/12492) Updating files: 78% (9744/12492) Updating files: 79% (9869/12492) Updating files: 80% (9994/12492) Updating files: 81% (10119/12492) Updating files: 82% (10244/12492) Updating files: 83% (10369/12492) Updating files: 84% (10494/12492) Updating files: 85% (10619/12492) Updating files: 86% (10744/12492) Updating files: 87% (10869/12492) Updating files: 88% (10993/12492) Updating files: 89% (11118/12492) Updating files: 90% (11243/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T18:48:48.589 INFO:tasks.workunit.client.0.smithi112.stderr:Note: switching to '7797745e6d8b3ac036504dc61491d368615024da'. 2024-04-25T18:48:48.589 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T18:48:48.589 INFO:tasks.workunit.client.0.smithi112.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T18:48:48.589 INFO:tasks.workunit.client.0.smithi112.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr: git switch -c 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr:Or undo this operation with: 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr: git switch - 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T18:48:48.590 INFO:tasks.workunit.client.0.smithi112.stderr:HEAD is now at 7797745e6d8 qa/suites/fs: add quiescer to the fs suite 2024-04-25T18:48:48.593 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T18:48:48.660 INFO:tasks.workunit.client.0.smithi112.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T18:48:48.667 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T18:48:48.668 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T18:48:48.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:48 smithi171 ceph-mon[33266]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T18:48:48.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:48 smithi161 ceph-mon[32609]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T18:48:48.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:48 smithi112 ceph-mon[27864]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T18:48:49.514 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T18:48:49.621 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T18:48:49.709 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T18:48:49.714 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T18:48:49.714 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T18:48:49.794 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T18:48:49.803 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:48:49.803 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T18:48:49.852 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-25T18:48:49.854 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-25T18:48:49.854 DEBUG:teuthology.orchestra.run.smithi112:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7797745e6d8b3ac036504dc61491d368615024da TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-25T18:48:50.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:50 smithi171 ceph-mon[33266]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T18:48:50.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:50 smithi161 ceph-mon[32609]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T18:48:50.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:50 smithi112 ceph-mon[27864]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T18:48:52.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:52 smithi171 ceph-mon[33266]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T18:48:52.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:52 smithi161 ceph-mon[32609]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T18:48:52.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:52 smithi112 ceph-mon[27864]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T18:48:54.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:54 smithi171 ceph-mon[33266]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T18:48:54.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:54 smithi161 ceph-mon[32609]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T18:48:54.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:54 smithi112 ceph-mon[27864]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T18:48:56.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:56 smithi171 ceph-mon[33266]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T18:48:56.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:56 smithi161 ceph-mon[32609]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T18:48:56.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:56 smithi112 ceph-mon[27864]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T18:48:57.208 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:48:57.668 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:48:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:58 smithi171 ceph-mon[33266]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:48:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:48:58 smithi171 ceph-mon[33266]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T18:48:58.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:58 smithi112 ceph-mon[27864]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:48:58.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:48:58 smithi112 ceph-mon[27864]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T18:48:58.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:58 smithi161 ceph-mon[32609]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:48:58.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:48:58 smithi161 ceph-mon[32609]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T18:49:00.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.111+0000 7f6891000700 1 -- 172.21.15.112:0/583340762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c10f440 msgr2=0x7f688c111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:00.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.111+0000 7f6891000700 1 --2- 172.21.15.112:0/583340762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c10f440 0x7f688c111830 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6884009f90 tx=0x7f688400b330 comp rx=0 tx=0).stop 2024-04-25T18:49:00.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.111+0000 7f6891000700 1 -- 172.21.15.112:0/583340762 shutdown_connections 2024-04-25T18:49:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.111+0000 7f6891000700 1 --2- 172.21.15.112:0/583340762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c10f440 0x7f688c111830 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.111+0000 7f6891000700 1 --2- 172.21.15.112:0/583340762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f688c074c70 0x7f688c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.111+0000 7f6891000700 1 --2- 172.21.15.112:0/583340762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688c075590 0x7f688c10ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.111+0000 7f6891000700 1 -- 172.21.15.112:0/583340762 >> 172.21.15.112:0/583340762 conn(0x7f688c079ba0 msgr2=0x7f688c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.112+0000 7f6891000700 1 -- 172.21.15.112:0/583340762 shutdown_connections 2024-04-25T18:49:00.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.112+0000 7f6891000700 1 -- 172.21.15.112:0/583340762 wait complete. 2024-04-25T18:49:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.113+0000 7f6891000700 1 Processor -- start 2024-04-25T18:49:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.113+0000 7f6891000700 1 -- start start 2024-04-25T18:49:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6891000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f688c074c70 0x7f688c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6891000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c075590 0x7f688c19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6891000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688c10f440 0x7f688c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6891000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f688c114e70 con 0x7f688c10f440 2024-04-25T18:49:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6891000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f688c114cf0 con 0x7f688c075590 2024-04-25T18:49:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6891000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f688c114ff0 con 0x7f688c074c70 2024-04-25T18:49:00.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6889e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f688c074c70 0x7f688c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f688a800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688c10f440 0x7f688c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6889e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f688c074c70 0x7f688c19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:36886/0 (socket says 172.21.15.112:36886) 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6889e00700 1 -- 172.21.15.112:0/1808249286 learned_addr learned my addr 172.21.15.112:0/1808249286 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6889400700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c075590 0x7f688c19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6889e00700 1 -- 172.21.15.112:0/1808249286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c075590 msgr2=0x7f688c19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.114+0000 7f6889e00700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c075590 0x7f688c19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.115+0000 7f6889e00700 1 -- 172.21.15.112:0/1808249286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688c10f440 msgr2=0x7f688c19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.115+0000 7f6889e00700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688c10f440 0x7f688c19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.115+0000 7f6889e00700 1 -- 172.21.15.112:0/1808249286 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c1a0290 con 0x7f688c074c70 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.115+0000 7f688a800700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688c10f440 0x7f688c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.115+0000 7f6889e00700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f688c074c70 0x7f688c19b280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f687c00e9a0 tx=0x7f687c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.115+0000 7f6882a00700 1 -- 172.21.15.112:0/1808249286 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c00cbd0 con 0x7f688c074c70 2024-04-25T18:49:00.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.115+0000 7f6882a00700 1 -- 172.21.15.112:0/1808249286 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687c00cd70 con 0x7f688c074c70 2024-04-25T18:49:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.115+0000 7f6891000700 1 -- 172.21.15.112:0/1808249286 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688c1a0580 con 0x7f688c074c70 2024-04-25T18:49:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.115+0000 7f6882a00700 1 -- 172.21.15.112:0/1808249286 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c018680 con 0x7f688c074c70 2024-04-25T18:49:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.115+0000 7f6891000700 1 -- 172.21.15.112:0/1808249286 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688c1a7e00 con 0x7f688c074c70 2024-04-25T18:49:00.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.116+0000 7f6891000700 1 -- 172.21.15.112:0/1808249286 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f688c069730 con 0x7f688c074c70 2024-04-25T18:49:00.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.118+0000 7f6882a00700 1 -- 172.21.15.112:0/1808249286 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f687c010430 con 0x7f688c074c70 2024-04-25T18:49:00.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.120+0000 7f6882a00700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f68700a3e30 0x7f68700a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.120+0000 7f6882a00700 1 -- 172.21.15.112:0/1808249286 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f687c0d0ec0 con 0x7f688c074c70 2024-04-25T18:49:00.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.120+0000 7f6889400700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f68700a3e30 0x7f68700a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.121+0000 7f6889400700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f68700a3e30 0x7f68700a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f688c111100 tx=0x7f6878009310 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:00.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.122+0000 7f6882a00700 1 -- 172.21.15.112:0/1808249286 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f687c09d500 con 0x7f688c074c70 2024-04-25T18:49:00.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.416+0000 7f6891000700 1 -- 172.21.15.112:0/1808249286 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f688c02df80 con 0x7f688c074c70 2024-04-25T18:49:00.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.417+0000 7f6882a00700 1 -- 172.21.15.112:0/1808249286 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7f687c09ce00 con 0x7f688c074c70 2024-04-25T18:49:00.418 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:49:00.419 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:47:25.174463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24493,24511,24512]},"id":1}]} 2024-04-25T18:49:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.421+0000 7f6891000700 1 -- 172.21.15.112:0/1808249286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f68700a3e30 msgr2=0x7f68700a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:00.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.421+0000 7f6891000700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f68700a3e30 0x7f68700a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f688c111100 tx=0x7f6878009310 comp rx=0 tx=0).stop 2024-04-25T18:49:00.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.421+0000 7f6891000700 1 -- 172.21.15.112:0/1808249286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f688c074c70 msgr2=0x7f688c19b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:00.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.421+0000 7f6891000700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f688c074c70 0x7f688c19b280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f687c00e9a0 tx=0x7f687c00ed60 comp rx=0 tx=0).stop 2024-04-25T18:49:00.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.422+0000 7f6891000700 1 -- 172.21.15.112:0/1808249286 shutdown_connections 2024-04-25T18:49:00.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.422+0000 7f6891000700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f68700a3e30 0x7f68700a62f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.422+0000 7f6891000700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f688c10f440 0x7f688c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.422+0000 7f6891000700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f688c074c70 0x7f688c19b280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.422+0000 7f6891000700 1 --2- 172.21.15.112:0/1808249286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c075590 0x7f688c19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.422+0000 7f6891000700 1 -- 172.21.15.112:0/1808249286 >> 172.21.15.112:0/1808249286 conn(0x7f688c079ba0 msgr2=0x7f688c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:00.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.422+0000 7f6891000700 1 -- 172.21.15.112:0/1808249286 shutdown_connections 2024-04-25T18:49:00.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:00.422+0000 7f6891000700 1 -- 172.21.15.112:0/1808249286 wait complete. 2024-04-25T18:49:00.425 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-04-25T18:49:00.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:00 smithi171 ceph-mon[33266]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T18:49:00.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:00 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1808249286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:49:00.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:00 smithi161 ceph-mon[32609]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T18:49:00.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:00 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1808249286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:49:00.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:00 smithi112 ceph-mon[27864]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T18:49:00.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:00 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1808249286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:49:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.850+0000 7ff4bec00640 1 -- 172.21.15.112:0/4246229941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b81092a0 msgr2=0x7ff4b8110e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.850+0000 7ff4bec00640 1 --2- 172.21.15.112:0/4246229941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b81092a0 0x7ff4b8110e70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009860 tx=0x7ff4a802a340 comp rx=0 tx=0).stop 2024-04-25T18:49:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.850+0000 7ff4bec00640 1 -- 172.21.15.112:0/4246229941 shutdown_connections 2024-04-25T18:49:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.850+0000 7ff4bec00640 1 --2- 172.21.15.112:0/4246229941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b81092a0 0x7ff4b8110e70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.850+0000 7ff4bec00640 1 --2- 172.21.15.112:0/4246229941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4b8104b80 0x7ff4b8108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.850+0000 7ff4bec00640 1 --2- 172.21.15.112:0/4246229941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4b81041d0 0x7ff4b81045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.850+0000 7ff4bec00640 1 -- 172.21.15.112:0/4246229941 >> 172.21.15.112:0/4246229941 conn(0x7ff4b80ffa70 msgr2=0x7ff4b8101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.850+0000 7ff4bec00640 1 -- 172.21.15.112:0/4246229941 shutdown_connections 2024-04-25T18:49:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.850+0000 7ff4bec00640 1 -- 172.21.15.112:0/4246229941 wait complete. 2024-04-25T18:49:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.851+0000 7ff4bec00640 1 Processor -- start 2024-04-25T18:49:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.851+0000 7ff4bec00640 1 -- start start 2024-04-25T18:49:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.852+0000 7ff4bec00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b81041d0 0x7ff4b81a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.852+0000 7ff4bec00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4b81092a0 0x7ff4b81a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.852+0000 7ff4bec00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4b819a5f0 0x7ff4b819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.852+0000 7ff4bec00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b8114210 con 0x7ff4b81092a0 2024-04-25T18:49:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.852+0000 7ff4bec00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b8114090 con 0x7ff4b81041d0 2024-04-25T18:49:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.852+0000 7ff4bec00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b8114390 con 0x7ff4b819a5f0 2024-04-25T18:49:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.852+0000 7ff4bce00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4b81092a0 0x7ff4b81a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.852+0000 7ff4bce00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4b81092a0 0x7ff4b81a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55762/0 (socket says 172.21.15.112:55762) 2024-04-25T18:49:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.852+0000 7ff4bce00640 1 -- 172.21.15.112:0/1482196348 learned_addr learned my addr 172.21.15.112:0/1482196348 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.852+0000 7ff4bd800640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b81041d0 0x7ff4b81a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.852+0000 7ff4be200640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4b819a5f0 0x7ff4b819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.853+0000 7ff4bce00640 1 -- 172.21.15.112:0/1482196348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4b819a5f0 msgr2=0x7ff4b819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.853+0000 7ff4bce00640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4b819a5f0 0x7ff4b819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.853+0000 7ff4bce00640 1 -- 172.21.15.112:0/1482196348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b81041d0 msgr2=0x7ff4b81a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.853+0000 7ff4bce00640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b81041d0 0x7ff4b81a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.853+0000 7ff4bce00640 1 -- 172.21.15.112:0/1482196348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b819b110 con 0x7ff4b81092a0 2024-04-25T18:49:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.853+0000 7ff4bce00640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4b81092a0 0x7ff4b81a0980 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c3f0 tx=0x7ff4a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.853+0000 7ff4b6000640 1 -- 172.21.15.112:0/1482196348 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4004b20 con 0x7ff4b81092a0 2024-04-25T18:49:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.853+0000 7ff4bec00640 1 -- 172.21.15.112:0/1482196348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b819b400 con 0x7ff4b81092a0 2024-04-25T18:49:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.853+0000 7ff4b6000640 1 -- 172.21.15.112:0/1482196348 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a4003e20 con 0x7ff4b81092a0 2024-04-25T18:49:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.853+0000 7ff4b6000640 1 -- 172.21.15.112:0/1482196348 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4004800 con 0x7ff4b81092a0 2024-04-25T18:49:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.853+0000 7ff4bec00640 1 -- 172.21.15.112:0/1482196348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b81a7650 con 0x7ff4b81092a0 2024-04-25T18:49:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.857+0000 7ff4c31e4740 1 Processor -- start 2024-04-25T18:49:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.857+0000 7ff4c31e4740 1 -- start start 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.857+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db09b0 0x5624c3db0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.857+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db12d0 0x5624c3dbb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.857+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3dbbb90 0x5624c3dbdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.857+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5624c3d16900 con 0x5624c3dbbb90 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.857+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5624c3cbcef0 con 0x5624c3db09b0 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.857+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5624c3d128c0 con 0x5624c3db12d0 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.857+0000 7ff4b6000640 1 -- 172.21.15.112:0/1482196348 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7ff4a401d020 con 0x7ff4b81092a0 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.858+0000 7ff4be200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db09b0 0x5624c3db0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.858+0000 7ff4bd800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db12d0 0x5624c3dbb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.858+0000 7ff4be200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db09b0 0x5624c3db0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:36682/0 (socket says 172.21.15.112:36682) 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.858+0000 7ff4be200640 1 -- 172.21.15.112:0/1377044384 learned_addr learned my addr 172.21.15.112:0/1377044384 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.858+0000 7ff4bce00640 1 --2- 172.21.15.112:0/1377044384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3dbbb90 0x5624c3dbdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.858+0000 7ff4be200640 1 -- 172.21.15.112:0/1377044384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db12d0 msgr2=0x5624c3dbb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.858+0000 7ff4be200640 1 --2- 172.21.15.112:0/1377044384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db12d0 0x5624c3dbb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.858+0000 7ff4be200640 1 -- 172.21.15.112:0/1377044384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3dbbb90 msgr2=0x5624c3dbdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.858+0000 7ff4be200640 1 --2- 172.21.15.112:0/1377044384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3dbbb90 0x5624c3dbdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.858+0000 7ff4be200640 1 -- 172.21.15.112:0/1377044384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c3a29210 con 0x5624c3db09b0 2024-04-25T18:49:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.858+0000 7ff4be200640 1 --2- 172.21.15.112:0/1377044384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db09b0 0x5624c3db0d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8035040 tx=0x7ff4a8005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff52bb9623ec150a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.859+0000 7ff4aea00640 1 -- 172.21.15.112:0/1377044384 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8037040 con 0x5624c3db09b0 2024-04-25T18:49:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.859+0000 7ff4aea00640 1 -- 172.21.15.112:0/1377044384 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a8030450 con 0x5624c3db09b0 2024-04-25T18:49:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.859+0000 7ff4aea00640 1 -- 172.21.15.112:0/1377044384 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8003810 con 0x5624c3db09b0 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.859+0000 7ff4b6000640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff49c0a3d10 0x7ff49c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.859+0000 7ff4b6000640 1 -- 172.21.15.112:0/1482196348 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7ff4a40d07c0 con 0x7ff4b81092a0 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.859+0000 7ff4bd800640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff49c0a3d10 0x7ff49c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.860+0000 7ff4c31e4740 1 -- 172.21.15.112:0/1377044384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db09b0 msgr2=0x5624c3db0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.860+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/1377044384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db09b0 0x5624c3db0d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8035040 tx=0x7ff4a8005d90 comp rx=0 tx=0).stop 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.860+0000 7ff4c31e4740 1 -- 172.21.15.112:0/1377044384 shutdown_connections 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.860+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/1377044384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3dbbb90 0x5624c3dbdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.860+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/1377044384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db12d0 0x5624c3dbb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.860+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/1377044384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db09b0 0x5624c3db0d90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.860+0000 7ff4c31e4740 1 -- 172.21.15.112:0/1377044384 >> 172.21.15.112:0/1377044384 conn(0x5624c3dacec0 msgr2=0x5624c3dad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.860+0000 7ff4c31e4740 1 -- 172.21.15.112:0/1377044384 shutdown_connections 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.860+0000 7ff4c31e4740 1 -- 172.21.15.112:0/1377044384 wait complete. 2024-04-25T18:49:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.860+0000 7ff4bd800640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff49c0a3d10 0x7ff49c0a61d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5624c3db1f50 tx=0x7ff4b000b410 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.860+0000 7ff4c31e4740 1 -- start start 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.861+0000 7ff4c31e4740 1 Processor -- start 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.861+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 0x5624c3db8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.861+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db12d0 0x5624c3db9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.861+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3dbbb90 0x5624c3db95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.861+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5624c3dbe760 con 0x5624c3db09b0 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.861+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5624c3cd0310 con 0x5624c3db12d0 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.861+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5624c3dbe8e0 con 0x5624c3dbbb90 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.861+0000 7ff4c31e4740 10 client.0 _ll_register_callbacks cb 0x5624c3c9ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4bd800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 0x5624c3db8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4be200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3dbbb90 0x5624c3db95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4be200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3dbbb90 0x5624c3db95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:36914/0 (socket says 172.21.15.112:36914) 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4be200640 1 -- 172.21.15.112:0/4144535957 learned_addr learned my addr 172.21.15.112:0/4144535957 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4bce00640 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db12d0 0x5624c3db9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4be200640 1 -- 172.21.15.112:0/4144535957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db12d0 msgr2=0x5624c3db9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4be200640 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db12d0 0x5624c3db9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4be200640 1 -- 172.21.15.112:0/4144535957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 msgr2=0x5624c3db8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4be200640 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 0x5624c3db8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4be200640 1 -- 172.21.15.112:0/4144535957 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c3af5620 con 0x5624c3dbbb90 2024-04-25T18:49:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4be200640 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3dbbb90 0x5624c3db95d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009830 tx=0x7ff4a8030600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9f52f2c7806d86c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.862+0000 7ff4a3400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:49:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.863+0000 7ff4a3400640 1 -- 172.21.15.112:0/4144535957 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8037030 con 0x5624c3dbbb90 2024-04-25T18:49:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.863+0000 7ff4c31e4740 1 -- 172.21.15.112:0/4144535957 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a8030450 con 0x5624c3dbbb90 2024-04-25T18:49:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.863+0000 7ff4a3400640 1 -- 172.21.15.112:0/4144535957 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7ff4a8005230 con 0x5624c3dbbb90 2024-04-25T18:49:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.863+0000 7ff4a3400640 1 -- 172.21.15.112:0/4144535957 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff4a8055050 con 0x5624c3dbbb90 2024-04-25T18:49:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.863+0000 7ff4c31e4740 10 client.25387 fetch_fsmap learned FSMap version 26 2024-04-25T18:49:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.863+0000 7ff4c31e4740 1 -- 172.21.15.112:0/4144535957 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624c3dbe7b0 con 0x5624c3dbbb90 2024-04-25T18:49:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.863+0000 7ff4a3400640 1 -- 172.21.15.112:0/4144535957 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a8030450 con 0x5624c3dbbb90 2024-04-25T18:49:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.863+0000 7ff4a3400640 1 -- 172.21.15.112:0/4144535957 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff4a803d5a0 con 0x5624c3dbbb90 2024-04-25T18:49:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.863+0000 7ff4c31e4740 10 client.25387 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:49:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.863+0000 7ff4c31e4740 10 client.25387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:49:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.864+0000 7ff4c31e4740 20 client.25387 populate_metadata read hostname 'smithi112' 2024-04-25T18:49:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.864+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5624c3dcbc60 0x5624c3dce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.864+0000 7ff4c31e4740 4 client.25387 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:49:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.864+0000 7ff4c31e4740 1 -- 172.21.15.112:0/4144535957 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624c3d2a8b0 con 0x5624c3dcbc60 2024-04-25T18:49:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.875+0000 7ff4bd800640 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5624c3dcbc60 0x5624c3dce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.882+0000 7ff4bd800640 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5624c3dcbc60 0x5624c3dce050 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ccae9778821ff01 server_cookie=86e332794082d7e1 in_seq=0 out_seq=0 2024-04-25T18:49:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.882+0000 7ff4a3400640 10 client.25387 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.883+0000 7ff4a3400640 1 -- 172.21.15.112:0/4144535957 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624c3d2a8b0 con 0x5624c3dcbc60 2024-04-25T18:49:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.883+0000 7ff4a3400640 10 client.25387 handle_command_reply: tid=0 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.883+0000 7ff4c31e4740 1 client.25387 shutdown 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.883+0000 7ff4c31e4740 1 -- 172.21.15.112:0/4144535957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3dbbb90 msgr2=0x5624c3db95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.883+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3dbbb90 0x5624c3db95d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009830 tx=0x7ff4a8030600 comp rx=0 tx=0).stop 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.884+0000 7ff4c31e4740 1 -- 172.21.15.112:0/4144535957 shutdown_connections 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.884+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3dbbb90 0x5624c3db95d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.884+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3db12d0 0x5624c3db9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.884+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5624c3dcbc60 0x5624c3dce050 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.884+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/4144535957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 0x5624c3db8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.884+0000 7ff4a3400640 0 client.25387 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.884+0000 7ff4c31e4740 1 -- 172.21.15.112:0/4144535957 >> 172.21.15.112:0/4144535957 conn(0x5624c3dacec0 msgr2=0x5624c3db2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.884+0000 7ff4c31e4740 1 -- 172.21.15.112:0/4144535957 shutdown_connections 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.884+0000 7ff4c31e4740 1 -- 172.21.15.112:0/4144535957 wait complete. 2024-04-25T18:49:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.884+0000 7ff4c31e4740 20 client.25387 trim_cache size 0 max 16384 2024-04-25T18:49:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.900+0000 7ff4c31e4740 1 Processor -- start 2024-04-25T18:49:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.900+0000 7ff4c31e4740 1 -- start start 2024-04-25T18:49:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.900+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db09b0 0x5624c3dc7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.900+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db12d0 0x5624c3db9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.900+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3dbbb90 0x5624c3dd37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.900+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81b2460 con 0x5624c3db12d0 2024-04-25T18:49:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.900+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81afd30 con 0x5624c3dbbb90 2024-04-25T18:49:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.900+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81adb80 con 0x5624c3db09b0 2024-04-25T18:49:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.901+0000 7ff4bce00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db12d0 0x5624c3db9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.901+0000 7ff4bce00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db12d0 0x5624c3db9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55786/0 (socket says 172.21.15.112:55786) 2024-04-25T18:49:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.901+0000 7ff4bce00640 1 -- 172.21.15.112:0/151658545 learned_addr learned my addr 172.21.15.112:0/151658545 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.901+0000 7ff4bd800640 1 --2- 172.21.15.112:0/151658545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db09b0 0x5624c3dc7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.901+0000 7ff4be200640 1 --2- 172.21.15.112:0/151658545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3dbbb90 0x5624c3dd37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.901+0000 7ff4bce00640 1 -- 172.21.15.112:0/151658545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db09b0 msgr2=0x5624c3dc7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.901+0000 7ff4bce00640 1 --2- 172.21.15.112:0/151658545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db09b0 0x5624c3dc7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.901+0000 7ff4bce00640 1 -- 172.21.15.112:0/151658545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3dbbb90 msgr2=0x5624c3dd37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.901+0000 7ff4bce00640 1 --2- 172.21.15.112:0/151658545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3dbbb90 0x5624c3dd37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.901+0000 7ff4bce00640 1 -- 172.21.15.112:0/151658545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c3dc8b40 con 0x5624c3db12d0 2024-04-25T18:49:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.901+0000 7ff4bce00640 1 --2- 172.21.15.112:0/151658545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db12d0 0x5624c3db9220 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400cdb0 tx=0x7ff4a4004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f666f121e729e8b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.902+0000 7ff4ae000640 1 -- 172.21.15.112:0/151658545 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a401a040 con 0x5624c3db12d0 2024-04-25T18:49:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.902+0000 7ff4ae000640 1 -- 172.21.15.112:0/151658545 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a409ce00 con 0x5624c3db12d0 2024-04-25T18:49:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.902+0000 7ff4ae000640 1 -- 172.21.15.112:0/151658545 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40a5370 con 0x5624c3db12d0 2024-04-25T18:49:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.902+0000 7ff4c31e4740 1 -- 172.21.15.112:0/151658545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db12d0 msgr2=0x5624c3db9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.902+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/151658545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db12d0 0x5624c3db9220 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400cdb0 tx=0x7ff4a4004f30 comp rx=0 tx=0).stop 2024-04-25T18:49:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.903+0000 7ff4c31e4740 1 -- 172.21.15.112:0/151658545 shutdown_connections 2024-04-25T18:49:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.903+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/151658545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3dbbb90 0x5624c3dd37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.903+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/151658545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db12d0 0x5624c3db9220 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.903+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/151658545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db09b0 0x5624c3dc7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.903+0000 7ff4c31e4740 1 -- 172.21.15.112:0/151658545 >> 172.21.15.112:0/151658545 conn(0x5624c3dacec0 msgr2=0x5624c3db8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.903+0000 7ff4c31e4740 1 -- 172.21.15.112:0/151658545 shutdown_connections 2024-04-25T18:49:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.903+0000 7ff4c31e4740 1 -- 172.21.15.112:0/151658545 wait complete. 2024-04-25T18:49:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.903+0000 7ff4c31e4740 1 -- start start 2024-04-25T18:49:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.903+0000 7ff4c31e4740 1 Processor -- start 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 0x5624c3dc1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db12d0 0x5624c3dc1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4c31e4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3dbbb90 0x5624c3dc22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81ae1f0 con 0x5624c3db09b0 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81af660 con 0x5624c3dbbb90 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4c31e4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81ae8c0 con 0x5624c3db12d0 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4c31e4740 10 client.0 _ll_register_callbacks cb 0x7ff4a8009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4bd800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 0x5624c3dc1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4bd800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 0x5624c3dc1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55792/0 (socket says 172.21.15.112:55792) 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4bd800640 1 -- 172.21.15.112:0/2630072644 learned_addr learned my addr 172.21.15.112:0/2630072644 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4bce00640 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db12d0 0x5624c3dc1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4be200640 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3dbbb90 0x5624c3dc22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4bd800640 1 -- 172.21.15.112:0/2630072644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db12d0 msgr2=0x5624c3dc1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4bd800640 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db12d0 0x5624c3dc1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4bd800640 1 -- 172.21.15.112:0/2630072644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3dbbb90 msgr2=0x5624c3dc22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4bd800640 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3dbbb90 0x5624c3dc22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.904+0000 7ff4bd800640 1 -- 172.21.15.112:0/2630072644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c3e31380 con 0x5624c3db09b0 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.905+0000 7ff4bd800640 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 0x5624c3dc1820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5624c3dce750 tx=0x7ff4b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ac3120b6ea7dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.905+0000 7ff4a3400640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.905+0000 7ff4a3400640 1 -- 172.21.15.112:0/2630072644 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b002a380 con 0x5624c3db09b0 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.905+0000 7ff4a3400640 1 -- 172.21.15.112:0/2630072644 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7ff4b002a520 con 0x5624c3db09b0 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.905+0000 7ff4c31e4740 1 -- 172.21.15.112:0/2630072644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a409ce00 con 0x5624c3db09b0 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.905+0000 7ff4a3400640 1 -- 172.21.15.112:0/2630072644 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff4b0039050 con 0x5624c3db09b0 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.905+0000 7ff4c31e4740 10 client.15234 fetch_fsmap learned FSMap version 26 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.905+0000 7ff4c31e4740 1 -- 172.21.15.112:0/2630072644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624c3e32690 con 0x5624c3db09b0 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.906+0000 7ff4a3400640 1 -- 172.21.15.112:0/2630072644 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a409ce00 con 0x5624c3db09b0 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.906+0000 7ff4a3400640 1 -- 172.21.15.112:0/2630072644 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff4b00049d0 con 0x5624c3db09b0 2024-04-25T18:49:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.906+0000 7ff4c31e4740 10 client.15234 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:49:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.906+0000 7ff4c31e4740 10 client.15234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:49:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.906+0000 7ff4c31e4740 20 client.15234 populate_metadata read hostname 'smithi112' 2024-04-25T18:49:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.906+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5624c3e32d60 0x5624c3e35150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.906+0000 7ff4c31e4740 4 client.15234 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:49:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.906+0000 7ff4c31e4740 1 -- 172.21.15.112:0/2630072644 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624c3d24320 con 0x5624c3e32d60 2024-04-25T18:49:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.906+0000 7ff4bce00640 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5624c3e32d60 0x5624c3e35150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.907+0000 7ff4bce00640 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5624c3e32d60 0x5624c3e35150 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd393a18fb496c1b server_cookie=296ce65db35d1d51 in_seq=0 out_seq=0 2024-04-25T18:49:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:00.907+0000 7ff4a3400640 10 client.15234 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:01.355 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24512 quiesce db --set-id b8363855 --release --await' 2024-04-25T18:49:01.356 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24512 quiesce db --set-id b8363855 --release --await 2024-04-25T18:49:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4a3400640 1 -- 172.21.15.112:0/2630072644 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624c3d24320 con 0x5624c3e32d60 2024-04-25T18:49:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4a3400640 10 client.15234 handle_command_reply: tid=0 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4c31e4740 1 client.15234 shutdown 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4c31e4740 1 -- 172.21.15.112:0/2630072644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 msgr2=0x5624c3dc1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 0x5624c3dc1820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5624c3dce750 tx=0x7ff4b00046c0 comp rx=0 tx=0).stop 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4c31e4740 1 -- 172.21.15.112:0/2630072644 shutdown_connections 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624c3dbbb90 0x5624c3dc22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624c3db12d0 0x5624c3dc1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5624c3e32d60 0x5624c3e35150 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4c31e4740 1 --2- 172.21.15.112:0/2630072644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5624c3db09b0 0x5624c3dc1820 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4a3400640 0 client.15234 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.414+0000 7ff4c31e4740 1 -- 172.21.15.112:0/2630072644 >> 172.21.15.112:0/2630072644 conn(0x5624c3dacec0 msgr2=0x5624c3db2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.415+0000 7ff4c31e4740 1 -- 172.21.15.112:0/2630072644 shutdown_connections 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.415+0000 7ff4c31e4740 1 -- 172.21.15.112:0/2630072644 wait complete. 2024-04-25T18:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.415+0000 7ff4c31e4740 20 client.15234 trim_cache size 0 max 16384 2024-04-25T18:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 -- 172.21.15.112:0/1482196348 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff49c0a3d10 msgr2=0x7ff49c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff49c0a3d10 0x7ff49c0a61d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5624c3db1f50 tx=0x7ff4b000b410 comp rx=0 tx=0).stop 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 -- 172.21.15.112:0/1482196348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4b81092a0 msgr2=0x7ff4b81a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4b81092a0 0x7ff4b81a0980 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c3f0 tx=0x7ff4a400c8b0 comp rx=0 tx=0).stop 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 -- 172.21.15.112:0/1482196348 shutdown_connections 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff49c0a3d10 0x7ff49c0a61d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4b819a5f0 0x7ff4b819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4b81092a0 0x7ff4b81a0980 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 --2- 172.21.15.112:0/1482196348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b81041d0 0x7ff4b81a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 -- 172.21.15.112:0/1482196348 >> 172.21.15.112:0/1482196348 conn(0x7ff4b80ffa70 msgr2=0x7ff4b8100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 -- 172.21.15.112:0/1482196348 shutdown_connections 2024-04-25T18:49:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:01.416+0000 7ff4bec00640 1 -- 172.21.15.112:0/1482196348 wait complete. 2024-04-25T18:49:01.799 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:49:02.226 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: *** Caught signal (Segmentation fault) ** 2024-04-25T18:49:02.226 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: in thread 7f5884200700 thread_name:md_log_replay 2024-04-25T18:49:02.226 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T18:49:02.226 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f58987d8d20] 2024-04-25T18:49:02.226 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55ca86e7838f] 2024-04-25T18:49:02.226 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ca86e7bb23] 2024-04-25T18:49:02.226 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 4: (MDLog::_replay_thread()+0x617) [0x55ca86ed6647] 2024-04-25T18:49:02.226 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ca86edfd31] 2024-04-25T18:49:02.226 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f58987ce1ca] 2024-04-25T18:49:02.226 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 7: clone() 2024-04-25T18:49:02.226 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 2024-04-25T18:49:01.964+0000 7f5884200700 -1 *** Caught signal (Segmentation fault) ** 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: in thread 7f5884200700 thread_name:md_log_replay 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f58987d8d20] 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55ca86e7838f] 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ca86e7bb23] 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 4: (MDLog::_replay_thread()+0x617) [0x55ca86ed6647] 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ca86edfd31] 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f58987ce1ca] 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 7: clone() 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 0> 2024-04-25T18:49:01.964+0000 7f5884200700 -1 *** Caught signal (Segmentation fault) ** 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: in thread 7f5884200700 thread_name:md_log_replay 2024-04-25T18:49:02.227 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 2024-04-25T18:49:02.228 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T18:49:02.228 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f58987d8d20] 2024-04-25T18:49:02.228 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55ca86e7838f] 2024-04-25T18:49:02.228 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ca86e7bb23] 2024-04-25T18:49:02.228 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 4: (MDLog::_replay_thread()+0x617) [0x55ca86ed6647] 2024-04-25T18:49:02.228 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ca86edfd31] 2024-04-25T18:49:02.228 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f58987ce1ca] 2024-04-25T18:49:02.228 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 7: clone() 2024-04-25T18:49:02.228 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T18:49:02.228 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 2024-04-25T18:49:02.228 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: -3> 2024-04-25T18:49:01.964+0000 7f5884200700 -1 *** Caught signal (Segmentation fault) ** 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: in thread 7f5884200700 thread_name:md_log_replay 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f58987d8d20] 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55ca86e7838f] 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55ca86e7bb23] 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 4: (MDLog::_replay_thread()+0x617) [0x55ca86ed6647] 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55ca86edfd31] 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f58987ce1ca] 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 7: clone() 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[66762]: 2024-04-25T18:49:02.229 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 podman[86196]: 2024-04-25 18:49:02.093384651 +0000 UTC m=+0.044200737 container died 0135fb6acc076f5b567cf23eda5f7ac428febbd4c66c8bc3c72f5bd40cf4cb1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T18:49:02.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:02 smithi171 ceph-mon[33266]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T18:49:02.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:02 smithi161 ceph-mon[32609]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T18:49:02.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:02 smithi112 ceph-mon[27864]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T18:49:03.215 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 podman[86196]: 2024-04-25 18:49:02.928898944 +0000 UTC m=+0.879715042 container remove 0135fb6acc076f5b567cf23eda5f7ac428febbd4c66c8bc3c72f5bd40cf4cb1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T18:49:03.215 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:02 smithi112 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.g.service: Main process exited, code=exited, status=139/n/a 2024-04-25T18:49:04.616 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:04 smithi112 ceph-mon[27864]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T18:49:04.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:04 smithi171 ceph-mon[33266]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T18:49:04.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:04 smithi161 ceph-mon[32609]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T18:49:04.905 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:04 smithi112 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.g.service: Failed with result 'exit-code'. 2024-04-25T18:49:04.905 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:04 smithi112 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.g.service: Consumed 2.542s CPU time. 2024-04-25T18:49:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.994+0000 7f2ce9400700 1 -- 172.21.15.112:0/2682172968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ce4075590 msgr2=0x7f2ce4113730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.994+0000 7f2ce9400700 1 --2- 172.21.15.112:0/2682172968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ce4075590 0x7f2ce4113730 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0098f0 tx=0x7f2ccc009c00 comp rx=0 tx=0).stop 2024-04-25T18:49:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.995+0000 7f2ce9400700 1 -- 172.21.15.112:0/2682172968 shutdown_connections 2024-04-25T18:49:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.995+0000 7f2ce9400700 1 --2- 172.21.15.112:0/2682172968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4113c70 0x7f2ce4116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.995+0000 7f2ce9400700 1 --2- 172.21.15.112:0/2682172968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ce4074c70 0x7f2ce4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.995+0000 7f2ce9400700 1 --2- 172.21.15.112:0/2682172968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ce4075590 0x7f2ce4113730 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.995+0000 7f2ce9400700 1 -- 172.21.15.112:0/2682172968 >> 172.21.15.112:0/2682172968 conn(0x7f2ce4079b90 msgr2=0x7f2ce4079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.995+0000 7f2ce9400700 1 -- 172.21.15.112:0/2682172968 shutdown_connections 2024-04-25T18:49:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.995+0000 7f2ce9400700 1 -- 172.21.15.112:0/2682172968 wait complete. 2024-04-25T18:49:04.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.996+0000 7f2ce9400700 1 Processor -- start 2024-04-25T18:49:04.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.996+0000 7f2ce9400700 1 -- start start 2024-04-25T18:49:04.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.996+0000 7f2ce9400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4074c70 0x7f2ce41a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:04.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.996+0000 7f2ce9400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ce4075590 0x7f2ce41a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:04.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.996+0000 7f2ce9400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ce4113c70 0x7f2ce419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:04.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.996+0000 7f2ce9400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce41192b0 con 0x7f2ce4075590 2024-04-25T18:49:04.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.996+0000 7f2ce9400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce4119130 con 0x7f2ce4113c70 2024-04-25T18:49:04.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.996+0000 7f2ce9400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce4119430 con 0x7f2ce4074c70 2024-04-25T18:49:04.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.997+0000 7f2cd9e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ce4075590 0x7f2ce41a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:04.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.997+0000 7f2ce2800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4074c70 0x7f2ce41a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:04.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.997+0000 7f2cd9e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ce4075590 0x7f2ce41a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55820/0 (socket says 172.21.15.112:55820) 2024-04-25T18:49:04.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.997+0000 7f2cd9e00700 1 -- 172.21.15.112:0/3438517930 learned_addr learned my addr 172.21.15.112:0/3438517930 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:04.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.997+0000 7f2ce3200700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ce4113c70 0x7f2ce419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:04.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.997+0000 7f2cd9e00700 1 -- 172.21.15.112:0/3438517930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4074c70 msgr2=0x7f2ce41a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.997+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4074c70 0x7f2ce41a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.997+0000 7f2cd9e00700 1 -- 172.21.15.112:0/3438517930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ce4113c70 msgr2=0x7f2ce419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.997+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ce4113c70 0x7f2ce419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.997+0000 7f2cd9e00700 1 -- 172.21.15.112:0/3438517930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce419ffd0 con 0x7f2ce4075590 2024-04-25T18:49:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.997+0000 7f2ce2800700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4074c70 0x7f2ce41a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:49:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.998+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ce4075590 0x7f2ce41a5b60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009230 tx=0x7f2ccc00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:04.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.998+0000 7f2cdbe00700 1 -- 172.21.15.112:0/3438517930 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0050c0 con 0x7f2ce4075590 2024-04-25T18:49:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.998+0000 7f2ce9400700 1 -- 172.21.15.112:0/3438517930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce41a0260 con 0x7f2ce4075590 2024-04-25T18:49:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.998+0000 7f2cdbe00700 1 -- 172.21.15.112:0/3438517930 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ccc00b560 con 0x7f2ce4075590 2024-04-25T18:49:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.998+0000 7f2cdbe00700 1 -- 172.21.15.112:0/3438517930 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc003a00 con 0x7f2ce4075590 2024-04-25T18:49:05.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:04.998+0000 7f2ce9400700 1 -- 172.21.15.112:0/3438517930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce41ac3d0 con 0x7f2ce4075590 2024-04-25T18:49:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.001+0000 7f2cdbe00700 1 -- 172.21.15.112:0/3438517930 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f2ccc00b710 con 0x7f2ce4075590 2024-04-25T18:49:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.003+0000 7f2cdbe00700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2cc80a3db0 0x7f2cc80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.003+0000 7f2cf3db9b80 1 Processor -- start 2024-04-25T18:49:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.003+0000 7f2cdbe00700 1 -- 172.21.15.112:0/3438517930 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2ccc0d1730 con 0x7f2ce4075590 2024-04-25T18:49:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.003+0000 7f2cf3db9b80 1 -- start start 2024-04-25T18:49:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.003+0000 7f2ce2800700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2cc80a3db0 0x7f2cc80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 0x563029a2e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a38f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a3b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563029980f90 con 0x563029a2dd40 2024-04-25T18:49:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563029927830 con 0x563029a2e660 2024-04-25T18:49:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56302997cf50 con 0x563029a394d0 2024-04-25T18:49:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2ce2800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 0x563029a2e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cd9e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a38f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2ce2800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 0x563029a2e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55824/0 (socket says 172.21.15.112:55824) 2024-04-25T18:49:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2ce2800700 1 -- 172.21.15.112:0/1028918175 learned_addr learned my addr 172.21.15.112:0/1028918175 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2ce3200700 1 --2- 172.21.15.112:0/1028918175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a3b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2ce2800700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2cc80a3db0 0x7f2cc80a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009750 tx=0x7f2cd40119b0 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cd9e00700 1 -- 172.21.15.112:0/1028918175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 msgr2=0x563029a3b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/1028918175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a3b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cd9e00700 1 -- 172.21.15.112:0/1028918175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 msgr2=0x563029a2e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/1028918175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 0x563029a2e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2cd9e00700 1 -- 172.21.15.112:0/1028918175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302991d780 con 0x563029a2e660 2024-04-25T18:49:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.004+0000 7f2ce3200700 1 --2- 172.21.15.112:0/1028918175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a3b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:49:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.005+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/1028918175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a38f90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0cee20 tx=0x7f2ccc09d240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c3ab0d15176159 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.005+0000 7f2cd3e00700 1 -- 172.21.15.112:0/1028918175 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc021380 con 0x563029a2e660 2024-04-25T18:49:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.005+0000 7f2cd3e00700 1 -- 172.21.15.112:0/1028918175 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ccc09de00 con 0x563029a2e660 2024-04-25T18:49:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.005+0000 7f2cd3e00700 1 -- 172.21.15.112:0/1028918175 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc09d960 con 0x563029a2e660 2024-04-25T18:49:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.006+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/1028918175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 msgr2=0x563029a38f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.006+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/1028918175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a38f90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0cee20 tx=0x7f2ccc09d240 comp rx=0 tx=0).stop 2024-04-25T18:49:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.006+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/1028918175 shutdown_connections 2024-04-25T18:49:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.006+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/1028918175 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a3b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.006+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/1028918175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 0x563029a2e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.006+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/1028918175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a38f90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.006+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/1028918175 >> 172.21.15.112:0/1028918175 conn(0x56302995d050 msgr2=0x563029a2a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.006+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/1028918175 shutdown_connections 2024-04-25T18:49:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.006+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/1028918175 wait complete. 2024-04-25T18:49:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.007+0000 7f2cf3db9b80 1 -- start start 2024-04-25T18:49:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.007+0000 7f2cf3db9b80 1 Processor -- start 2024-04-25T18:49:05.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.007+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2dd40 0x563029a38c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.007+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2e660 0x563029a35f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.007+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a364d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.007+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563029927830 con 0x563029a2e660 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.007+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56302993a960 con 0x563029a2dd40 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.007+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563029a3be00 con 0x563029a394d0 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.007+0000 7f2cf3db9b80 10 client.0 _ll_register_callbacks cb 0x563029919820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.008+0000 7f2cd9e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2e660 0x563029a35f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.008+0000 7f2ce2800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2dd40 0x563029a38c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.008+0000 7f2cd9e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2e660 0x563029a35f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55838/0 (socket says 172.21.15.112:55838) 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.008+0000 7f2cd9e00700 1 -- 172.21.15.112:0/3386898179 learned_addr learned my addr 172.21.15.112:0/3386898179 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.008+0000 7f2ce3200700 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.008+0000 7f2cd9e00700 1 -- 172.21.15.112:0/3386898179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 msgr2=0x563029a364d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.008+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a364d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.008+0000 7f2cd9e00700 1 -- 172.21.15.112:0/3386898179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2dd40 msgr2=0x563029a38c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:05.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.008+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2dd40 0x563029a38c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.008+0000 7f2cd9e00700 1 -- 172.21.15.112:0/3386898179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630298ce9d0 con 0x563029a2e660 2024-04-25T18:49:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.008+0000 7f2ce2800700 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2dd40 0x563029a38c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:49:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.009+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2e660 0x563029a35f90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc09dfb0 tx=0x7f2ccc088230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6534badb3d06073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.009+0000 7f2cd0c00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:49:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.009+0000 7f2cd0c00700 1 -- 172.21.15.112:0/3386898179 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc021380 con 0x563029a2e660 2024-04-25T18:49:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.009+0000 7f2cd0c00700 1 -- 172.21.15.112:0/3386898179 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2ccc09ba00 con 0x563029a2e660 2024-04-25T18:49:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.009+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/3386898179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ccc09de00 con 0x563029a2e660 2024-04-25T18:49:05.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.009+0000 7f2cd0c00700 1 -- 172.21.15.112:0/3386898179 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2ccc09d700 con 0x563029a2e660 2024-04-25T18:49:05.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.009+0000 7f2cf3db9b80 10 client.15246 fetch_fsmap learned FSMap version 26 2024-04-25T18:49:05.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.009+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/3386898179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563029917bf0 con 0x563029a2e660 2024-04-25T18:49:05.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.009+0000 7f2cd0c00700 1 -- 172.21.15.112:0/3386898179 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ccc09de00 con 0x563029a2e660 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.010+0000 7f2cd0c00700 1 -- 172.21.15.112:0/3386898179 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2ccc0a5e50 con 0x563029a2e660 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.010+0000 7f2cf3db9b80 10 client.15246 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.010+0000 7f2cf3db9b80 10 client.15246 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.010+0000 7f2cf3db9b80 20 client.15246 populate_metadata read hostname 'smithi112' 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.010+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x563029a48800 0x563029a4abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.010+0000 7f2cf3db9b80 4 client.15246 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.010+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/3386898179 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56302998e780 con 0x563029a48800 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.010+0000 7f2ce2800700 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x563029a48800 0x563029a4abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.011+0000 7f2cd0c00700 10 client.15246 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.011+0000 7f2ce2800700 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x563029a48800 0x563029a4abf0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa0507a3b8963a8 server_cookie=c4f1627cf32fc9a3 in_seq=0 out_seq=0 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cd0c00700 1 -- 172.21.15.112:0/3386898179 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56302998e780 con 0x563029a48800 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cd0c00700 10 client.15246 handle_command_reply: tid=0 2024-04-25T18:49:05.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cf3db9b80 1 client.15246 shutdown 2024-04-25T18:49:05.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/3386898179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2e660 msgr2=0x563029a35f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:05.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2e660 0x563029a35f90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc09dfb0 tx=0x7f2ccc088230 comp rx=0 tx=0).stop 2024-04-25T18:49:05.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/3386898179 shutdown_connections 2024-04-25T18:49:05.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x563029a48800 0x563029a4abf0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2dd40 0x563029a38c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/3386898179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2e660 0x563029a35f90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cd0c00700 0 client.15246 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/3386898179 >> 172.21.15.112:0/3386898179 conn(0x56302995d050 msgr2=0x563029a29e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/3386898179 shutdown_connections 2024-04-25T18:49:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.012+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/3386898179 wait complete. 2024-04-25T18:49:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.013+0000 7f2cf3db9b80 20 client.15246 trim_cache size 0 max 16384 2024-04-25T18:49:05.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.034+0000 7f2cf3db9b80 1 Processor -- start 2024-04-25T18:49:05.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.034+0000 7f2cf3db9b80 1 -- start start 2024-04-25T18:49:05.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.034+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 0x563029a44b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.034+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a37ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.034+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a50270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.034+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce41b61b0 con 0x563029a2dd40 2024-04-25T18:49:05.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.034+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce41b3a40 con 0x563029a2e660 2024-04-25T18:49:05.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.034+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce41b1890 con 0x563029a394d0 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.034+0000 7f2ce2800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 0x563029a44b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.035+0000 7f2cd9e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a37ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.035+0000 7f2ce2800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 0x563029a44b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55842/0 (socket says 172.21.15.112:55842) 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.035+0000 7f2ce2800700 1 -- 172.21.15.112:0/443652534 learned_addr learned my addr 172.21.15.112:0/443652534 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.035+0000 7f2ce3200700 1 --2- 172.21.15.112:0/443652534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a50270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.035+0000 7f2ce2800700 1 -- 172.21.15.112:0/443652534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 msgr2=0x563029a50270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.035+0000 7f2ce2800700 1 --2- 172.21.15.112:0/443652534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a50270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.035+0000 7f2ce2800700 1 -- 172.21.15.112:0/443652534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 msgr2=0x563029a37ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.035+0000 7f2ce2800700 1 --2- 172.21.15.112:0/443652534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a37ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.035+0000 7f2ce2800700 1 -- 172.21.15.112:0/443652534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563029a2ece0 con 0x563029a2dd40 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.035+0000 7f2ce3200700 1 --2- 172.21.15.112:0/443652534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a50270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.036+0000 7f2ce2800700 1 --2- 172.21.15.112:0/443652534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 0x563029a44b40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009fd0 tx=0x7f2cd401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a73d6620c5578f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.036+0000 7f2cd3400700 1 -- 172.21.15.112:0/443652534 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4030070 con 0x563029a2dd40 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.036+0000 7f2cd3400700 1 -- 172.21.15.112:0/443652534 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd4019050 con 0x563029a2dd40 2024-04-25T18:49:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.036+0000 7f2cd3400700 1 -- 172.21.15.112:0/443652534 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4017040 con 0x563029a2dd40 2024-04-25T18:49:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.037+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/443652534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 msgr2=0x563029a44b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.037+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/443652534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 0x563029a44b40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009fd0 tx=0x7f2cd401c000 comp rx=0 tx=0).stop 2024-04-25T18:49:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.037+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/443652534 shutdown_connections 2024-04-25T18:49:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.037+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/443652534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a394d0 0x563029a50270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.037+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/443652534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a2dd40 0x563029a44b40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.037+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/443652534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a37ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.037+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/443652534 >> 172.21.15.112:0/443652534 conn(0x56302995d050 msgr2=0x563029a377c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.037+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/443652534 shutdown_connections 2024-04-25T18:49:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.037+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/443652534 wait complete. 2024-04-25T18:49:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.037+0000 7f2cf3db9b80 1 -- start start 2024-04-25T18:49:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.037+0000 7f2cf3db9b80 1 Processor -- start 2024-04-25T18:49:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a2dd40 0x563029a3e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a3e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2cf3db9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a394d0 0x563029a3eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce41b1f00 con 0x563029a394d0 2024-04-25T18:49:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce41b3370 con 0x563029a2e660 2024-04-25T18:49:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2cf3db9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce41b25d0 con 0x563029a2dd40 2024-04-25T18:49:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2cf3db9b80 10 client.0 _ll_register_callbacks cb 0x563029919820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:49:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2cd9e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a3e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2ce2800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a2dd40 0x563029a3e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2ce2800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a2dd40 0x563029a3e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:37004/0 (socket says 172.21.15.112:37004) 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2ce2800700 1 -- 172.21.15.112:0/2973196032 learned_addr learned my addr 172.21.15.112:0/2973196032 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.038+0000 7f2ce3200700 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a394d0 0x563029a3eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.039+0000 7f2ce2800700 1 -- 172.21.15.112:0/2973196032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 msgr2=0x563029a3e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.039+0000 7f2ce2800700 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a3e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.039+0000 7f2ce2800700 1 -- 172.21.15.112:0/2973196032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a394d0 msgr2=0x563029a3eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.039+0000 7f2ce2800700 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a394d0 0x563029a3eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.039+0000 7f2ce2800700 1 -- 172.21.15.112:0/2973196032 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563029a2ea80 con 0x563029a2dd40 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.039+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a3e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.039+0000 7f2ce2800700 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a2dd40 0x563029a3e020 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2cd402e040 tx=0x7f2cd40042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61bd14640218bd68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.039+0000 7f2cd0c00700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.039+0000 7f2cd0c00700 1 -- 172.21.15.112:0/2973196032 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4030030 con 0x563029a2dd40 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.039+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/2973196032 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd4019050 con 0x563029a2dd40 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.039+0000 7f2cd0c00700 1 -- 172.21.15.112:0/2973196032 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2cd40044f0 con 0x563029a2dd40 2024-04-25T18:49:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.040+0000 7f2cd0c00700 1 -- 172.21.15.112:0/2973196032 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2cd4017040 con 0x563029a2dd40 2024-04-25T18:49:05.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.040+0000 7f2cf3db9b80 10 client.25411 fetch_fsmap learned FSMap version 26 2024-04-25T18:49:05.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.040+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/2973196032 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563029aafea0 con 0x563029a2dd40 2024-04-25T18:49:05.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.040+0000 7f2cd0c00700 1 -- 172.21.15.112:0/2973196032 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd4019050 con 0x563029a2dd40 2024-04-25T18:49:05.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.040+0000 7f2cd0c00700 1 -- 172.21.15.112:0/2973196032 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2cd4005020 con 0x563029a2dd40 2024-04-25T18:49:05.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.040+0000 7f2cf3db9b80 10 client.25411 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:49:05.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.040+0000 7f2cf3db9b80 10 client.25411 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-25T18:49:05.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.040+0000 7f2cf3db9b80 20 client.25411 populate_metadata read hostname 'smithi112' 2024-04-25T18:49:05.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.040+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x563029ab0510 0x563029ab2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:05.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.040+0000 7f2cf3db9b80 4 client.25411 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b8363855", "release": true, "await": true}] 2024-04-25T18:49:05.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.040+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/2973196032 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b8363855", "release": true, "await": true}) -- 0x563029988820 con 0x563029ab0510 2024-04-25T18:49:05.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.041+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x563029ab0510 0x563029ab2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.041+0000 7f2cd9e00700 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x563029ab0510 0x563029ab2930 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ae3c6f234a6ab5e server_cookie=de897d1aaec66f90 in_seq=0 out_seq=0 2024-04-25T18:49:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.041+0000 7f2cd0c00700 10 client.25411 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:05.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cd0c00700 1 -- 172.21.15.112:0/2973196032 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x563029988820 con 0x563029ab0510 2024-04-25T18:49:05.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cd0c00700 10 client.25411 handle_command_reply: tid=0 2024-04-25T18:49:05.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 1 client.25411 shutdown 2024-04-25T18:49:05.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/2973196032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a2dd40 msgr2=0x563029a3e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:05.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a2dd40 0x563029a3e020 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2cd402e040 tx=0x7f2cd40042d0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/2973196032 shutdown_connections 2024-04-25T18:49:05.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x563029ab0510 0x563029ab2930 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563029a394d0 0x563029a3eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563029a2dd40 0x563029a3e020 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 1 --2- 172.21.15.112:0/2973196032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563029a2e660 0x563029a3e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cd0c00700 0 client.25411 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:05.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/2973196032 >> 172.21.15.112:0/2973196032 conn(0x56302995d050 msgr2=0x563029a29c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:05.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/2973196032 shutdown_connections 2024-04-25T18:49:05.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 1 -- 172.21.15.112:0/2973196032 wait complete. 2024-04-25T18:49:05.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.045+0000 7f2cf3db9b80 20 client.25411 trim_cache size 0 max 16384 2024-04-25T18:49:05.047 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:49:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 26, 2024-04-25T18:49:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "leader": 24512, 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "set_version": 4, 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "sets": { 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "b8363855": { 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "version": 4, 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "age_ref": 0.0, 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "state": { 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RELEASED", 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 0.0 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "timeout": 60.0, 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "expiration": 80.8, 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "members": { 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "file:/": { 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "excluded": false, 2024-04-25T18:49:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "state": { 2024-04-25T18:49:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RELEASED", 2024-04-25T18:49:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 0.0 2024-04-25T18:49:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:49:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:49:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:49:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:49:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:49:05.049 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:49:05.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 -- 172.21.15.112:0/3438517930 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2cc80a3db0 msgr2=0x7f2cc80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:05.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2cc80a3db0 0x7f2cc80a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009750 tx=0x7f2cd40119b0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 -- 172.21.15.112:0/3438517930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ce4075590 msgr2=0x7f2ce41a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:05.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ce4075590 0x7f2ce41a5b60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009230 tx=0x7f2ccc00fa70 comp rx=0 tx=0).stop 2024-04-25T18:49:05.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 -- 172.21.15.112:0/3438517930 shutdown_connections 2024-04-25T18:49:05.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2cc80a3db0 0x7f2cc80a6270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ce4113c70 0x7f2ce419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4074c70 0x7f2ce41a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 --2- 172.21.15.112:0/3438517930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ce4075590 0x7f2ce41a5b60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:05.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 -- 172.21.15.112:0/3438517930 >> 172.21.15.112:0/3438517930 conn(0x7f2ce4079b90 msgr2=0x7f2ce4110b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:05.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 -- 172.21.15.112:0/3438517930 shutdown_connections 2024-04-25T18:49:05.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:05.046+0000 7f2ce9400700 1 -- 172.21.15.112:0/3438517930 wait complete. 2024-04-25T18:49:05.058 INFO:tasks.workunit.client.0.smithi112.stderr:+ set -e 2024-04-25T18:49:05.058 INFO:tasks.workunit.client.0.smithi112.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-25T18:49:05.063 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into 'xfstests-dev'... 2024-04-25T18:49:06.008 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b8363855', seconds elapsed: 37.6 2024-04-25T18:49:06.008 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.8 2024-04-25T18:49:06.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:49:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:49:06.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:06 smithi161 ceph-mon[32609]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-25T18:49:06.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:06 smithi112 ceph-mon[27864]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-25T18:49:06.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:06 smithi171 ceph-mon[33266]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-25T18:49:07.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:07 smithi171 ceph-mon[33266]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 36 KiB/s wr, 24 op/s 2024-04-25T18:49:07.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:07 smithi161 ceph-mon[32609]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 36 KiB/s wr, 24 op/s 2024-04-25T18:49:07.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:07 smithi112 ceph-mon[27864]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 36 KiB/s wr, 24 op/s 2024-04-25T18:49:08.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.864+0000 7fc2bcc00700 1 -- 172.21.15.112:0/716247811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b810f440 msgr2=0x7fc2b8111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:08.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.864+0000 7fc2bcc00700 1 --2- 172.21.15.112:0/716247811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b810f440 0x7fc2b8111830 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000b0a0 tx=0x7fc2b000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:49:08.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.864+0000 7fc2bcc00700 1 -- 172.21.15.112:0/716247811 shutdown_connections 2024-04-25T18:49:08.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.864+0000 7fc2bcc00700 1 --2- 172.21.15.112:0/716247811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b810f440 0x7fc2b8111830 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:08.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.864+0000 7fc2bcc00700 1 --2- 172.21.15.112:0/716247811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b8074c70 0x7fc2b8075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:08.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.864+0000 7fc2bcc00700 1 --2- 172.21.15.112:0/716247811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2b8075590 0x7fc2b810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:08.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.864+0000 7fc2bcc00700 1 -- 172.21.15.112:0/716247811 >> 172.21.15.112:0/716247811 conn(0x7fc2b8079ba0 msgr2=0x7fc2b8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:08.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.865+0000 7fc2bcc00700 1 -- 172.21.15.112:0/716247811 shutdown_connections 2024-04-25T18:49:08.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.865+0000 7fc2bcc00700 1 -- 172.21.15.112:0/716247811 wait complete. 2024-04-25T18:49:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.866+0000 7fc2bcc00700 1 Processor -- start 2024-04-25T18:49:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.866+0000 7fc2bcc00700 1 -- start start 2024-04-25T18:49:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.866+0000 7fc2bcc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b8074c70 0x7fc2b819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.866+0000 7fc2bcc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2b8075590 0x7fc2b819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.866+0000 7fc2bcc00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b810f440 0x7fc2b819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.866+0000 7fc2bcc00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b8114e20 con 0x7fc2b810f440 2024-04-25T18:49:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.866+0000 7fc2bcc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b8114ca0 con 0x7fc2b8074c70 2024-04-25T18:49:08.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.866+0000 7fc2bcc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b8114fa0 con 0x7fc2b8075590 2024-04-25T18:49:08.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.866+0000 7fc2b5e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b8074c70 0x7fc2b819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.867+0000 7fc2b5e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b8074c70 0x7fc2b819b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:36762/0 (socket says 172.21.15.112:36762) 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.867+0000 7fc2b5e00700 1 -- 172.21.15.112:0/957968938 learned_addr learned my addr 172.21.15.112:0/957968938 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.867+0000 7fc2b5400700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2b8075590 0x7fc2b819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.867+0000 7fc2b6800700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b810f440 0x7fc2b819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.867+0000 7fc2b5e00700 1 -- 172.21.15.112:0/957968938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2b8075590 msgr2=0x7fc2b819b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.867+0000 7fc2b5e00700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2b8075590 0x7fc2b819b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.867+0000 7fc2b5e00700 1 -- 172.21.15.112:0/957968938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b810f440 msgr2=0x7fc2b819fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.867+0000 7fc2b5e00700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b810f440 0x7fc2b819fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.867+0000 7fc2b5e00700 1 -- 172.21.15.112:0/957968938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b81a0270 con 0x7fc2b8074c70 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.867+0000 7fc2b5e00700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b8074c70 0x7fc2b819b2b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800e9a0 tx=0x7fc2a800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.868+0000 7fc2aea00700 1 -- 172.21.15.112:0/957968938 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a800ca10 con 0x7fc2b8074c70 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.868+0000 7fc2aea00700 1 -- 172.21.15.112:0/957968938 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a800cbb0 con 0x7fc2b8074c70 2024-04-25T18:49:08.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.868+0000 7fc2aea00700 1 -- 172.21.15.112:0/957968938 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8018680 con 0x7fc2b8074c70 2024-04-25T18:49:08.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.868+0000 7fc2bcc00700 1 -- 172.21.15.112:0/957968938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b81a0560 con 0x7fc2b8074c70 2024-04-25T18:49:08.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.868+0000 7fc2bcc00700 1 -- 172.21.15.112:0/957968938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b81a7de0 con 0x7fc2b8074c70 2024-04-25T18:49:08.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.869+0000 7fc2bcc00700 1 -- 172.21.15.112:0/957968938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b8069730 con 0x7fc2b8074c70 2024-04-25T18:49:08.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.871+0000 7fc2aea00700 1 -- 172.21.15.112:0/957968938 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fc2a8010430 con 0x7fc2b8074c70 2024-04-25T18:49:08.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.872+0000 7fc2aea00700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc29c0a3e30 0x7fc29c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:08.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.872+0000 7fc2aea00700 1 -- 172.21.15.112:0/957968938 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fc2a80d09a0 con 0x7fc2b8074c70 2024-04-25T18:49:08.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.872+0000 7fc2b5400700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc29c0a3e30 0x7fc29c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:08.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.873+0000 7fc2b5400700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc29c0a3e30 0x7fc29c0a62f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8111100 tx=0x7fc2a4007400 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:08.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:08.876+0000 7fc2aea00700 1 -- 172.21.15.112:0/957968938 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2a809cfe0 con 0x7fc2b8074c70 2024-04-25T18:49:09.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.152+0000 7fc2bcc00700 1 -- 172.21.15.112:0/957968938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2b802df80 con 0x7fc2b8074c70 2024-04-25T18:49:09.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.153+0000 7fc2aea00700 1 -- 172.21.15.112:0/957968938 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11893 (secure 0 0 0) 0x7fc2a809c8e0 con 0x7fc2b8074c70 2024-04-25T18:49:09.155 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:49:09.155 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:47:25.174463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3241331981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3241331981},{"type":"v1","addr":"172.21.15.112:6839","nonce":3241331981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24493,24511,24512]},"id":1}]} 2024-04-25T18:49:09.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 -- 172.21.15.112:0/957968938 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc29c0a3e30 msgr2=0x7fc29c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:09.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc29c0a3e30 0x7fc29c0a62f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8111100 tx=0x7fc2a4007400 comp rx=0 tx=0).stop 2024-04-25T18:49:09.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 -- 172.21.15.112:0/957968938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b8074c70 msgr2=0x7fc2b819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:09.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b8074c70 0x7fc2b819b2b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800e9a0 tx=0x7fc2a800ecb0 comp rx=0 tx=0).stop 2024-04-25T18:49:09.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 -- 172.21.15.112:0/957968938 shutdown_connections 2024-04-25T18:49:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc29c0a3e30 0x7fc29c0a62f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2b810f440 0x7fc2b819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b8074c70 0x7fc2b819b2b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 --2- 172.21.15.112:0/957968938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2b8075590 0x7fc2b819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 -- 172.21.15.112:0/957968938 >> 172.21.15.112:0/957968938 conn(0x7fc2b8079ba0 msgr2=0x7fc2b80ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 -- 172.21.15.112:0/957968938 shutdown_connections 2024-04-25T18:49:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:09.157+0000 7fc2bcc00700 1 -- 172.21.15.112:0/957968938 wait complete. 2024-04-25T18:49:09.161 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-04-25T18:49:09.888 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:09 smithi112 ceph-mon[27864]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-25T18:49:09.888 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:09 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/957968938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:49:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:09 smithi171 ceph-mon[33266]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-25T18:49:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:09 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/957968938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:49:10.211 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24512 quiesce db / --timeout 60 --expiration 80.8 --await-for 42.0' 2024-04-25T18:49:10.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24512 quiesce db / --timeout 60 --expiration 80.8 --await-for 42.0 2024-04-25T18:49:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:09 smithi161 ceph-mon[32609]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-25T18:49:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:09 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/957968938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:49:10.648 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:49:12.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:11 smithi171 ceph-mon[33266]: pgmap v56: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.0 MiB/s wr, 24 op/s 2024-04-25T18:49:12.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:11 smithi161 ceph-mon[32609]: pgmap v56: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.0 MiB/s wr, 24 op/s 2024-04-25T18:49:12.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:11 smithi112 ceph-mon[27864]: pgmap v56: 97 pgs: 97 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.0 MiB/s wr, 24 op/s 2024-04-25T18:49:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.118+0000 7f7c19400700 1 -- 172.21.15.112:0/1555582483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c14074c70 msgr2=0x7f7c14075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.118+0000 7f7c19400700 1 --2- 172.21.15.112:0/1555582483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c14074c70 0x7f7c14075050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009990 tx=0x7f7c04009ca0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.118+0000 7f7c19400700 1 -- 172.21.15.112:0/1555582483 shutdown_connections 2024-04-25T18:49:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.118+0000 7f7c19400700 1 --2- 172.21.15.112:0/1555582483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c1410f440 0x7f7c14111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.118+0000 7f7c19400700 1 --2- 172.21.15.112:0/1555582483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c14074c70 0x7f7c14075050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.118+0000 7f7c19400700 1 --2- 172.21.15.112:0/1555582483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c14075590 0x7f7c1410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.118+0000 7f7c19400700 1 -- 172.21.15.112:0/1555582483 >> 172.21.15.112:0/1555582483 conn(0x7f7c14079bf0 msgr2=0x7f7c1407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.119+0000 7f7c19400700 1 -- 172.21.15.112:0/1555582483 shutdown_connections 2024-04-25T18:49:13.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.119+0000 7f7c19400700 1 -- 172.21.15.112:0/1555582483 wait complete. 2024-04-25T18:49:13.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.120+0000 7f7c19400700 1 Processor -- start 2024-04-25T18:49:13.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.120+0000 7f7c19400700 1 -- start start 2024-04-25T18:49:13.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.121+0000 7f7c19400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c14074c70 0x7f7c1419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.121+0000 7f7c19400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c14075590 0x7f7c1419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.121+0000 7f7c19400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c1410f440 0x7f7c1419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.121+0000 7f7c19400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14114e30 con 0x7f7c14075590 2024-04-25T18:49:13.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.121+0000 7f7c19400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14114cb0 con 0x7f7c14074c70 2024-04-25T18:49:13.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.121+0000 7f7c19400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14114fb0 con 0x7f7c1410f440 2024-04-25T18:49:13.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.121+0000 7f7c11e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c14075590 0x7f7c1419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.121+0000 7f7c12800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c14074c70 0x7f7c1419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.121+0000 7f7c11e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c14075590 0x7f7c1419b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37932/0 (socket says 172.21.15.112:37932) 2024-04-25T18:49:13.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.121+0000 7f7c11e00700 1 -- 172.21.15.112:0/1481098615 learned_addr learned my addr 172.21.15.112:0/1481098615 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:13.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.121+0000 7f7c13200700 1 --2- 172.21.15.112:0/1481098615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c1410f440 0x7f7c1419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.122+0000 7f7c12800700 1 -- 172.21.15.112:0/1481098615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c1410f440 msgr2=0x7f7c1419fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.122+0000 7f7c12800700 1 --2- 172.21.15.112:0/1481098615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c1410f440 0x7f7c1419fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.122+0000 7f7c12800700 1 -- 172.21.15.112:0/1481098615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c14075590 msgr2=0x7f7c1419b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.122+0000 7f7c12800700 1 --2- 172.21.15.112:0/1481098615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c14075590 0x7f7c1419b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.122+0000 7f7c12800700 1 -- 172.21.15.112:0/1481098615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c141a0270 con 0x7f7c14074c70 2024-04-25T18:49:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.122+0000 7f7c11e00700 1 --2- 172.21.15.112:0/1481098615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c14075590 0x7f7c1419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:49:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.122+0000 7f7c12800700 1 --2- 172.21.15.112:0/1481098615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c14074c70 0x7f7c1419b260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7c04006010 tx=0x7f7c040057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.122+0000 7f7c0b400700 1 -- 172.21.15.112:0/1481098615 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04027070 con 0x7f7c14074c70 2024-04-25T18:49:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.123+0000 7f7c19400700 1 -- 172.21.15.112:0/1481098615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c141a0500 con 0x7f7c14074c70 2024-04-25T18:49:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.123+0000 7f7c0b400700 1 -- 172.21.15.112:0/1481098615 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c04022470 con 0x7f7c14074c70 2024-04-25T18:49:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.123+0000 7f7c0b400700 1 -- 172.21.15.112:0/1481098615 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04017650 con 0x7f7c14074c70 2024-04-25T18:49:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.123+0000 7f7c19400700 1 -- 172.21.15.112:0/1481098615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c141a7de0 con 0x7f7c14074c70 2024-04-25T18:49:13.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.126+0000 7f7c0b400700 1 -- 172.21.15.112:0/1481098615 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f7c0402a070 con 0x7f7c14074c70 2024-04-25T18:49:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.127+0000 7f7c0b400700 1 --2- 172.21.15.112:0/1481098615 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f7bf80a3e50 0x7f7bf80a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.127+0000 7f7c23cdcb80 1 Processor -- start 2024-04-25T18:49:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.127+0000 7f7c0b400700 1 -- 172.21.15.112:0/1481098615 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f7c040d13c0 con 0x7f7c14074c70 2024-04-25T18:49:13.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.128+0000 7f7c23cdcb80 1 -- start start 2024-04-25T18:49:13.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.128+0000 7f7c11e00700 1 --2- 172.21.15.112:0/1481098615 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f7bf80a3e50 0x7f7bf80a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.128+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a5761b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.128+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 0x55c71a581020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.128+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a583950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.128+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c71a4c9020 con 0x55c71a5766f0 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.128+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c71a46f8c0 con 0x55c71a581560 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.128+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c71a4c4fe0 con 0x55c71a575dd0 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.128+0000 7f7c12800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a5761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.128+0000 7f7c12800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a5761b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:38372/0 (socket says 172.21.15.112:38372) 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7c12800700 1 -- 172.21.15.112:0/3564769050 learned_addr learned my addr 172.21.15.112:0/3564769050 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7c11e00700 1 --2- 172.21.15.112:0/3564769050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 0x55c71a581020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7c13200700 1 --2- 172.21.15.112:0/3564769050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a583950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7c12800700 1 -- 172.21.15.112:0/3564769050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 msgr2=0x55c71a583950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7c12800700 1 --2- 172.21.15.112:0/3564769050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a583950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7c12800700 1 -- 172.21.15.112:0/3564769050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 msgr2=0x55c71a581020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7c12800700 1 --2- 172.21.15.112:0/3564769050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 0x55c71a581020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7c11e00700 1 --2- 172.21.15.112:0/1481098615 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f7bf80a3e50 0x7f7bf80a6310 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7c14111070 tx=0x7f7c00011950 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7c12800700 1 -- 172.21.15.112:0/3564769050 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c71a465810 con 0x55c71a575dd0 2024-04-25T18:49:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7c13200700 1 --2- 172.21.15.112:0/3564769050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a583950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:49:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7c12800700 1 --2- 172.21.15.112:0/3564769050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a5761b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7c040a5a40 tx=0x7f7c04099760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62a1950e31fced76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.129+0000 7f7bffe00700 1 -- 172.21.15.112:0/3564769050 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0409ccf0 con 0x55c71a575dd0 2024-04-25T18:49:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.130+0000 7f7bffe00700 1 -- 172.21.15.112:0/3564769050 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c0409da90 con 0x55c71a575dd0 2024-04-25T18:49:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.130+0000 7f7bffe00700 1 -- 172.21.15.112:0/3564769050 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04099b60 con 0x55c71a575dd0 2024-04-25T18:49:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.130+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3564769050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 msgr2=0x55c71a5761b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.130+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3564769050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a5761b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7c040a5a40 tx=0x7f7c04099760 comp rx=0 tx=0).stop 2024-04-25T18:49:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.130+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3564769050 shutdown_connections 2024-04-25T18:49:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.130+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3564769050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a583950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.130+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3564769050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a5761b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.130+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3564769050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 0x55c71a581020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.130+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3564769050 >> 172.21.15.112:0/3564769050 conn(0x55c71a4a50e0 msgr2=0x55c71a5722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:13.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.131+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3564769050 shutdown_connections 2024-04-25T18:49:13.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.131+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3564769050 wait complete. 2024-04-25T18:49:13.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.131+0000 7f7c23cdcb80 1 -- start start 2024-04-25T18:49:13.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.131+0000 7f7c23cdcb80 1 Processor -- start 2024-04-25T18:49:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a580ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a5766f0 0x55c71a57e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a581560 0x55c71a57e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c71a583e90 con 0x55c71a581560 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c71a4829f0 con 0x55c71a5766f0 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c71a584010 con 0x55c71a575dd0 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c23cdcb80 10 client.0 _ll_register_callbacks cb 0x55c71a4618b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c11e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a5766f0 0x55c71a57e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c12800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a580ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c13200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a581560 0x55c71a57e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c11e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a5766f0 0x55c71a57e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:52400/0 (socket says 172.21.15.112:52400) 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.132+0000 7f7c11e00700 1 -- 172.21.15.112:0/3336569447 learned_addr learned my addr 172.21.15.112:0/3336569447 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.133+0000 7f7c11e00700 1 -- 172.21.15.112:0/3336569447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 msgr2=0x55c71a580ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.133+0000 7f7c11e00700 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a580ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.133+0000 7f7c11e00700 1 -- 172.21.15.112:0/3336569447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a581560 msgr2=0x55c71a57e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.133+0000 7f7c11e00700 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a581560 0x55c71a57e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.133+0000 7f7c11e00700 1 -- 172.21.15.112:0/3336569447 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c71a416a60 con 0x55c71a5766f0 2024-04-25T18:49:13.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.133+0000 7f7c12800700 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a580ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.133+0000 7f7c11e00700 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a5766f0 0x55c71a57e010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7c0001ea70 tx=0x7f7c000047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1c0b1d4dcb5d808 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.133+0000 7f7bfcc00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.133+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3336569447 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00017040 con 0x55c71a5766f0 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.133+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3336569447 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c0409da90 con 0x55c71a5766f0 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.133+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3336569447 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f7c0000fd50 con 0x55c71a5766f0 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.134+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3336569447 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7c00004830 con 0x55c71a5766f0 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.134+0000 7f7c23cdcb80 10 client.25412 fetch_fsmap learned FSMap version 26 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.134+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3336569447 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c71a45fc80 con 0x55c71a5766f0 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.134+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3336569447 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c0409da90 con 0x55c71a5766f0 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.134+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3336569447 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7c00025030 con 0x55c71a5766f0 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.134+0000 7f7c23cdcb80 10 client.25412 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.134+0000 7f7c23cdcb80 10 client.25412 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.134+0000 7f7c23cdcb80 20 client.25412 populate_metadata read hostname 'smithi112' 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.134+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55c71a590b80 0x55c71a592f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.134+0000 7f7c23cdcb80 4 client.25412 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:49:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.134+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3336569447 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c71a4d6810 con 0x55c71a590b80 2024-04-25T18:49:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.135+0000 7f7c12800700 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55c71a590b80 0x55c71a592f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.135+0000 7f7c12800700 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55c71a590b80 0x55c71a592f90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f04707c6cd86dd20 server_cookie=9ca3030731ae22e2 in_seq=0 out_seq=0 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.135+0000 7f7bfcc00700 10 client.25412 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.136+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3336569447 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c71a4d6810 con 0x55c71a590b80 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.136+0000 7f7bfcc00700 10 client.25412 handle_command_reply: tid=0 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.136+0000 7f7c23cdcb80 1 client.25412 shutdown 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.136+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3336569447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a5766f0 msgr2=0x55c71a57e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.136+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a5766f0 0x55c71a57e010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7c0001ea70 tx=0x7f7c000047c0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.137+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3336569447 shutdown_connections 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.137+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55c71a590b80 0x55c71a592f90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.137+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a581560 0x55c71a57e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.137+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a580ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.137+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3336569447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a5766f0 0x55c71a57e010 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.137+0000 7f7bfcc00700 0 client.25412 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.137+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3336569447 >> 172.21.15.112:0/3336569447 conn(0x55c71a4a50e0 msgr2=0x55c71a571ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.137+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3336569447 shutdown_connections 2024-04-25T18:49:13.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.137+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3336569447 wait complete. 2024-04-25T18:49:13.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.137+0000 7f7c23cdcb80 20 client.25412 trim_cache size 0 max 16384 2024-04-25T18:49:13.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.158+0000 7f7c23cdcb80 1 Processor -- start 2024-04-25T18:49:13.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.158+0000 7f7c23cdcb80 1 -- start start 2024-04-25T18:49:13.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.158+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a575dd0 0x55c71a58ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.158+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a5766f0 0x55c71a571ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a5720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141b1d40 con 0x55c71a575dd0 2024-04-25T18:49:13.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141af5d0 con 0x55c71a581560 2024-04-25T18:49:13.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141ad420 con 0x55c71a5766f0 2024-04-25T18:49:13.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c12800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a575dd0 0x55c71a58ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c12800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a575dd0 0x55c71a58ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37960/0 (socket says 172.21.15.112:37960) 2024-04-25T18:49:13.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c12800700 1 -- 172.21.15.112:0/1631492463 learned_addr learned my addr 172.21.15.112:0/1631492463 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:13.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c11e00700 1 --2- 172.21.15.112:0/1631492463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a5766f0 0x55c71a571ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c13200700 1 --2- 172.21.15.112:0/1631492463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a5720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c12800700 1 -- 172.21.15.112:0/1631492463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a5766f0 msgr2=0x55c71a571ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c12800700 1 --2- 172.21.15.112:0/1631492463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a5766f0 0x55c71a571ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c12800700 1 -- 172.21.15.112:0/1631492463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 msgr2=0x55c71a5720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c12800700 1 --2- 172.21.15.112:0/1631492463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a5720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c12800700 1 -- 172.21.15.112:0/1631492463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c71a57ef30 con 0x55c71a575dd0 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c11e00700 1 --2- 172.21.15.112:0/1631492463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a5766f0 0x55c71a571ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.159+0000 7f7c13200700 1 --2- 172.21.15.112:0/1631492463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a5720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.160+0000 7f7c12800700 1 --2- 172.21.15.112:0/1631492463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a575dd0 0x55c71a58ccc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c0409d320 tx=0x7f7c04005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cc1875fd863bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.160+0000 7f7bff400700 1 -- 172.21.15.112:0/1631492463 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0409da90 con 0x55c71a575dd0 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.160+0000 7f7bff400700 1 -- 172.21.15.112:0/1631492463 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c040a1740 con 0x55c71a575dd0 2024-04-25T18:49:13.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.160+0000 7f7bff400700 1 -- 172.21.15.112:0/1631492463 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0409d850 con 0x55c71a575dd0 2024-04-25T18:49:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.161+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/1631492463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a575dd0 msgr2=0x55c71a58ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.161+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/1631492463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a575dd0 0x55c71a58ccc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c0409d320 tx=0x7f7c04005040 comp rx=0 tx=0).stop 2024-04-25T18:49:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.161+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/1631492463 shutdown_connections 2024-04-25T18:49:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.161+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/1631492463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a5720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.161+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/1631492463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a575dd0 0x55c71a58ccc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.161+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/1631492463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a5766f0 0x55c71a571ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.161+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/1631492463 >> 172.21.15.112:0/1631492463 conn(0x55c71a4a50e0 msgr2=0x55c71a571490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.161+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/1631492463 shutdown_connections 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.161+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/1631492463 wait complete. 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.161+0000 7f7c23cdcb80 1 -- start start 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.161+0000 7f7c23cdcb80 1 Processor -- start 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a585ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 0x55c71a586420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c23cdcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a586960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141ada90 con 0x55c71a5766f0 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141aef00 con 0x55c71a581560 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c23cdcb80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141ae160 con 0x55c71a575dd0 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c23cdcb80 10 client.0 _ll_register_callbacks cb 0x55c71a4618b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c11e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 0x55c71a586420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c11e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 0x55c71a586420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37974/0 (socket says 172.21.15.112:37974) 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c11e00700 1 -- 172.21.15.112:0/3404639508 learned_addr learned my addr 172.21.15.112:0/3404639508 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c12800700 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a585ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.162+0000 7f7c13200700 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a586960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.163+0000 7f7c11e00700 1 -- 172.21.15.112:0/3404639508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 msgr2=0x55c71a585ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.163+0000 7f7c11e00700 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a585ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.163+0000 7f7c11e00700 1 -- 172.21.15.112:0/3404639508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 msgr2=0x55c71a586960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.163+0000 7f7c11e00700 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a586960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.163+0000 7f7c11e00700 1 -- 172.21.15.112:0/3404639508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c71a5ffb90 con 0x55c71a5766f0 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.163+0000 7f7c12800700 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a585ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.163+0000 7f7c11e00700 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 0x55c71a586420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c00026440 tx=0x7f7c0000fd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b257691a48270b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.163+0000 7f7bfcc00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.163+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3404639508 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00015070 con 0x55c71a5766f0 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.163+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3404639508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c040a1740 con 0x55c71a5766f0 2024-04-25T18:49:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.163+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3404639508 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f7c0002d360 con 0x55c71a5766f0 2024-04-25T18:49:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.164+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3404639508 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7c0003f050 con 0x55c71a5766f0 2024-04-25T18:49:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.164+0000 7f7c23cdcb80 10 client.15276 fetch_fsmap learned FSMap version 26 2024-04-25T18:49:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.164+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3404639508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c71a600ea0 con 0x55c71a5766f0 2024-04-25T18:49:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.164+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3404639508 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c040a1740 con 0x55c71a5766f0 2024-04-25T18:49:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.164+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3404639508 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7c00037020 con 0x55c71a5766f0 2024-04-25T18:49:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.164+0000 7f7c23cdcb80 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:49:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.164+0000 7f7c23cdcb80 10 client.15276 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-25T18:49:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.164+0000 7f7c23cdcb80 20 client.15276 populate_metadata read hostname 'smithi112' 2024-04-25T18:49:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.164+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55c71a6016a0 0x55c71a603ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.164+0000 7f7c23cdcb80 4 client.15276 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.8, "await_for": 42.0}] 2024-04-25T18:49:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.164+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3404639508 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.8, "await_for": 42.0}) -- 0x55c71a4d08b0 con 0x55c71a6016a0 2024-04-25T18:49:13.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.165+0000 7f7c12800700 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55c71a6016a0 0x55c71a603ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:13.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.165+0000 7f7c12800700 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55c71a6016a0 0x55c71a603ac0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6f564f0d599fe0f server_cookie=2682d92a96f43096 in_seq=0 out_seq=0 2024-04-25T18:49:13.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:13.165+0000 7f7bfcc00700 10 client.15276 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:14.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:13 smithi171 ceph-mon[33266]: pgmap v57: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2024-04-25T18:49:14.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:13 smithi161 ceph-mon[32609]: pgmap v57: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2024-04-25T18:49:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:13 smithi112 ceph-mon[27864]: pgmap v57: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2024-04-25T18:49:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.566+0000 7f1cfc800640 1 -- 172.21.15.112:0/1856609687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cf4103050 msgr2=0x7f1cf4100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.566+0000 7f1cfc800640 1 --2- 172.21.15.112:0/1856609687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cf4103050 0x7f1cf4100450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009900 tx=0x7f1ce802a410 comp rx=0 tx=0).stop 2024-04-25T18:49:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.567+0000 7f1cfc800640 1 -- 172.21.15.112:0/1856609687 shutdown_connections 2024-04-25T18:49:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.567+0000 7f1cfc800640 1 --2- 172.21.15.112:0/1856609687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cf410ebd0 0x7f1cf4110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.567+0000 7f1cfc800640 1 --2- 172.21.15.112:0/1856609687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf4100990 0x7f1cf4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.567+0000 7f1cfc800640 1 --2- 172.21.15.112:0/1856609687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cf4103050 0x7f1cf4100450 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.567+0000 7f1cfc800640 1 -- 172.21.15.112:0/1856609687 >> 172.21.15.112:0/1856609687 conn(0x7f1cf40773d0 msgr2=0x7f1cf40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.567+0000 7f1cfc800640 1 -- 172.21.15.112:0/1856609687 shutdown_connections 2024-04-25T18:49:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.567+0000 7f1cfc800640 1 -- 172.21.15.112:0/1856609687 wait complete. 2024-04-25T18:49:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.568+0000 7f1cfc800640 1 Processor -- start 2024-04-25T18:49:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.568+0000 7f1cfc800640 1 -- start start 2024-04-25T18:49:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.568+0000 7f1cfc800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cf4100990 0x7f1cf41a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.568+0000 7f1cfc800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cf4103050 0x7f1cf41a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.568+0000 7f1cfc800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf410ebd0 0x7f1cf419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.568+0000 7f1cfc800640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41145d0 con 0x7f1cf4103050 2024-04-25T18:49:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.568+0000 7f1cfc800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4114450 con 0x7f1cf410ebd0 2024-04-25T18:49:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.568+0000 7f1cfc800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4114750 con 0x7f1cf4100990 2024-04-25T18:49:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.568+0000 7f1cf9400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cf4103050 0x7f1cf41a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.568+0000 7f1cf9e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cf4100990 0x7f1cf41a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.569+0000 7f1cf9e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cf4100990 0x7f1cf41a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:38412/0 (socket says 172.21.15.112:38412) 2024-04-25T18:49:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.569+0000 7f1cf9e00640 1 -- 172.21.15.112:0/3085764058 learned_addr learned my addr 172.21.15.112:0/3085764058 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.569+0000 7f1cfa800640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf410ebd0 0x7f1cf419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.569+0000 7f1cfa800640 1 -- 172.21.15.112:0/3085764058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cf4100990 msgr2=0x7f1cf41a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.569+0000 7f1cfa800640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cf4100990 0x7f1cf41a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.569+0000 7f1cfa800640 1 -- 172.21.15.112:0/3085764058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cf4103050 msgr2=0x7f1cf41a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.569+0000 7f1cfa800640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cf4103050 0x7f1cf41a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.569+0000 7f1cfa800640 1 -- 172.21.15.112:0/3085764058 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf419b030 con 0x7f1cf410ebd0 2024-04-25T18:49:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.569+0000 7f1cf9400640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cf4103050 0x7f1cf41a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:49:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.569+0000 7f1cfa800640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf410ebd0 0x7f1cf419a8b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009fd0 tx=0x7f1cf000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.570+0000 7f1ceea00640 1 -- 172.21.15.112:0/3085764058 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000ce20 con 0x7f1cf410ebd0 2024-04-25T18:49:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.570+0000 7f1cfc800640 1 -- 172.21.15.112:0/3085764058 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf419b320 con 0x7f1cf410ebd0 2024-04-25T18:49:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.570+0000 7f1ceea00640 1 -- 172.21.15.112:0/3085764058 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf0005020 con 0x7f1cf410ebd0 2024-04-25T18:49:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.570+0000 7f1ceea00640 1 -- 172.21.15.112:0/3085764058 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0010690 con 0x7f1cf410ebd0 2024-04-25T18:49:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.570+0000 7f1cfc800640 1 -- 172.21.15.112:0/3085764058 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf41a75c0 con 0x7f1cf410ebd0 2024-04-25T18:49:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.573+0000 7f1cff61e740 1 Processor -- start 2024-04-25T18:49:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.573+0000 7f1ceea00640 1 -- 172.21.15.112:0/3085764058 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f1cf00054d0 con 0x7f1cf410ebd0 2024-04-25T18:49:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.573+0000 7f1cff61e740 1 -- start start 2024-04-25T18:49:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4e820 0x560cd4c4ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 0x560cd4c5bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cff61e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560cd4bb4900 con 0x560cd4c4f140 2024-04-25T18:49:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cff61e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560cd4b5aef0 con 0x560cd4c4e820 2024-04-25T18:49:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cff61e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560cd4bb08c0 con 0x560cd4c59a00 2024-04-25T18:49:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cf9e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cf9e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38000/0 (socket says 172.21.15.112:38000) 2024-04-25T18:49:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cf9e00640 1 -- 172.21.15.112:0/1418218752 learned_addr learned my addr 172.21.15.112:0/1418218752 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cf9400640 1 --2- 172.21.15.112:0/1418218752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4e820 0x560cd4c4ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cf9400640 1 --2- 172.21.15.112:0/1418218752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 0x560cd4c5bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cf9e00640 1 -- 172.21.15.112:0/1418218752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 msgr2=0x560cd4c5bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cf9e00640 1 --2- 172.21.15.112:0/1418218752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 0x560cd4c5bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cf9e00640 1 -- 172.21.15.112:0/1418218752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4e820 msgr2=0x560cd4c4ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cf9e00640 1 --2- 172.21.15.112:0/1418218752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4e820 0x560cd4c4ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cf9e00640 1 -- 172.21.15.112:0/1418218752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cd48c7210 con 0x560cd4c4f140 2024-04-25T18:49:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.574+0000 7f1cf9400640 1 --2- 172.21.15.112:0/1418218752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 0x560cd4c5bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:49:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.575+0000 7f1cf9e00640 1 --2- 172.21.15.112:0/1418218752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c594c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009a20 tx=0x7f1ce8032b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc0dc295e6a4950 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.575+0000 7f1ceea00640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f1cdc0a3dc0 0x7f1cdc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.575+0000 7f1ce3000640 1 -- 172.21.15.112:0/1418218752 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8040070 con 0x560cd4c4f140 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.575+0000 7f1ce3000640 1 -- 172.21.15.112:0/1418218752 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce8004030 con 0x560cd4c4f140 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.575+0000 7f1ce3000640 1 -- 172.21.15.112:0/1418218752 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8035040 con 0x560cd4c4f140 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.575+0000 7f1ceea00640 1 -- 172.21.15.112:0/3085764058 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f1cf00d0e90 con 0x7f1cf410ebd0 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.575+0000 7f1cf9400640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f1cdc0a3dc0 0x7f1cdc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.576+0000 7f1cff61e740 1 -- 172.21.15.112:0/1418218752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 msgr2=0x560cd4c594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.576+0000 7f1cff61e740 1 --2- 172.21.15.112:0/1418218752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c594c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009a20 tx=0x7f1ce8032b30 comp rx=0 tx=0).stop 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.576+0000 7f1cf9400640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f1cdc0a3dc0 0x7f1cdc0a6280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x560cd4c4c150 tx=0x7f1ce4004730 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.576+0000 7f1cff61e740 1 -- 172.21.15.112:0/1418218752 shutdown_connections 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.576+0000 7f1cff61e740 1 --2- 172.21.15.112:0/1418218752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 0x560cd4c5bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.576+0000 7f1cff61e740 1 --2- 172.21.15.112:0/1418218752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c594c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.576+0000 7f1cff61e740 1 --2- 172.21.15.112:0/1418218752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4e820 0x560cd4c4ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.576+0000 7f1cff61e740 1 -- 172.21.15.112:0/1418218752 >> 172.21.15.112:0/1418218752 conn(0x560cd4c4ad30 msgr2=0x560cd4c4b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.576+0000 7f1cff61e740 1 -- 172.21.15.112:0/1418218752 shutdown_connections 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.576+0000 7f1cff61e740 1 -- 172.21.15.112:0/1418218752 wait complete. 2024-04-25T18:49:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.577+0000 7f1cff61e740 1 -- start start 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.577+0000 7f1cff61e740 1 Processor -- start 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.577+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c4e820 0x560cd4c56880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.577+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4f140 0x560cd4c56dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.577+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c59a00 0x560cd4c57300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.577+0000 7f1cff61e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560cd4c5c5d0 con 0x560cd4c59a00 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.577+0000 7f1cff61e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560cd4b6e310 con 0x560cd4c4f140 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.577+0000 7f1cff61e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560cd4c5c750 con 0x560cd4c4e820 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.577+0000 7f1cff61e740 10 client.0 _ll_register_callbacks cb 0x560cd4b3ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cf9e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c4e820 0x560cd4c56880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cfa800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c59a00 0x560cd4c57300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cf9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4f140 0x560cd4c56dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cfa800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c59a00 0x560cd4c57300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38012/0 (socket says 172.21.15.112:38012) 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cf9e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c4e820 0x560cd4c56880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:38434/0 (socket says 172.21.15.112:38434) 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cf9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4f140 0x560cd4c56dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:52478/0 (socket says 172.21.15.112:52478) 2024-04-25T18:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cfa800640 1 -- 172.21.15.112:0/2758329941 learned_addr learned my addr 172.21.15.112:0/2758329941 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cfa800640 1 -- 172.21.15.112:0/2758329941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c4e820 msgr2=0x560cd4c56880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cfa800640 1 --2- 172.21.15.112:0/2758329941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c4e820 0x560cd4c56880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cfa800640 1 -- 172.21.15.112:0/2758329941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4f140 msgr2=0x560cd4c56dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cfa800640 1 --2- 172.21.15.112:0/2758329941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4f140 0x560cd4c56dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cfa800640 1 -- 172.21.15.112:0/2758329941 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cd4993620 con 0x560cd4c59a00 2024-04-25T18:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cfa800640 1 --2- 172.21.15.112:0/2758329941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c59a00 0x560cd4c57300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000f80 tx=0x7f1cf009d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f3b41d5b9febba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.578+0000 7f1cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.579+0000 7f1cd7400640 1 -- 172.21.15.112:0/2758329941 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00a5a40 con 0x560cd4c59a00 2024-04-25T18:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.579+0000 7f1cd7400640 1 -- 172.21.15.112:0/2758329941 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f1cf00a55b0 con 0x560cd4c59a00 2024-04-25T18:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.579+0000 7f1cff61e740 1 -- 172.21.15.112:0/2758329941 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce8004030 con 0x560cd4c59a00 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.579+0000 7f1cd7400640 1 -- 172.21.15.112:0/2758329941 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1cf00a57b0 con 0x560cd4c59a00 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.579+0000 7f1cff61e740 10 client.15288 fetch_fsmap learned FSMap version 26 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.579+0000 7f1cff61e740 1 -- 172.21.15.112:0/2758329941 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560cd4c687f0 con 0x560cd4c59a00 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.579+0000 7f1cd7400640 1 -- 172.21.15.112:0/2758329941 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce8004030 con 0x560cd4c59a00 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.579+0000 7f1cd7400640 1 -- 172.21.15.112:0/2758329941 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1cf009bc90 con 0x560cd4c59a00 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.579+0000 7f1cff61e740 10 client.15288 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.579+0000 7f1cff61e740 10 client.15288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.580+0000 7f1cff61e740 20 client.15288 populate_metadata read hostname 'smithi112' 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.580+0000 7f1cff61e740 1 --2- 172.21.15.112:0/2758329941 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560cd4c69d00 0x560cd4c6c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.580+0000 7f1cff61e740 4 client.15288 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.580+0000 7f1cff61e740 1 -- 172.21.15.112:0/2758329941 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cd4bc88b0 con 0x560cd4c69d00 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.580+0000 7f1cf9e00640 1 --2- 172.21.15.112:0/2758329941 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560cd4c69d00 0x560cd4c6c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.581+0000 7f1cf9e00640 1 --2- 172.21.15.112:0/2758329941 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560cd4c69d00 0x560cd4c6c110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a255dd58dd3b976 server_cookie=c5298019ba8f6e99 in_seq=0 out_seq=0 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.581+0000 7f1cd7400640 10 client.15288 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.581+0000 7f1cd7400640 1 -- 172.21.15.112:0/2758329941 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560cd4bc88b0 con 0x560cd4c69d00 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.581+0000 7f1cd7400640 10 client.15288 handle_command_reply: tid=0 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.581+0000 7f1cff61e740 1 client.15288 shutdown 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cff61e740 1 -- 172.21.15.112:0/2758329941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c59a00 msgr2=0x560cd4c57300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cff61e740 1 --2- 172.21.15.112:0/2758329941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c59a00 0x560cd4c57300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000f80 tx=0x7f1cf009d610 comp rx=0 tx=0).stop 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cff61e740 1 -- 172.21.15.112:0/2758329941 shutdown_connections 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cff61e740 1 --2- 172.21.15.112:0/2758329941 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560cd4c69d00 0x560cd4c6c110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cff61e740 1 --2- 172.21.15.112:0/2758329941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c59a00 0x560cd4c57300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cd7400640 0 client.15288 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cff61e740 1 --2- 172.21.15.112:0/2758329941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4f140 0x560cd4c56dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cff61e740 1 --2- 172.21.15.112:0/2758329941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c4e820 0x560cd4c56880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cff61e740 1 -- 172.21.15.112:0/2758329941 >> 172.21.15.112:0/2758329941 conn(0x560cd4c4ad30 msgr2=0x560cd4c50ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cff61e740 1 -- 172.21.15.112:0/2758329941 shutdown_connections 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cff61e740 1 -- 172.21.15.112:0/2758329941 wait complete. 2024-04-25T18:49:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.582+0000 7f1cff61e740 20 client.15288 trim_cache size 0 max 16384 2024-04-25T18:49:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.599+0000 7f1cff61e740 1 Processor -- start 2024-04-25T18:49:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.599+0000 7f1cff61e740 1 -- start start 2024-04-25T18:49:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.599+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4e820 0x560cd4c654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.599+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4f140 0x560cd4c559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.599+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 0x560cd4c49d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.599+0000 7f1cff61e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41b2610 con 0x560cd4c4e820 2024-04-25T18:49:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.599+0000 7f1cff61e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41afee0 con 0x560cd4c4f140 2024-04-25T18:49:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.599+0000 7f1cff61e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41add30 con 0x560cd4c59a00 2024-04-25T18:49:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.600+0000 7f1cfa800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 0x560cd4c49d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.600+0000 7f1cf9e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4e820 0x560cd4c654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.600+0000 7f1cfa800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 0x560cd4c49d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:38442/0 (socket says 172.21.15.112:38442) 2024-04-25T18:49:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.600+0000 7f1cfa800640 1 -- 172.21.15.112:0/451335246 learned_addr learned my addr 172.21.15.112:0/451335246 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.600+0000 7f1cf9400640 1 --2- 172.21.15.112:0/451335246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4f140 0x560cd4c559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.600+0000 7f1cf9e00640 1 -- 172.21.15.112:0/451335246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 msgr2=0x560cd4c49d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.600+0000 7f1cf9e00640 1 --2- 172.21.15.112:0/451335246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 0x560cd4c49d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.600+0000 7f1cf9e00640 1 -- 172.21.15.112:0/451335246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4f140 msgr2=0x560cd4c559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.600+0000 7f1cf9e00640 1 --2- 172.21.15.112:0/451335246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4f140 0x560cd4c559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.600+0000 7f1cf9e00640 1 -- 172.21.15.112:0/451335246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cd4c54dd0 con 0x560cd4c4e820 2024-04-25T18:49:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.600+0000 7f1cf9e00640 1 --2- 172.21.15.112:0/451335246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4e820 0x560cd4c654c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8000c00 tx=0x7f1ce8005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71adef44dde280 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.601+0000 7f1ce2600640 1 -- 172.21.15.112:0/451335246 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8040040 con 0x560cd4c4e820 2024-04-25T18:49:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.601+0000 7f1ce2600640 1 -- 172.21.15.112:0/451335246 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce8005040 con 0x560cd4c4e820 2024-04-25T18:49:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.601+0000 7f1ce2600640 1 -- 172.21.15.112:0/451335246 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8035040 con 0x560cd4c4e820 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.602+0000 7f1cff61e740 1 -- 172.21.15.112:0/451335246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4e820 msgr2=0x560cd4c654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.602+0000 7f1cff61e740 1 --2- 172.21.15.112:0/451335246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4e820 0x560cd4c654c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8000c00 tx=0x7f1ce8005870 comp rx=0 tx=0).stop 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.602+0000 7f1cff61e740 1 -- 172.21.15.112:0/451335246 shutdown_connections 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.602+0000 7f1cff61e740 1 --2- 172.21.15.112:0/451335246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c59a00 0x560cd4c49d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.602+0000 7f1cff61e740 1 --2- 172.21.15.112:0/451335246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c4f140 0x560cd4c559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.602+0000 7f1cff61e740 1 --2- 172.21.15.112:0/451335246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4e820 0x560cd4c654c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.602+0000 7f1cff61e740 1 -- 172.21.15.112:0/451335246 >> 172.21.15.112:0/451335246 conn(0x560cd4c4ad30 msgr2=0x560cd4c49600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.602+0000 7f1cff61e740 1 -- 172.21.15.112:0/451335246 shutdown_connections 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.602+0000 7f1cff61e740 1 -- 172.21.15.112:0/451335246 wait complete. 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.602+0000 7f1cff61e740 1 -- start start 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.602+0000 7f1cff61e740 1 Processor -- start 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c4e820 0x560cd4c61bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c5cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cff61e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c59a00 0x560cd4c5d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cff61e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41ae3a0 con 0x560cd4c4f140 2024-04-25T18:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cff61e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41af810 con 0x560cd4c59a00 2024-04-25T18:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cff61e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41aea70 con 0x560cd4c4e820 2024-04-25T18:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cff61e740 10 client.0 _ll_register_callbacks cb 0x7f1cf009c730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cf9400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c5cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cf9400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c5cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38040/0 (socket says 172.21.15.112:38040) 2024-04-25T18:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cf9400640 1 -- 172.21.15.112:0/1369503483 learned_addr learned my addr 172.21.15.112:0/1369503483 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cf9e00640 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c4e820 0x560cd4c61bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.603+0000 7f1cfa800640 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c59a00 0x560cd4c5d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.604+0000 7f1cf9400640 1 -- 172.21.15.112:0/1369503483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c4e820 msgr2=0x560cd4c61bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.604+0000 7f1cf9400640 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c4e820 0x560cd4c61bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.604+0000 7f1cf9400640 1 -- 172.21.15.112:0/1369503483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c59a00 msgr2=0x560cd4c5d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.604+0000 7f1cf9400640 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c59a00 0x560cd4c5d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.604+0000 7f1cf9400640 1 -- 172.21.15.112:0/1369503483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cd4c5dab0 con 0x560cd4c4f140 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.604+0000 7f1cf9400640 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c5cf30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40ff1b0 tx=0x7f1ce40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886ba35b83b7626d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.604+0000 7f1cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.604+0000 7f1cd7400640 1 -- 172.21.15.112:0/1369503483 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce401e470 con 0x560cd4c4f140 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.604+0000 7f1cd7400640 1 -- 172.21.15.112:0/1369503483 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f1ce401e610 con 0x560cd4c4f140 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.604+0000 7f1cff61e740 1 -- 172.21.15.112:0/1369503483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce8005040 con 0x560cd4c4f140 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.605+0000 7f1cd7400640 1 -- 172.21.15.112:0/1369503483 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ce4005020 con 0x560cd4c4f140 2024-04-25T18:49:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.605+0000 7f1cff61e740 10 client.15294 fetch_fsmap learned FSMap version 26 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.605+0000 7f1cff61e740 1 -- 172.21.15.112:0/1369503483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560cd4cd8970 con 0x560cd4c4f140 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.605+0000 7f1cd7400640 1 -- 172.21.15.112:0/1369503483 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce8005040 con 0x560cd4c4f140 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.605+0000 7f1cd7400640 1 -- 172.21.15.112:0/1369503483 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1ce40047a0 con 0x560cd4c4f140 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.605+0000 7f1cff61e740 10 client.15294 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.605+0000 7f1cff61e740 10 client.15294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.605+0000 7f1cff61e740 20 client.15294 populate_metadata read hostname 'smithi112' 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.605+0000 7f1cff61e740 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560cd4cd9170 0x560cd4cdb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.605+0000 7f1cff61e740 4 client.15294 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.605+0000 7f1cff61e740 1 -- 172.21.15.112:0/1369503483 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560cd4bc2320 con 0x560cd4cd9170 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.606+0000 7f1cf9e00640 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560cd4cd9170 0x560cd4cdb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.606+0000 7f1cf9e00640 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560cd4cd9170 0x560cd4cdb590 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f445ec0e640781a server_cookie=9d09a380d376b2ea in_seq=0 out_seq=0 2024-04-25T18:49:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:49:14.606+0000 7f1cd7400640 10 client.15294 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:15.130 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:14 smithi112 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.g.service: Scheduled restart job, restart counter is at 1. 2024-04-25T18:49:15.130 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:14 smithi112 systemd[1]: Stopped Ceph mds.g for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:49:15.130 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:14 smithi112 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.g.service: Consumed 2.542s CPU time. 2024-04-25T18:49:15.130 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:14 smithi112 systemd[1]: Starting Ceph mds.g for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:49:15.473 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:15 smithi112 podman[86772]: 2024-04-25 18:49:15.032924536 +0000 UTC m=+0.036863104 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:49:15.474 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:15 smithi112 podman[86772]: 2024-04-25 18:49:15.163551113 +0000 UTC m=+0.167489716 container create bb6a0b6f2936a37191de5db5c455224f873181f6a900f5cb5d2eac2dbda5f124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T18:49:15.774 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:15 smithi112 podman[86772]: 2024-04-25 18:49:15.48385587 +0000 UTC m=+0.487794451 container init bb6a0b6f2936a37191de5db5c455224f873181f6a900f5cb5d2eac2dbda5f124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:49:15.774 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:15 smithi112 podman[86772]: 2024-04-25 18:49:15.491427363 +0000 UTC m=+0.495365932 container start bb6a0b6f2936a37191de5db5c455224f873181f6a900f5cb5d2eac2dbda5f124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T18:49:15.774 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:15 smithi112 ceph-mds[86791]: -- 172.21.15.112:0/453257693 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562633e07a40 con 0x562633ddb000 2024-04-25T18:49:15.774 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: starting mds.g at 2024-04-25T18:49:15.775 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:15 smithi112 bash[86772]: bb6a0b6f2936a37191de5db5c455224f873181f6a900f5cb5d2eac2dbda5f124 2024-04-25T18:49:15.775 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:49:15 smithi112 systemd[1]: Started Ceph mds.g for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:49:16.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:15 smithi171 ceph-mon[33266]: pgmap v58: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-04-25T18:49:16.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:15 smithi171 ceph-mon[33266]: daemon mds.g restarted 2024-04-25T18:49:16.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:15 smithi161 ceph-mon[32609]: pgmap v58: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-04-25T18:49:16.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:15 smithi161 ceph-mon[32609]: daemon mds.g restarted 2024-04-25T18:49:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:15 smithi112 ceph-mon[27864]: pgmap v58: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-04-25T18:49:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:15 smithi112 ceph-mon[27864]: daemon mds.g restarted 2024-04-25T18:49:17.083 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:16 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:49:17.083 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:16 smithi171 ceph-mon[33266]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T18:49:17.084 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:16 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.112:6838/3485388562,v1:172.21.15.112:6839/3485388562] up:boot 2024-04-25T18:49:17.084 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:16 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T18:49:17.084 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:16 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:49:17.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:16 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:49:17.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:16 smithi161 ceph-mon[32609]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T18:49:17.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:16 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.112:6838/3485388562,v1:172.21.15.112:6839/3485388562] up:boot 2024-04-25T18:49:17.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:16 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T18:49:17.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:16 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:49:17.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:16 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:49:17.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:16 smithi112 ceph-mon[27864]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T18:49:17.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:16 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.112:6838/3485388562,v1:172.21.15.112:6839/3485388562] up:boot 2024-04-25T18:49:17.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:16 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T18:49:17.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:16 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:49:18.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:17 smithi171 ceph-mon[33266]: pgmap v60: 97 pgs: 97 active+clean; 20 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-04-25T18:49:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:17 smithi161 ceph-mon[32609]: pgmap v60: 97 pgs: 97 active+clean; 20 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-04-25T18:49:18.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:17 smithi112 ceph-mon[27864]: pgmap v60: 97 pgs: 97 active+clean; 20 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-04-25T18:49:20.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:19 smithi171 ceph-mon[33266]: pgmap v61: 97 pgs: 97 active+clean; 20 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-04-25T18:49:20.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:19 smithi161 ceph-mon[32609]: pgmap v61: 97 pgs: 97 active+clean; 20 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-04-25T18:49:20.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:19 smithi112 ceph-mon[27864]: pgmap v61: 97 pgs: 97 active+clean; 20 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-04-25T18:49:22.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:21 smithi171 ceph-mon[33266]: pgmap v62: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 748 KiB/s wr, 176 op/s 2024-04-25T18:49:22.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:21 smithi161 ceph-mon[32609]: pgmap v62: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 748 KiB/s wr, 176 op/s 2024-04-25T18:49:22.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:21 smithi112 ceph-mon[27864]: pgmap v62: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 748 KiB/s wr, 176 op/s 2024-04-25T18:49:24.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:23 smithi171 ceph-mon[33266]: pgmap v63: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 597 KiB/s wr, 175 op/s 2024-04-25T18:49:24.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:23 smithi161 ceph-mon[32609]: pgmap v63: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 597 KiB/s wr, 175 op/s 2024-04-25T18:49:24.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:23 smithi112 ceph-mon[27864]: pgmap v63: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 597 KiB/s wr, 175 op/s 2024-04-25T18:49:26.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:25 smithi171 ceph-mon[33266]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 597 KiB/s wr, 175 op/s 2024-04-25T18:49:26.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:25 smithi161 ceph-mon[32609]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 597 KiB/s wr, 175 op/s 2024-04-25T18:49:26.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:25 smithi112 ceph-mon[27864]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 597 KiB/s wr, 175 op/s 2024-04-25T18:49:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:27 smithi171 ceph-mon[33266]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-04-25T18:49:28.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:27 smithi161 ceph-mon[32609]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-04-25T18:49:28.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:27 smithi112 ceph-mon[27864]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-04-25T18:49:29.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:29 smithi112 ceph-mon[27864]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 5.9 KiB/s wr, 7 op/s 2024-04-25T18:49:30.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:29 smithi171 ceph-mon[33266]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 5.9 KiB/s wr, 7 op/s 2024-04-25T18:49:30.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:29 smithi161 ceph-mon[32609]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 5.9 KiB/s wr, 7 op/s 2024-04-25T18:49:32.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:31 smithi171 ceph-mon[33266]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T18:49:32.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:31 smithi161 ceph-mon[32609]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T18:49:32.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:31 smithi112 ceph-mon[27864]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T18:49:34.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:33 smithi171 ceph-mon[33266]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-04-25T18:49:34.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:33 smithi161 ceph-mon[32609]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-04-25T18:49:34.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:33 smithi112 ceph-mon[27864]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-04-25T18:49:36.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:35 smithi171 ceph-mon[33266]: pgmap v69: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-25T18:49:36.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:35 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:36.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:35 smithi161 ceph-mon[32609]: pgmap v69: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-25T18:49:36.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:35 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:36.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:35 smithi112 ceph-mon[27864]: pgmap v69: 97 pgs: 97 active+clean; 20 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-25T18:49:36.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:35 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:37.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:36 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:37.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:36 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:37.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:36 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:49:37.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:36 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:37.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:36 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:37.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:36 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:49:37.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:36 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:37.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:36 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:37.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:36 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:49:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:38 smithi171 ceph-mon[33266]: pgmap v70: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 90 KiB/s wr, 25 op/s 2024-04-25T18:49:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:38 smithi161 ceph-mon[32609]: pgmap v70: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 90 KiB/s wr, 25 op/s 2024-04-25T18:49:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:38 smithi112 ceph-mon[27864]: pgmap v70: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 90 KiB/s wr, 25 op/s 2024-04-25T18:49:39.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:39 smithi161 ceph-mon[32609]: pgmap v71: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 86 KiB/s wr, 22 op/s 2024-04-25T18:49:39.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:39 smithi112 ceph-mon[27864]: pgmap v71: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 86 KiB/s wr, 22 op/s 2024-04-25T18:49:40.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:39 smithi171 ceph-mon[33266]: pgmap v71: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 86 KiB/s wr, 22 op/s 2024-04-25T18:49:42.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:41 smithi171 ceph-mon[33266]: pgmap v72: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 89 KiB/s wr, 25 op/s 2024-04-25T18:49:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:41 smithi161 ceph-mon[32609]: pgmap v72: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 89 KiB/s wr, 25 op/s 2024-04-25T18:49:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:41 smithi112 ceph-mon[27864]: pgmap v72: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 89 KiB/s wr, 25 op/s 2024-04-25T18:49:44.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:43 smithi171 ceph-mon[33266]: pgmap v73: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 89 KiB/s wr, 22 op/s 2024-04-25T18:49:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:43 smithi161 ceph-mon[32609]: pgmap v73: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 89 KiB/s wr, 22 op/s 2024-04-25T18:49:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:43 smithi112 ceph-mon[27864]: pgmap v73: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 89 KiB/s wr, 22 op/s 2024-04-25T18:49:46.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:45 smithi171 ceph-mon[33266]: pgmap v74: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 89 KiB/s wr, 22 op/s 2024-04-25T18:49:46.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:45 smithi161 ceph-mon[32609]: pgmap v74: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 89 KiB/s wr, 22 op/s 2024-04-25T18:49:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:45 smithi112 ceph-mon[27864]: pgmap v74: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 89 KiB/s wr, 22 op/s 2024-04-25T18:49:48.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:47 smithi171 ceph-mon[33266]: pgmap v75: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 90 KiB/s wr, 25 op/s 2024-04-25T18:49:48.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:47 smithi161 ceph-mon[32609]: pgmap v75: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 90 KiB/s wr, 25 op/s 2024-04-25T18:49:48.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:47 smithi112 ceph-mon[27864]: pgmap v75: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 90 KiB/s wr, 25 op/s 2024-04-25T18:49:50.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:49 smithi171 ceph-mon[33266]: pgmap v76: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-04-25T18:49:50.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:49 smithi161 ceph-mon[32609]: pgmap v76: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-04-25T18:49:50.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:49 smithi112 ceph-mon[27864]: pgmap v76: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-04-25T18:49:52.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:51 smithi171 ceph-mon[33266]: pgmap v77: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-04-25T18:49:52.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:51 smithi161 ceph-mon[32609]: pgmap v77: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-04-25T18:49:52.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:51 smithi112 ceph-mon[27864]: pgmap v77: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-04-25T18:49:54.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:53 smithi171 ceph-mon[33266]: pgmap v78: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:49:54.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:53 smithi161 ceph-mon[32609]: pgmap v78: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:49:54.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:53 smithi112 ceph-mon[27864]: pgmap v78: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:49:55.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.166+0000 7f7bfcc00700 1 -- 172.21.15.112:0/3404639508 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: -115 ) ==== 8+0+592 (crc 0 0 0) 0x55c71a4d08b0 con 0x55c71a6016a0 2024-04-25T18:49:55.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.166+0000 7f7bfcc00700 10 client.15276 handle_command_reply: tid=0 2024-04-25T18:49:55.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.166+0000 7f7c23cdcb80 1 client.15276 shutdown 2024-04-25T18:49:55.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.166+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3404639508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 msgr2=0x55c71a586420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:55.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.166+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 0x55c71a586420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c00026440 tx=0x7f7c0000fd80 comp rx=0 tx=0).stop 2024-04-25T18:49:55.171 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:49:55.171 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 28, 2024-04-25T18:49:55.171 INFO:teuthology.orchestra.run.smithi112.stdout: "leader": 24512, 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "set_version": 5, 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "sets": { 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "5aed9937": { 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "version": 5, 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "age_ref": 0.0, 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "state": { 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "QUIESCING", 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 42.0 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "timeout": 60.0, 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "expiration": 80.8, 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "members": { 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "file:/": { 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "excluded": false, 2024-04-25T18:49:55.172 INFO:teuthology.orchestra.run.smithi112.stdout: "state": { 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "QUIESCING", 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 42.0 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.167+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3404639508 shutdown_connections 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.167+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55c71a6016a0 0x55c71a603ac0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.167+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c71a581560 0x55c71a586960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:55.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.167+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c71a575dd0 0x55c71a585ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.167+0000 7f7c23cdcb80 1 --2- 172.21.15.112:0/3404639508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c71a5766f0 0x55c71a586420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.167+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3404639508 >> 172.21.15.112:0/3404639508 conn(0x55c71a4a50e0 msgr2=0x55c71a571d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.167+0000 7f7bfcc00700 0 client.15276 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:49:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.167+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3404639508 shutdown_connections 2024-04-25T18:49:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.167+0000 7f7c23cdcb80 1 -- 172.21.15.112:0/3404639508 wait complete. 2024-04-25T18:49:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:55.167+0000 7f7c23cdcb80 20 client.15276 trim_cache size 0 max 16384 2024-04-25T18:49:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:Error EINPROGRESS: 2024-04-25T18:49:55.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:55 smithi112 ceph-mon[27864]: pgmap v79: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:49:56.124 DEBUG:teuthology.orchestra.run:got remote process result: 115 2024-04-25T18:49:56.125 WARNING:tasks.quiescer.fs.[cephfs]:Set '5aed9937' hasn't quiesced after 50.1 seconds (timeout: 60). Dumping ops with locks from all ranks. 2024-04-25T18:49:56.126 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:49:56.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:55 smithi171 ceph-mon[33266]: pgmap v79: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:49:56.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:55 smithi161 ceph-mon[32609]: pgmap v79: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:49:56.579 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:49:57.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:57 smithi112 ceph-mon[27864]: pgmap v80: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-25T18:49:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:57 smithi171 ceph-mon[33266]: pgmap v80: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-25T18:49:58.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:57 smithi161 ceph-mon[32609]: pgmap v80: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-25T18:49:58.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.974+0000 7f0ef7200700 1 -- 172.21.15.112:0/1334199877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0108a30 msgr2=0x7f0ef0108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.974+0000 7f0ef7200700 1 --2- 172.21.15.112:0/1334199877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0108a30 0x7f0ef0108e10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009990 tx=0x7f0ee0009ca0 comp rx=0 tx=0).stop 2024-04-25T18:49:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.974+0000 7f0ef7200700 1 -- 172.21.15.112:0/1334199877 shutdown_connections 2024-04-25T18:49:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.974+0000 7f0ef7200700 1 --2- 172.21.15.112:0/1334199877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef006b0f0 0x7f0ef010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.974+0000 7f0ef7200700 1 --2- 172.21.15.112:0/1334199877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0108a30 0x7f0ef0108e10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.974+0000 7f0ef7200700 1 --2- 172.21.15.112:0/1334199877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef006a750 0x7f0ef006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.974+0000 7f0ef7200700 1 -- 172.21.15.112:0/1334199877 >> 172.21.15.112:0/1334199877 conn(0x7f0ef00788f0 msgr2=0x7f0ef0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.974+0000 7f0ef7200700 1 -- 172.21.15.112:0/1334199877 shutdown_connections 2024-04-25T18:49:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.974+0000 7f0ef7200700 1 -- 172.21.15.112:0/1334199877 wait complete. 2024-04-25T18:49:58.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.976+0000 7f0ef7200700 1 Processor -- start 2024-04-25T18:49:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.976+0000 7f0ef7200700 1 -- start start 2024-04-25T18:49:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.976+0000 7f0ef7200700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef006a750 0x7f0ef0101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.976+0000 7f0ef7200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef006b0f0 0x7f0ef0101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.976+0000 7f0ef7200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef0108a30 0x7f0ef0102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0ef7200700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0112bd0 con 0x7f0ef006a750 2024-04-25T18:49:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0ef7200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0112a50 con 0x7f0ef0108a30 2024-04-25T18:49:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0ef7200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0112d50 con 0x7f0ef006b0f0 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0eef400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef006b0f0 0x7f0ef0101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0eefe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef006a750 0x7f0ef0101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0eef400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef006b0f0 0x7f0ef0101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:37394/0 (socket says 172.21.15.112:37394) 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0eef400700 1 -- 172.21.15.112:0/761678513 learned_addr learned my addr 172.21.15.112:0/761678513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0ef5200700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef0108a30 0x7f0ef0102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0eef400700 1 -- 172.21.15.112:0/761678513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef0108a30 msgr2=0x7f0ef0102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0eef400700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef0108a30 0x7f0ef0102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0eef400700 1 -- 172.21.15.112:0/761678513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef006a750 msgr2=0x7f0ef0101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0eef400700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef006a750 0x7f0ef0101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0eef400700 1 -- 172.21.15.112:0/761678513 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef01a7670 con 0x7f0ef006b0f0 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0eefe00700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef006a750 0x7f0ef0101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:49:58.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.977+0000 7f0ef5200700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef0108a30 0x7f0ef0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:49:58.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.978+0000 7f0eef400700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef006b0f0 0x7f0ef0101f50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00e9c0 tx=0x7f0edc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:58.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.978+0000 7f0eecc00700 1 -- 172.21.15.112:0/761678513 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00cbd0 con 0x7f0ef006b0f0 2024-04-25T18:49:58.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.978+0000 7f0eecc00700 1 -- 172.21.15.112:0/761678513 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0edc00cd70 con 0x7f0ef006b0f0 2024-04-25T18:49:58.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.978+0000 7f0ef7200700 1 -- 172.21.15.112:0/761678513 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef01a7960 con 0x7f0ef006b0f0 2024-04-25T18:49:58.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.978+0000 7f0eecc00700 1 -- 172.21.15.112:0/761678513 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc005610 con 0x7f0ef006b0f0 2024-04-25T18:49:58.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.978+0000 7f0ef7200700 1 -- 172.21.15.112:0/761678513 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef01a7e50 con 0x7f0ef006b0f0 2024-04-25T18:49:58.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.979+0000 7f0ef7200700 1 -- 172.21.15.112:0/761678513 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef0069730 con 0x7f0ef006b0f0 2024-04-25T18:49:58.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.981+0000 7f0eecc00700 1 -- 172.21.15.112:0/761678513 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f0edc004750 con 0x7f0ef006b0f0 2024-04-25T18:49:58.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.982+0000 7f0eecc00700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f0ed40a3db0 0x7f0ed40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:58.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.983+0000 7f0eecc00700 1 -- 172.21.15.112:0/761678513 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f0edc0d0b90 con 0x7f0ef006b0f0 2024-04-25T18:49:58.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.983+0000 7f0eefe00700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f0ed40a3db0 0x7f0ed40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:58.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.983+0000 7f0eefe00700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f0ed40a3db0 0x7f0ed40a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b2e0 tx=0x7f0ee001a040 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:58.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:58.987+0000 7f0eecc00700 1 -- 172.21.15.112:0/761678513 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0edc09d1d0 con 0x7f0ef006b0f0 2024-04-25T18:49:59.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.263+0000 7f0ef7200700 1 -- 172.21.15.112:0/761678513 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ef002df80 con 0x7f0ef006b0f0 2024-04-25T18:49:59.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.264+0000 7f0eecc00700 1 -- 172.21.15.112:0/761678513 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11893 (secure 0 0 0) 0x7f0edc09cad0 con 0x7f0ef006b0f0 2024-04-25T18:49:59.266 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:49:59.266 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:49:15.768258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25459":{"gid":25459,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3485388562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3485388562},{"type":"v1","addr":"172.21.15.112:6839","nonce":3485388562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24511,24512,24493]},"id":1}]} 2024-04-25T18:49:59.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 -- 172.21.15.112:0/761678513 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f0ed40a3db0 msgr2=0x7f0ed40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:59.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f0ed40a3db0 0x7f0ed40a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b2e0 tx=0x7f0ee001a040 comp rx=0 tx=0).stop 2024-04-25T18:49:59.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 -- 172.21.15.112:0/761678513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef006b0f0 msgr2=0x7f0ef0101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:59.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef006b0f0 0x7f0ef0101f50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00e9c0 tx=0x7f0edc00ecd0 comp rx=0 tx=0).stop 2024-04-25T18:49:59.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 -- 172.21.15.112:0/761678513 shutdown_connections 2024-04-25T18:49:59.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f0ed40a3db0 0x7f0ed40a6270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:59.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef0108a30 0x7f0ef0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:59.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef006a750 0x7f0ef0101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 --2- 172.21.15.112:0/761678513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef006b0f0 0x7f0ef0101f50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 -- 172.21.15.112:0/761678513 >> 172.21.15.112:0/761678513 conn(0x7f0ef00788f0 msgr2=0x7f0ef00ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 -- 172.21.15.112:0/761678513 shutdown_connections 2024-04-25T18:49:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:49:59.267+0000 7f0ef7200700 1 -- 172.21.15.112:0/761678513 wait complete. 2024-04-25T18:49:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 28 2024-04-25T18:49:59.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:59 smithi112 ceph-mon[27864]: pgmap v81: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:49:59.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:49:59 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/761678513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:50:00.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:59 smithi171 ceph-mon[33266]: pgmap v81: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:50:00.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:49:59 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/761678513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:50:00.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:59 smithi161 ceph-mon[32609]: pgmap v81: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:50:00.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:49:59 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/761678513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:50:00.316 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 2 (i) to a remote file /var/run/ceph/1/ops-5aed9937-1-mds.i.json 2024-04-25T18:50:00.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.1:2 ops --flags=locks --path=/var/run/ceph/ops-5aed9937-1-mds.i.json 2024-04-25T18:50:00.759 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:50:01.129 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:00 smithi112 ceph-mon[27864]: overall HEALTH_OK 2024-04-25T18:50:01.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:00 smithi171 ceph-mon[33266]: overall HEALTH_OK 2024-04-25T18:50:01.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:00 smithi161 ceph-mon[32609]: overall HEALTH_OK 2024-04-25T18:50:02.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:01 smithi171 ceph-mon[33266]: pgmap v82: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T18:50:02.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:01 smithi161 ceph-mon[32609]: pgmap v82: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T18:50:02.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:01 smithi112 ceph-mon[27864]: pgmap v82: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T18:50:03.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.285+0000 7f8df7800700 1 -- 172.21.15.112:0/3261464317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df0108a50 msgr2=0x7f8df0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:03.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.285+0000 7f8df7800700 1 --2- 172.21.15.112:0/3261464317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df0108a50 0x7f8df0108e30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009990 tx=0x7f8de4009ca0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.286+0000 7f8df7800700 1 -- 172.21.15.112:0/3261464317 shutdown_connections 2024-04-25T18:50:03.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.286+0000 7f8df7800700 1 --2- 172.21.15.112:0/3261464317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8df006bab0 0x7f8df010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.286+0000 7f8df7800700 1 --2- 172.21.15.112:0/3261464317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df0108a50 0x7f8df0108e30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.286+0000 7f8df7800700 1 --2- 172.21.15.112:0/3261464317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8df006b110 0x7f8df006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.286+0000 7f8df7800700 1 -- 172.21.15.112:0/3261464317 >> 172.21.15.112:0/3261464317 conn(0x7f8df00788f0 msgr2=0x7f8df0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.286+0000 7f8df7800700 1 -- 172.21.15.112:0/3261464317 shutdown_connections 2024-04-25T18:50:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.286+0000 7f8df7800700 1 -- 172.21.15.112:0/3261464317 wait complete. 2024-04-25T18:50:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.287+0000 7f8df7800700 1 Processor -- start 2024-04-25T18:50:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.287+0000 7f8df7800700 1 -- start start 2024-04-25T18:50:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.287+0000 7f8df7800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df006b110 0x7f8df0101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.287+0000 7f8df7800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8df006bab0 0x7f8df0102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.287+0000 7f8df7800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8df0108a50 0x7f8df0102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.287+0000 7f8df7800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8df0112be0 con 0x7f8df006b110 2024-04-25T18:50:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.287+0000 7f8df7800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8df0112a60 con 0x7f8df006bab0 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.287+0000 7f8df7800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8df0112d60 con 0x7f8df0108a50 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.288+0000 7f8df4e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df006b110 0x7f8df0101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.288+0000 7f8df4e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df006b110 0x7f8df0101df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39058/0 (socket says 172.21.15.112:39058) 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.288+0000 7f8df4e00700 1 -- 172.21.15.112:0/3833584163 learned_addr learned my addr 172.21.15.112:0/3833584163 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.288+0000 7f8defe00700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8df006bab0 0x7f8df0102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.288+0000 7f8df5800700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8df0108a50 0x7f8df0102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.288+0000 7f8df4e00700 1 -- 172.21.15.112:0/3833584163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8df0108a50 msgr2=0x7f8df0102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.288+0000 7f8df4e00700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8df0108a50 0x7f8df0102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.288+0000 7f8df4e00700 1 -- 172.21.15.112:0/3833584163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8df006bab0 msgr2=0x7f8df0102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.288+0000 7f8df4e00700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8df006bab0 0x7f8df0102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.288+0000 7f8df4e00700 1 -- 172.21.15.112:0/3833584163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df01a7680 con 0x7f8df006b110 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.289+0000 7f8df4e00700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df006b110 0x7f8df0101df0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8de40057f0 tx=0x7f8de400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.289+0000 7f8ded600700 1 -- 172.21.15.112:0/3833584163 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4026070 con 0x7f8df006b110 2024-04-25T18:50:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.289+0000 7f8ded600700 1 -- 172.21.15.112:0/3833584163 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de4021410 con 0x7f8df006b110 2024-04-25T18:50:03.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.289+0000 7f8df7800700 1 -- 172.21.15.112:0/3833584163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df01a7910 con 0x7f8df006b110 2024-04-25T18:50:03.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.289+0000 7f8ded600700 1 -- 172.21.15.112:0/3833584163 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4017650 con 0x7f8df006b110 2024-04-25T18:50:03.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.289+0000 7f8df7800700 1 -- 172.21.15.112:0/3833584163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df01a7e40 con 0x7f8df006b110 2024-04-25T18:50:03.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.292+0000 7f8ded600700 1 -- 172.21.15.112:0/3833584163 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f8de4029070 con 0x7f8df006b110 2024-04-25T18:50:03.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.294+0000 7f8ded600700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8dd80a3e30 0x7f8dd80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.294+0000 7f8e020b9b80 1 Processor -- start 2024-04-25T18:50:03.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.294+0000 7f8ded600700 1 -- 172.21.15.112:0/3833584163 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f8de40d1550 con 0x7f8df006b110 2024-04-25T18:50:03.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.294+0000 7f8e020b9b80 1 -- start start 2024-04-25T18:50:03.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.294+0000 7f8defe00700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8dd80a3e30 0x7f8dd80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.294+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6d7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.294+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d7640 0x558dee6e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.294+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 0x558dee6e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.294+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558dee629f70 con 0x558dee6e24b0 2024-04-25T18:50:03.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.294+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558dee5d0810 con 0x558dee6d7640 2024-04-25T18:50:03.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.294+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558dee625f30 con 0x558dee6d6d20 2024-04-25T18:50:03.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.295+0000 7f8df4e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.295+0000 7f8df5800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 0x558dee6e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.295+0000 7f8df4e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6d7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:37682/0 (socket says 172.21.15.112:37682) 2024-04-25T18:50:03.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.295+0000 7f8df4e00700 1 -- 172.21.15.112:0/1629504160 learned_addr learned my addr 172.21.15.112:0/1629504160 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:03.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.295+0000 7f8defe00700 1 --2- 172.21.15.112:0/1629504160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d7640 0x558dee6e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.295+0000 7f8defe00700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8dd80a3e30 0x7f8dd80a62f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8df0079030 tx=0x7f8ddc009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:03.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.295+0000 7f8df4e00700 1 -- 172.21.15.112:0/1629504160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d7640 msgr2=0x558dee6e1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:03.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.295+0000 7f8df4e00700 1 --2- 172.21.15.112:0/1629504160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d7640 0x558dee6e1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.295+0000 7f8df4e00700 1 -- 172.21.15.112:0/1629504160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 msgr2=0x558dee6e48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.295+0000 7f8df4e00700 1 --2- 172.21.15.112:0/1629504160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 0x558dee6e48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.295+0000 7f8df4e00700 1 -- 172.21.15.112:0/1629504160 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dee5c6760 con 0x558dee6d6d20 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.296+0000 7f8df4e00700 1 --2- 172.21.15.112:0/1629504160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6d7100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8de4099ef0 tx=0x7f8de409d370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef17fbda62e9f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.296+0000 7f8de2000700 1 -- 172.21.15.112:0/1629504160 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de409dc20 con 0x558dee6d6d20 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.296+0000 7f8de2000700 1 -- 172.21.15.112:0/1629504160 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de409d520 con 0x558dee6d6d20 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.296+0000 7f8de2000700 1 -- 172.21.15.112:0/1629504160 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de40218c0 con 0x558dee6d6d20 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.297+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1629504160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 msgr2=0x558dee6d7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.297+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1629504160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6d7100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8de4099ef0 tx=0x7f8de409d370 comp rx=0 tx=0).stop 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.297+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1629504160 shutdown_connections 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.297+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1629504160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 0x558dee6e48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.297+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1629504160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6d7100 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.297+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1629504160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d7640 0x558dee6e1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.297+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1629504160 >> 172.21.15.112:0/1629504160 conn(0x558dee606030 msgr2=0x558dee6d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:03.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.297+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1629504160 shutdown_connections 2024-04-25T18:50:03.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.297+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1629504160 wait complete. 2024-04-25T18:50:03.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.297+0000 7f8e020b9b80 1 -- start start 2024-04-25T18:50:03.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.298+0000 7f8e020b9b80 1 Processor -- start 2024-04-25T18:50:03.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.298+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d6d20 0x558dee6e1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.298+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6d7640 0x558dee6def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.298+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6e24b0 0x558dee6df490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.298+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558dee6e4de0 con 0x558dee6d7640 2024-04-25T18:50:03.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.298+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558dee5e3940 con 0x558dee6d6d20 2024-04-25T18:50:03.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.298+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558dee6e4f60 con 0x558dee6e24b0 2024-04-25T18:50:03.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.298+0000 7f8e020b9b80 10 client.0 _ll_register_callbacks cb 0x558dee5c2800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:03.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8df5800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6e24b0 0x558dee6df490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8df4e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d6d20 0x558dee6e1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8defe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6d7640 0x558dee6def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8df5800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6e24b0 0x558dee6df490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:37690/0 (socket says 172.21.15.112:37690) 2024-04-25T18:50:03.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8df5800700 1 -- 172.21.15.112:0/1308468025 learned_addr learned my addr 172.21.15.112:0/1308468025 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:03.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8df5800700 1 -- 172.21.15.112:0/1308468025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d6d20 msgr2=0x558dee6e1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:03.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8df5800700 1 --2- 172.21.15.112:0/1308468025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d6d20 0x558dee6e1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8df5800700 1 -- 172.21.15.112:0/1308468025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6d7640 msgr2=0x558dee6def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8df5800700 1 --2- 172.21.15.112:0/1308468025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6d7640 0x558dee6def50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8df5800700 1 -- 172.21.15.112:0/1308468025 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dee5779b0 con 0x558dee6e24b0 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8df5800700 1 --2- 172.21.15.112:0/1308468025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6e24b0 0x558dee6df490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8de8007bc0 tx=0x7f8de8009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb2948911a14f044 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.299+0000 7f8dd6a00700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.300+0000 7f8dd6a00700 1 -- 172.21.15.112:0/1308468025 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de801a030 con 0x558dee6e24b0 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.300+0000 7f8dd6a00700 1 -- 172.21.15.112:0/1308468025 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f8de8003a00 con 0x558dee6e24b0 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.300+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1308468025 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8de409d520 con 0x558dee6e24b0 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.300+0000 7f8dd6a00700 1 -- 172.21.15.112:0/1308468025 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8de8026050 con 0x558dee6e24b0 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.300+0000 7f8e020b9b80 10 client.25477 fetch_fsmap learned FSMap version 28 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.300+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1308468025 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558dee5c0bd0 con 0x558dee6e24b0 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.300+0000 7f8dd6a00700 1 -- 172.21.15.112:0/1308468025 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de409d520 con 0x558dee6e24b0 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.301+0000 7f8dd6a00700 1 -- 172.21.15.112:0/1308468025 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f8de8016870 con 0x558dee6e24b0 2024-04-25T18:50:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.301+0000 7f8e020b9b80 10 client.25477 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T18:50:03.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.301+0000 7f8e020b9b80 10 client.25477 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T18:50:03.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.301+0000 7f8e020b9b80 20 client.25477 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:03.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.301+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1308468025 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x558dee6f1a60 0x558dee6f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.301+0000 7f8e020b9b80 4 client.25477 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:50:03.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.301+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1308468025 --> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dee637760 con 0x558dee6f1a60 2024-04-25T18:50:03.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.301+0000 7f8df4e00700 1 --2- 172.21.15.112:0/1308468025 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x558dee6f1a60 0x558dee6f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.302+0000 7f8df4e00700 1 --2- 172.21.15.112:0/1308468025 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x558dee6f1a60 0x558dee6f3e80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a32b46da4fde554 server_cookie=72e123e66a751236 in_seq=0 out_seq=0 2024-04-25T18:50:03.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.302+0000 7f8dd6a00700 10 client.25477 ms_handle_connect on v2:172.21.15.171:6838/1642228003 2024-04-25T18:50:03.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8dd6a00700 1 -- 172.21.15.112:0/1308468025 <== mds.2 v2:172.21.15.171:6838/1642228003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558dee637760 con 0x558dee6f1a60 2024-04-25T18:50:03.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8dd6a00700 10 client.25477 handle_command_reply: tid=0 2024-04-25T18:50:03.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 1 client.25477 shutdown 2024-04-25T18:50:03.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1308468025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6e24b0 msgr2=0x558dee6df490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:03.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1308468025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6e24b0 0x558dee6df490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8de8007bc0 tx=0x7f8de8009510 comp rx=0 tx=0).stop 2024-04-25T18:50:03.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1308468025 shutdown_connections 2024-04-25T18:50:03.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1308468025 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x558dee6f1a60 0x558dee6f3e80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1308468025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6e24b0 0x558dee6df490 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1308468025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d6d20 0x558dee6e1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1308468025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6d7640 0x558dee6def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1308468025 >> 172.21.15.112:0/1308468025 conn(0x558dee606030 msgr2=0x558dee6d2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:03.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8dd6a00700 0 client.25477 ms_handle_reset on v2:172.21.15.171:6838/1642228003 2024-04-25T18:50:03.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1308468025 shutdown_connections 2024-04-25T18:50:03.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1308468025 wait complete. 2024-04-25T18:50:03.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.303+0000 7f8e020b9b80 20 client.25477 trim_cache size 0 max 16384 2024-04-25T18:50:03.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8e020b9b80 1 Processor -- start 2024-04-25T18:50:03.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8e020b9b80 1 -- start start 2024-04-25T18:50:03.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d6d20 0x558dee6d33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d7640 0x558dee6f9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 0x558dee6f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01b1e70 con 0x558dee6e24b0 2024-04-25T18:50:03.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01af700 con 0x558dee6d6d20 2024-04-25T18:50:03.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01ad550 con 0x558dee6d7640 2024-04-25T18:50:03.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8df5800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 0x558dee6f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8df5800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 0x558dee6f9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39082/0 (socket says 172.21.15.112:39082) 2024-04-25T18:50:03.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8df5800700 1 -- 172.21.15.112:0/1839144661 learned_addr learned my addr 172.21.15.112:0/1839144661 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:03.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.323+0000 7f8df4e00700 1 --2- 172.21.15.112:0/1839144661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d6d20 0x558dee6d33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.324+0000 7f8defe00700 1 --2- 172.21.15.112:0/1839144661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d7640 0x558dee6f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.324+0000 7f8df5800700 1 -- 172.21.15.112:0/1839144661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d7640 msgr2=0x558dee6f9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:03.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.324+0000 7f8df5800700 1 --2- 172.21.15.112:0/1839144661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d7640 0x558dee6f9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.324+0000 7f8df5800700 1 -- 172.21.15.112:0/1839144661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d6d20 msgr2=0x558dee6d33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:03.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.324+0000 7f8df5800700 1 --2- 172.21.15.112:0/1839144661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d6d20 0x558dee6d33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.324+0000 7f8df5800700 1 -- 172.21.15.112:0/1839144661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dee6ef4e0 con 0x558dee6e24b0 2024-04-25T18:50:03.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.324+0000 7f8df4e00700 1 --2- 172.21.15.112:0/1839144661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d6d20 0x558dee6d33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:50:03.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.324+0000 7f8df5800700 1 --2- 172.21.15.112:0/1839144661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 0x558dee6f9c90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8de8008fb0 tx=0x7f8de8005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d157e54a7dcc587 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.325+0000 7f8de1600700 1 -- 172.21.15.112:0/1839144661 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de801a040 con 0x558dee6e24b0 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.325+0000 7f8de1600700 1 -- 172.21.15.112:0/1839144661 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de8015720 con 0x558dee6e24b0 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.325+0000 7f8de1600700 1 -- 172.21.15.112:0/1839144661 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de801f6b0 con 0x558dee6e24b0 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.325+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1839144661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 msgr2=0x558dee6f9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.325+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1839144661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 0x558dee6f9c90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8de8008fb0 tx=0x7f8de8005aa0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.326+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1839144661 shutdown_connections 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.326+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1839144661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6e24b0 0x558dee6f9c90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.326+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1839144661 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6d6d20 0x558dee6d33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.326+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/1839144661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d7640 0x558dee6f9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.326+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1839144661 >> 172.21.15.112:0/1839144661 conn(0x558dee606030 msgr2=0x558dee6e1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.326+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1839144661 shutdown_connections 2024-04-25T18:50:03.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.326+0000 7f8e020b9b80 1 -- 172.21.15.112:0/1839144661 wait complete. 2024-04-25T18:50:03.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.326+0000 7f8e020b9b80 1 -- start start 2024-04-25T18:50:03.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.327+0000 7f8e020b9b80 1 Processor -- start 2024-04-25T18:50:03.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.327+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.327+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6d7640 0x558dee6e7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.327+0000 7f8e020b9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6e24b0 0x558dee6e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.327+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01adbc0 con 0x558dee6d7640 2024-04-25T18:50:03.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.327+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01af030 con 0x558dee6e24b0 2024-04-25T18:50:03.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.327+0000 7f8e020b9b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01ae290 con 0x558dee6d6d20 2024-04-25T18:50:03.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.327+0000 7f8e020b9b80 10 client.0 _ll_register_callbacks cb 0x558dee5c2800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:03.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8df4e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8df4e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6e6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:37700/0 (socket says 172.21.15.112:37700) 2024-04-25T18:50:03.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8df4e00700 1 -- 172.21.15.112:0/4018724015 learned_addr learned my addr 172.21.15.112:0/4018724015 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8defe00700 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6d7640 0x558dee6e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8df5800700 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6e24b0 0x558dee6e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8df4e00700 1 -- 172.21.15.112:0/4018724015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6e24b0 msgr2=0x558dee6e78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8df4e00700 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6e24b0 0x558dee6e78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8df4e00700 1 -- 172.21.15.112:0/4018724015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6d7640 msgr2=0x558dee6e7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8df4e00700 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6d7640 0x558dee6e7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8df4e00700 1 -- 172.21.15.112:0/4018724015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dee6d7350 con 0x558dee6d6d20 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8df4e00700 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6e6e40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009ce0 tx=0x7f8de4005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=397cae436b3973e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.328+0000 7f8dd6a00700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.329+0000 7f8dd6a00700 1 -- 172.21.15.112:0/4018724015 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4086070 con 0x558dee6d6d20 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.329+0000 7f8dd6a00700 1 -- 172.21.15.112:0/4018724015 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f8de4099b90 con 0x558dee6d6d20 2024-04-25T18:50:03.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.329+0000 7f8e020b9b80 1 -- 172.21.15.112:0/4018724015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8de8015720 con 0x558dee6d6d20 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.329+0000 7f8dd6a00700 1 -- 172.21.15.112:0/4018724015 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8de409d520 con 0x558dee6d6d20 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.329+0000 7f8e020b9b80 10 client.25483 fetch_fsmap learned FSMap version 28 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.329+0000 7f8e020b9b80 1 -- 172.21.15.112:0/4018724015 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558dee6d8b90 con 0x558dee6d6d20 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.329+0000 7f8dd6a00700 1 -- 172.21.15.112:0/4018724015 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de8015720 con 0x558dee6d6d20 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.330+0000 7f8dd6a00700 1 -- 172.21.15.112:0/4018724015 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f8de40218c0 con 0x558dee6d6d20 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.330+0000 7f8e020b9b80 10 client.25483 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.330+0000 7f8e020b9b80 10 client.25483 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.330+0000 7f8e020b9b80 20 client.25483 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.330+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x558dee759480 0x558dee75b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.330+0000 7f8e020b9b80 4 client.25483 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-5aed9937-1-mds.i.json"}] 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.330+0000 7f8e020b9b80 1 -- 172.21.15.112:0/4018724015 --> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-5aed9937-1-mds.i.json"}) -- 0x558dee631800 con 0x558dee759480 2024-04-25T18:50:03.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.330+0000 7f8defe00700 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x558dee759480 0x558dee75b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:03.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.331+0000 7f8defe00700 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x558dee759480 0x558dee75b8a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50c5a7d7fe0765e3 server_cookie=754b07274c5e1c5 in_seq=0 out_seq=0 2024-04-25T18:50:03.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.331+0000 7f8dd6a00700 10 client.25483 ms_handle_connect on v2:172.21.15.171:6838/1642228003 2024-04-25T18:50:03.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8dd6a00700 1 -- 172.21.15.112:0/4018724015 <== mds.2 v2:172.21.15.171:6838/1642228003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x558dee631800 con 0x558dee759480 2024-04-25T18:50:03.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8dd6a00700 10 client.25483 handle_command_reply: tid=0 2024-04-25T18:50:03.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8e020b9b80 1 client.25483 shutdown 2024-04-25T18:50:03.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8e020b9b80 1 -- 172.21.15.112:0/4018724015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 msgr2=0x558dee6e6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:03.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6e6e40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009ce0 tx=0x7f8de4005f00 comp rx=0 tx=0).stop 2024-04-25T18:50:03.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8e020b9b80 1 -- 172.21.15.112:0/4018724015 shutdown_connections 2024-04-25T18:50:03.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.171:6838/1642228003,v1:172.21.15.171:6839/1642228003] conn(0x558dee759480 0x558dee75b8a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dee6e24b0 0x558dee6e78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dee6d6d20 0x558dee6e6e40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8e020b9b80 1 --2- 172.21.15.112:0/4018724015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dee6d7640 0x558dee6e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8e020b9b80 1 -- 172.21.15.112:0/4018724015 >> 172.21.15.112:0/4018724015 conn(0x558dee606030 msgr2=0x558dee6d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:03.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8dd6a00700 0 client.25483 ms_handle_reset on v2:172.21.15.171:6838/1642228003 2024-04-25T18:50:03.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8e020b9b80 1 -- 172.21.15.112:0/4018724015 shutdown_connections 2024-04-25T18:50:03.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.332+0000 7f8e020b9b80 1 -- 172.21.15.112:0/4018724015 wait complete. 2024-04-25T18:50:03.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.333+0000 7f8e020b9b80 20 client.25483 trim_cache size 0 max 16384 2024-04-25T18:50:03.337 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:50:03.337 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/var/run/ceph/ops-5aed9937-1-mds.i.json" 2024-04-25T18:50:03.337 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:50:03.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.333+0000 7f8df7800700 1 -- 172.21.15.112:0/3833584163 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8dd80a3e30 msgr2=0x7f8dd80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:03.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.333+0000 7f8df7800700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8dd80a3e30 0x7f8dd80a62f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8df0079030 tx=0x7f8ddc009260 comp rx=0 tx=0).stop 2024-04-25T18:50:03.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.333+0000 7f8df7800700 1 -- 172.21.15.112:0/3833584163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df006b110 msgr2=0x7f8df0101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:03.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.333+0000 7f8df7800700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df006b110 0x7f8df0101df0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8de40057f0 tx=0x7f8de400bfd0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.333+0000 7f8df7800700 1 -- 172.21.15.112:0/3833584163 shutdown_connections 2024-04-25T18:50:03.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.333+0000 7f8df7800700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8dd80a3e30 0x7f8dd80a62f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.333+0000 7f8df7800700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8df0108a50 0x7f8df0102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.333+0000 7f8df7800700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df006b110 0x7f8df0101df0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.333+0000 7f8df7800700 1 --2- 172.21.15.112:0/3833584163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8df006bab0 0x7f8df0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:03.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.333+0000 7f8df7800700 1 -- 172.21.15.112:0/3833584163 >> 172.21.15.112:0/3833584163 conn(0x7f8df00788f0 msgr2=0x7f8df00fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:03.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.334+0000 7f8df7800700 1 -- 172.21.15.112:0/3833584163 shutdown_connections 2024-04-25T18:50:03.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:03.334+0000 7f8df7800700 1 -- 172.21.15.112:0/3833584163 wait complete. 2024-04-25T18:50:04.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:03 smithi171 ceph-mon[33266]: pgmap v83: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T18:50:04.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:03 smithi161 ceph-mon[32609]: pgmap v83: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T18:50:04.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:03 smithi112 ceph-mon[27864]: pgmap v83: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T18:50:04.402 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 1 (a) to a remote file /var/run/ceph/1/ops-5aed9937-1-mds.a.json 2024-04-25T18:50:04.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.1:1 ops --flags=locks --path=/var/run/ceph/ops-5aed9937-1-mds.a.json 2024-04-25T18:50:04.839 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:50:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:05 smithi171 ceph-mon[33266]: pgmap v84: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T18:50:06.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:05 smithi161 ceph-mon[32609]: pgmap v84: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T18:50:06.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:05 smithi112 ceph-mon[27864]: pgmap v84: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T18:50:07.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.320+0000 7f050b000700 1 -- 172.21.15.112:0/3145613122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0504103460 msgr2=0x7f050410f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.320+0000 7f050b000700 1 --2- 172.21.15.112:0/3145613122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0504103460 0x7f050410f940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00b0a0 tx=0x7f04fc00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.320+0000 7f050b000700 1 -- 172.21.15.112:0/3145613122 shutdown_connections 2024-04-25T18:50:07.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.320+0000 7f050b000700 1 --2- 172.21.15.112:0/3145613122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0504103460 0x7f050410f940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.320+0000 7f050b000700 1 --2- 172.21.15.112:0/3145613122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0504108a70 0x7f0504108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.320+0000 7f050b000700 1 --2- 172.21.15.112:0/3145613122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0504102ac0 0x7f0504102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.320+0000 7f050b000700 1 -- 172.21.15.112:0/3145613122 >> 172.21.15.112:0/3145613122 conn(0x7f0504079b10 msgr2=0x7f0504079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:07.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.320+0000 7f050b000700 1 -- 172.21.15.112:0/3145613122 shutdown_connections 2024-04-25T18:50:07.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.320+0000 7f050b000700 1 -- 172.21.15.112:0/3145613122 wait complete. 2024-04-25T18:50:07.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.322+0000 7f050b000700 1 Processor -- start 2024-04-25T18:50:07.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.322+0000 7f050b000700 1 -- start start 2024-04-25T18:50:07.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.322+0000 7f050b000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0504102ac0 0x7f050419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.322+0000 7f050b000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0504103460 0x7f050419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f050b000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0504108a70 0x7f050419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f050b000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0504112c80 con 0x7f0504103460 2024-04-25T18:50:07.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f050b000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0504112b00 con 0x7f0504108a70 2024-04-25T18:50:07.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f050b000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0504112e00 con 0x7f0504102ac0 2024-04-25T18:50:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f0503400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0504103460 0x7f050419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f0503e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0504102ac0 0x7f050419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f0503e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0504102ac0 0x7f050419b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:37720/0 (socket says 172.21.15.112:37720) 2024-04-25T18:50:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f0503e00700 1 -- 172.21.15.112:0/2706896936 learned_addr learned my addr 172.21.15.112:0/2706896936 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f0509000700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0504108a70 0x7f050419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f0503e00700 1 -- 172.21.15.112:0/2706896936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0504108a70 msgr2=0x7f050419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f0503e00700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0504108a70 0x7f050419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f0503e00700 1 -- 172.21.15.112:0/2706896936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0504103460 msgr2=0x7f050419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f0503e00700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0504103460 0x7f050419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.323+0000 7f0503e00700 1 -- 172.21.15.112:0/2706896936 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05041a02e0 con 0x7f0504102ac0 2024-04-25T18:50:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.324+0000 7f0503e00700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0504102ac0 0x7f050419b2d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f04f400e9a0 tx=0x7f04f400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.324+0000 7f0500c00700 1 -- 172.21.15.112:0/2706896936 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f400ca10 con 0x7f0504102ac0 2024-04-25T18:50:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.324+0000 7f0500c00700 1 -- 172.21.15.112:0/2706896936 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f400cbb0 con 0x7f0504102ac0 2024-04-25T18:50:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.324+0000 7f050b000700 1 -- 172.21.15.112:0/2706896936 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05041a05d0 con 0x7f0504102ac0 2024-04-25T18:50:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.324+0000 7f0500c00700 1 -- 172.21.15.112:0/2706896936 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4018680 con 0x7f0504102ac0 2024-04-25T18:50:07.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.324+0000 7f050b000700 1 -- 172.21.15.112:0/2706896936 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05041a7e50 con 0x7f0504102ac0 2024-04-25T18:50:07.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.329+0000 7f0500c00700 1 -- 172.21.15.112:0/2706896936 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f04f4010430 con 0x7f0504102ac0 2024-04-25T18:50:07.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.330+0000 7f0515905b80 1 Processor -- start 2024-04-25T18:50:07.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.330+0000 7f0515905b80 1 -- start start 2024-04-25T18:50:07.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.330+0000 7f0515905b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13fe7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.330+0000 7f0515905b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13ff2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.330+0000 7f0515905b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ff54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.330+0000 7f0515905b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ea13f3af70 con 0x55ea13ff30d0 2024-04-25T18:50:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.330+0000 7f0515905b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ea13ee1810 con 0x55ea13fe8220 2024-04-25T18:50:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.330+0000 7f0515905b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ea13f36f30 con 0x55ea13fe7900 2024-04-25T18:50:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0500c00700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f04e80a3e30 0x7f04e80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0503e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13ff2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0503e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13ff2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:43242/0 (socket says 172.21.15.112:43242) 2024-04-25T18:50:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0503400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ff54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0503e00700 1 -- 172.21.15.112:0/1878889929 learned_addr learned my addr 172.21.15.112:0/1878889929 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0509000700 1 --2- 172.21.15.112:0/1878889929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13fe7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0500c00700 1 -- 172.21.15.112:0/2706896936 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f04f40d0900 con 0x7f0504102ac0 2024-04-25T18:50:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0509000700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f04e80a3e30 0x7f04e80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0509000700 1 -- 172.21.15.112:0/1878889929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 msgr2=0x55ea13ff2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0509000700 1 --2- 172.21.15.112:0/1878889929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13ff2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0509000700 1 -- 172.21.15.112:0/1878889929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 msgr2=0x55ea13ff54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.331+0000 7f0509000700 1 --2- 172.21.15.112:0/1878889929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ff54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.332+0000 7f0509000700 1 -- 172.21.15.112:0/1878889929 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea13ed7760 con 0x55ea13fe7900 2024-04-25T18:50:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.332+0000 7f0509000700 1 --2- 172.21.15.112:0/1878889929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13fe7ce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0062a0 tx=0x7f04fc00bd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f2ec46e720abbac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.332+0000 7f0509000700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f04e80a3e30 0x7f04e80a62f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f04fc01a000 tx=0x7f04fc027000 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.332+0000 7f04f9600700 1 -- 172.21.15.112:0/1878889929 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0076e0 con 0x55ea13fe7900 2024-04-25T18:50:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.332+0000 7f04f9600700 1 -- 172.21.15.112:0/1878889929 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04fc009490 con 0x55ea13fe7900 2024-04-25T18:50:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.332+0000 7f04f9600700 1 -- 172.21.15.112:0/1878889929 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0056f0 con 0x55ea13fe7900 2024-04-25T18:50:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.333+0000 7f0515905b80 1 -- 172.21.15.112:0/1878889929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 msgr2=0x55ea13fe7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.333+0000 7f0515905b80 1 --2- 172.21.15.112:0/1878889929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13fe7ce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0062a0 tx=0x7f04fc00bd40 comp rx=0 tx=0).stop 2024-04-25T18:50:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.333+0000 7f0515905b80 1 -- 172.21.15.112:0/1878889929 shutdown_connections 2024-04-25T18:50:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.333+0000 7f0515905b80 1 --2- 172.21.15.112:0/1878889929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ff54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.333+0000 7f0515905b80 1 --2- 172.21.15.112:0/1878889929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13fe7ce0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.333+0000 7f0515905b80 1 --2- 172.21.15.112:0/1878889929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13ff2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.333+0000 7f0515905b80 1 -- 172.21.15.112:0/1878889929 >> 172.21.15.112:0/1878889929 conn(0x55ea13f16680 msgr2=0x55ea13fe3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.333+0000 7f0515905b80 1 -- 172.21.15.112:0/1878889929 shutdown_connections 2024-04-25T18:50:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.333+0000 7f0515905b80 1 -- 172.21.15.112:0/1878889929 wait complete. 2024-04-25T18:50:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.334+0000 7f0515905b80 1 -- start start 2024-04-25T18:50:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.334+0000 7f0515905b80 1 Processor -- start 2024-04-25T18:50:07.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0515905b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13ff2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0515905b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13fefb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0515905b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ff00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0515905b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ea13ff5b90 con 0x55ea13ff30d0 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0515905b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ea13ef4940 con 0x55ea13fe8220 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0515905b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ea13ff5d10 con 0x55ea13fe7900 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0515905b80 10 client.0 _ll_register_callbacks cb 0x55ea13ed3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0503e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13ff2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0503e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13ff2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:37728/0 (socket says 172.21.15.112:37728) 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0503e00700 1 -- 172.21.15.112:0/4041158036 learned_addr learned my addr 172.21.15.112:0/4041158036 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0509000700 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ff00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0503400700 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13fefb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0503e00700 1 -- 172.21.15.112:0/4041158036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 msgr2=0x55ea13fefb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0503e00700 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13fefb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0503e00700 1 -- 172.21.15.112:0/4041158036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 msgr2=0x55ea13ff00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0503e00700 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ff00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0503e00700 1 -- 172.21.15.112:0/4041158036 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea13e889b0 con 0x55ea13fe7900 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0509000700 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ff00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.335+0000 7f0503400700 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13fefb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.336+0000 7f0503e00700 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13ff2890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04f400c2d0 tx=0x7f04f4009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5604af0e42f06486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.336+0000 7f04ee000700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.336+0000 7f04ee000700 1 -- 172.21.15.112:0/4041158036 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f409cca0 con 0x55ea13fe7900 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.336+0000 7f04ee000700 1 -- 172.21.15.112:0/4041158036 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f04f40a5020 con 0x55ea13fe7900 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.336+0000 7f0515905b80 1 -- 172.21.15.112:0/4041158036 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04fc009490 con 0x55ea13fe7900 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.336+0000 7f04ee000700 1 -- 172.21.15.112:0/4041158036 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f04f40ba050 con 0x55ea13fe7900 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.336+0000 7f0515905b80 10 client.25495 fetch_fsmap learned FSMap version 28 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.336+0000 7f0515905b80 1 -- 172.21.15.112:0/4041158036 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea13ed1bd0 con 0x55ea13fe7900 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.336+0000 7f04ee000700 1 -- 172.21.15.112:0/4041158036 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04fc009490 con 0x55ea13fe7900 2024-04-25T18:50:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.337+0000 7f04ee000700 1 -- 172.21.15.112:0/4041158036 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f04f409bc90 con 0x55ea13fe7900 2024-04-25T18:50:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.337+0000 7f0515905b80 10 client.25495 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T18:50:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.337+0000 7f0515905b80 10 client.25495 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-25T18:50:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.337+0000 7f0515905b80 20 client.25495 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.337+0000 7f0515905b80 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x55ea14002900 0x55ea14004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.337+0000 7f0515905b80 4 client.25495 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:50:07.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.337+0000 7f0515905b80 1 -- 172.21.15.112:0/4041158036 --> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea13f48760 con 0x55ea14002900 2024-04-25T18:50:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.337+0000 7f0503400700 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x55ea14002900 0x55ea14004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.338+0000 7f0503400700 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x55ea14002900 0x55ea14004cf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=817cac3d3a2f50ff server_cookie=f5154bc5bdf206c4 in_seq=0 out_seq=0 2024-04-25T18:50:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.338+0000 7f04ee000700 10 client.25495 ms_handle_connect on v2:172.21.15.112:6840/1028859850 2024-04-25T18:50:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f04ee000700 1 -- 172.21.15.112:0/4041158036 <== mds.1 v2:172.21.15.112:6840/1028859850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea13f48760 con 0x55ea14002900 2024-04-25T18:50:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f04ee000700 10 client.25495 handle_command_reply: tid=0 2024-04-25T18:50:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 1 client.25495 shutdown 2024-04-25T18:50:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 1 -- 172.21.15.112:0/4041158036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 msgr2=0x55ea13ff2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13ff2890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04f400c2d0 tx=0x7f04f4009790 comp rx=0 tx=0).stop 2024-04-25T18:50:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 1 -- 172.21.15.112:0/4041158036 shutdown_connections 2024-04-25T18:50:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ff00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13ff2890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x55ea14002900 0x55ea14004cf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 1 --2- 172.21.15.112:0/4041158036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13fefb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 1 -- 172.21.15.112:0/4041158036 >> 172.21.15.112:0/4041158036 conn(0x55ea13f16680 msgr2=0x55ea13fe39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f04ee000700 0 client.25495 ms_handle_reset on v2:172.21.15.112:6840/1028859850 2024-04-25T18:50:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 1 -- 172.21.15.112:0/4041158036 shutdown_connections 2024-04-25T18:50:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 1 -- 172.21.15.112:0/4041158036 wait complete. 2024-04-25T18:50:07.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.339+0000 7f0515905b80 20 client.25495 trim_cache size 0 max 16384 2024-04-25T18:50:07.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.359+0000 7f0515905b80 1 Processor -- start 2024-04-25T18:50:07.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.359+0000 7f0515905b80 1 -- start start 2024-04-25T18:50:07.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.359+0000 7f0515905b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13ffea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.359+0000 7f0515905b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13ffafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.359+0000 7f0515905b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ffb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.359+0000 7f0515905b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05041b1e30 con 0x55ea13ff30d0 2024-04-25T18:50:07.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.359+0000 7f0515905b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05041af6c0 con 0x55ea13fe8220 2024-04-25T18:50:07.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.359+0000 7f0515905b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f05041ad510 con 0x55ea13fe7900 2024-04-25T18:50:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.359+0000 7f0503400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13ffafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0503400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13ffafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:43258/0 (socket says 172.21.15.112:43258) 2024-04-25T18:50:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0503400700 1 -- 172.21.15.112:0/3266685865 learned_addr learned my addr 172.21.15.112:0/3266685865 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0509000700 1 --2- 172.21.15.112:0/3266685865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ffb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0503e00700 1 --2- 172.21.15.112:0/3266685865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13ffea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0503400700 1 -- 172.21.15.112:0/3266685865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 msgr2=0x55ea13ffea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0503400700 1 --2- 172.21.15.112:0/3266685865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13ffea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0503400700 1 -- 172.21.15.112:0/3266685865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 msgr2=0x55ea13ffb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0503400700 1 --2- 172.21.15.112:0/3266685865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ffb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0503400700 1 -- 172.21.15.112:0/3266685865 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea140003d0 con 0x55ea13fe8220 2024-04-25T18:50:07.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0503e00700 1 --2- 172.21.15.112:0/3266685865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13ffea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:50:07.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0509000700 1 --2- 172.21.15.112:0/3266685865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ffb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:50:07.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f0503400700 1 --2- 172.21.15.112:0/3266685865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13ffafa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55ea13ff1160 tx=0x7f04f0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330aa5a033d5b98d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:07.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.360+0000 7f04f8c00700 1 -- 172.21.15.112:0/3266685865 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0003b80 con 0x55ea13fe8220 2024-04-25T18:50:07.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.361+0000 7f04f8c00700 1 -- 172.21.15.112:0/3266685865 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f0023020 con 0x55ea13fe8220 2024-04-25T18:50:07.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.361+0000 7f04f8c00700 1 -- 172.21.15.112:0/3266685865 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0010690 con 0x55ea13fe8220 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.362+0000 7f0515905b80 1 -- 172.21.15.112:0/3266685865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 msgr2=0x55ea13ffafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.362+0000 7f0515905b80 1 --2- 172.21.15.112:0/3266685865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13ffafa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55ea13ff1160 tx=0x7f04f0009260 comp rx=0 tx=0).stop 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.362+0000 7f0515905b80 1 -- 172.21.15.112:0/3266685865 shutdown_connections 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.362+0000 7f0515905b80 1 --2- 172.21.15.112:0/3266685865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ffb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.362+0000 7f0515905b80 1 --2- 172.21.15.112:0/3266685865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe7900 0x55ea13ffea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.362+0000 7f0515905b80 1 --2- 172.21.15.112:0/3266685865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe8220 0x55ea13ffafa0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.362+0000 7f0515905b80 1 -- 172.21.15.112:0/3266685865 >> 172.21.15.112:0/3266685865 conn(0x55ea13f16680 msgr2=0x55ea13fe2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.362+0000 7f0515905b80 1 -- 172.21.15.112:0/3266685865 shutdown_connections 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.362+0000 7f0515905b80 1 -- 172.21.15.112:0/3266685865 wait complete. 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.362+0000 7f0515905b80 1 -- start start 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.363+0000 7f0515905b80 1 Processor -- start 2024-04-25T18:50:07.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.363+0000 7f0515905b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe7900 0x55ea13ffb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.363+0000 7f0515905b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe8220 0x55ea13ff98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.363+0000 7f0515905b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ffa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.363+0000 7f0515905b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05041adb80 con 0x55ea13ff30d0 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.363+0000 7f0515905b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05041aeff0 con 0x55ea13fe7900 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.363+0000 7f0515905b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f05041ae250 con 0x55ea13fe8220 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.363+0000 7f0515905b80 10 client.0 _ll_register_callbacks cb 0x55ea13ed3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.363+0000 7f0503e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe7900 0x55ea13ffb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.363+0000 7f0503e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe7900 0x55ea13ffb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:43260/0 (socket says 172.21.15.112:43260) 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.363+0000 7f0503e00700 1 -- 172.21.15.112:0/2098591030 learned_addr learned my addr 172.21.15.112:0/2098591030 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f0509000700 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ffa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f0503400700 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe8220 0x55ea13ff98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f0503e00700 1 -- 172.21.15.112:0/2098591030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe8220 msgr2=0x55ea13ff98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f0503e00700 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe8220 0x55ea13ff98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f0503e00700 1 -- 172.21.15.112:0/2098591030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 msgr2=0x55ea13ffa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:07.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f0503e00700 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ffa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f0503e00700 1 -- 172.21.15.112:0/2098591030 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea14068e30 con 0x55ea13fe7900 2024-04-25T18:50:07.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f0503e00700 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe7900 0x55ea13ffb480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f04f400e970 tx=0x7f04f409bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=549d7d9a0ccd79e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:07.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f04ee000700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:50:07.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f04ee000700 1 -- 172.21.15.112:0/2098591030 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4098ae0 con 0x55ea13fe7900 2024-04-25T18:50:07.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f04ee000700 1 -- 172.21.15.112:0/2098591030 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f04f409caa0 con 0x55ea13fe7900 2024-04-25T18:50:07.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.364+0000 7f0515905b80 1 -- 172.21.15.112:0/2098591030 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04f0023020 con 0x55ea13fe7900 2024-04-25T18:50:07.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.365+0000 7f04ee000700 1 -- 172.21.15.112:0/2098591030 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f04f40a0bf0 con 0x55ea13fe7900 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.365+0000 7f0515905b80 10 client.25484 fetch_fsmap learned FSMap version 28 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.365+0000 7f0515905b80 1 -- 172.21.15.112:0/2098591030 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea14069f10 con 0x55ea13fe7900 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.365+0000 7f04ee000700 1 -- 172.21.15.112:0/2098591030 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f0023020 con 0x55ea13fe7900 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.365+0000 7f04ee000700 1 -- 172.21.15.112:0/2098591030 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f04f4010840 con 0x55ea13fe7900 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.365+0000 7f0515905b80 10 client.25484 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.365+0000 7f0515905b80 10 client.25484 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.365+0000 7f0515905b80 20 client.25484 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.365+0000 7f0515905b80 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x55ea1406a570 0x55ea1406c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.365+0000 7f0515905b80 4 client.25484 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-5aed9937-1-mds.a.json"}] 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.365+0000 7f0515905b80 1 -- 172.21.15.112:0/2098591030 --> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-5aed9937-1-mds.a.json"}) -- 0x55ea13f42800 con 0x55ea1406a570 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.366+0000 7f0503400700 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x55ea1406a570 0x55ea1406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.366+0000 7f0503400700 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x55ea1406a570 0x55ea1406c990 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1181000d92514002 server_cookie=5a221bf98230749 in_seq=0 out_seq=0 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.366+0000 7f04ee000700 10 client.25484 ms_handle_connect on v2:172.21.15.112:6840/1028859850 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f04ee000700 1 -- 172.21.15.112:0/2098591030 <== mds.1 v2:172.21.15.112:6840/1028859850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x55ea13f42800 con 0x55ea1406a570 2024-04-25T18:50:07.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f04ee000700 10 client.25484 handle_command_reply: tid=0 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 1 client.25484 shutdown 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 1 -- 172.21.15.112:0/2098591030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe7900 msgr2=0x55ea13ffb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe7900 0x55ea13ffb480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f04f400e970 tx=0x7f04f409bf40 comp rx=0 tx=0).stop 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 1 -- 172.21.15.112:0/2098591030 shutdown_connections 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea13ff30d0 0x55ea13ffa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.112:6840/1028859850,v1:172.21.15.112:6841/1028859850] conn(0x55ea1406a570 0x55ea1406c990 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ea13fe7900 0x55ea13ffb480 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f04ee000700 0 client.25484 ms_handle_reset on v2:172.21.15.112:6840/1028859850 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 1 --2- 172.21.15.112:0/2098591030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ea13fe8220 0x55ea13ff98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 1 -- 172.21.15.112:0/2098591030 >> 172.21.15.112:0/2098591030 conn(0x55ea13f16680 msgr2=0x55ea13fe3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 1 -- 172.21.15.112:0/2098591030 shutdown_connections 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 1 -- 172.21.15.112:0/2098591030 wait complete. 2024-04-25T18:50:07.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.368+0000 7f0515905b80 20 client.25484 trim_cache size 0 max 16384 2024-04-25T18:50:07.373 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:50:07.373 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/var/run/ceph/ops-5aed9937-1-mds.a.json" 2024-04-25T18:50:07.373 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:50:07.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 -- 172.21.15.112:0/2706896936 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f04e80a3e30 msgr2=0x7f04e80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f04e80a3e30 0x7f04e80a62f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f04fc01a000 tx=0x7f04fc027000 comp rx=0 tx=0).stop 2024-04-25T18:50:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 -- 172.21.15.112:0/2706896936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0504102ac0 msgr2=0x7f050419b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0504102ac0 0x7f050419b2d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f04f400e9a0 tx=0x7f04f400ecb0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 -- 172.21.15.112:0/2706896936 shutdown_connections 2024-04-25T18:50:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f04e80a3e30 0x7f04e80a62f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0504108a70 0x7f050419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0504102ac0 0x7f050419b2d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 --2- 172.21.15.112:0/2706896936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0504103460 0x7f050419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 -- 172.21.15.112:0/2706896936 >> 172.21.15.112:0/2706896936 conn(0x7f0504079b10 msgr2=0x7f0504100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 -- 172.21.15.112:0/2706896936 shutdown_connections 2024-04-25T18:50:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:07.369+0000 7f050b000700 1 -- 172.21.15.112:0/2706896936 wait complete. 2024-04-25T18:50:08.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:07 smithi171 ceph-mon[33266]: pgmap v85: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-04-25T18:50:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:07 smithi161 ceph-mon[32609]: pgmap v85: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-04-25T18:50:08.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:07 smithi112 ceph-mon[27864]: pgmap v85: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-04-25T18:50:08.390 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 0 (b) to a remote file /var/run/ceph/1/ops-5aed9937-1-mds.b.json 2024-04-25T18:50:08.390 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph tell mds.1:0 ops --flags=locks --path=/var/run/ceph/ops-5aed9937-1-mds.b.json 2024-04-25T18:50:08.828 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:50:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:09 smithi171 ceph-mon[33266]: pgmap v86: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T18:50:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:09 smithi161 ceph-mon[32609]: pgmap v86: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T18:50:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:09 smithi112 ceph-mon[27864]: pgmap v86: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T18:50:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.293+0000 7f9077c00700 1 -- 172.21.15.112:0/1499346307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f907006b900 msgr2=0x7f907010efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.293+0000 7f9077c00700 1 --2- 172.21.15.112:0/1499346307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f907006b900 0x7f907010efa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f905c0098f0 tx=0x7f905c009c00 comp rx=0 tx=0).stop 2024-04-25T18:50:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.294+0000 7f9077c00700 1 -- 172.21.15.112:0/1499346307 shutdown_connections 2024-04-25T18:50:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.294+0000 7f9077c00700 1 --2- 172.21.15.112:0/1499346307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f907010f4e0 0x7f90701118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.294+0000 7f9077c00700 1 --2- 172.21.15.112:0/1499346307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907006afe0 0x7f907006b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.294+0000 7f9077c00700 1 --2- 172.21.15.112:0/1499346307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f907006b900 0x7f907010efa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.294+0000 7f9077c00700 1 -- 172.21.15.112:0/1499346307 >> 172.21.15.112:0/1499346307 conn(0x7f90700788b0 msgr2=0x7f9070078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.294+0000 7f9077c00700 1 -- 172.21.15.112:0/1499346307 shutdown_connections 2024-04-25T18:50:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.294+0000 7f9077c00700 1 -- 172.21.15.112:0/1499346307 wait complete. 2024-04-25T18:50:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.295+0000 7f9077c00700 1 Processor -- start 2024-04-25T18:50:11.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.295+0000 7f9077c00700 1 -- start start 2024-04-25T18:50:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9077c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f907006afe0 0x7f907019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9077c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f907006b900 0x7f907019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9077c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907010f4e0 0x7f907019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9077c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9070114e40 con 0x7f907010f4e0 2024-04-25T18:50:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9077c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9070114cc0 con 0x7f907006b900 2024-04-25T18:50:11.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9077c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9070114fc0 con 0x7f907006afe0 2024-04-25T18:50:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9075c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907010f4e0 0x7f907019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9075200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f907006afe0 0x7f907019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9075200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f907006afe0 0x7f907019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:35820/0 (socket says 172.21.15.112:35820) 2024-04-25T18:50:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f906fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f907006b900 0x7f907019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9075c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907010f4e0 0x7f907019fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41994/0 (socket says 172.21.15.112:41994) 2024-04-25T18:50:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9075200700 1 -- 172.21.15.112:0/2299975731 learned_addr learned my addr 172.21.15.112:0/2299975731 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9075200700 1 -- 172.21.15.112:0/2299975731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f907006b900 msgr2=0x7f907019b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.296+0000 7f9075200700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f907006b900 0x7f907019b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.297+0000 7f9075200700 1 -- 172.21.15.112:0/2299975731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907010f4e0 msgr2=0x7f907019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.297+0000 7f9075200700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907010f4e0 0x7f907019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.297+0000 7f9075200700 1 -- 172.21.15.112:0/2299975731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90701a0390 con 0x7f907006afe0 2024-04-25T18:50:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.297+0000 7f9075c00700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907010f4e0 0x7f907019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:50:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.297+0000 7f9075200700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f907006afe0 0x7f907019b3d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f906400ec10 tx=0x7f906400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.297+0000 7f906d600700 1 -- 172.21.15.112:0/2299975731 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906400cc90 con 0x7f907006afe0 2024-04-25T18:50:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.297+0000 7f906d600700 1 -- 172.21.15.112:0/2299975731 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906400ce30 con 0x7f907006afe0 2024-04-25T18:50:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.297+0000 7f9077c00700 1 -- 172.21.15.112:0/2299975731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90701a0680 con 0x7f907006afe0 2024-04-25T18:50:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.297+0000 7f906d600700 1 -- 172.21.15.112:0/2299975731 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064010430 con 0x7f907006afe0 2024-04-25T18:50:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.297+0000 7f9077c00700 1 -- 172.21.15.112:0/2299975731 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701a7f00 con 0x7f907006afe0 2024-04-25T18:50:11.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.302+0000 7f906d600700 1 -- 172.21.15.112:0/2299975731 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f9064004000 con 0x7f907006afe0 2024-04-25T18:50:11.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.303+0000 7f9082508b80 1 Processor -- start 2024-04-25T18:50:11.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.303+0000 7f9082508b80 1 -- start start 2024-04-25T18:50:11.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.303+0000 7f906d600700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f90580a3db0 0x7f90580a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:11.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f9082508b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463e900 0x56469463ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:11.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f906d600700 1 -- 172.21.15.112:0/2299975731 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f90640d0a90 con 0x7f907006afe0 2024-04-25T18:50:11.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f9082508b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 0x564694649b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:11.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f9082508b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469464a0d0 0x56469464c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:11.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f9082508b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564694591f70 con 0x56469463f220 2024-04-25T18:50:11.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f9082508b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564694538810 con 0x56469463e900 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f9082508b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56469458df30 con 0x56469464a0d0 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f9075c00700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f90580a3db0 0x7f90580a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f9075200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463e900 0x56469463ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f906fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 0x564694649b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f9075200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463e900 0x56469463ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:39576/0 (socket says 172.21.15.112:39576) 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f9075200700 1 -- 172.21.15.112:0/3704424419 learned_addr learned my addr 172.21.15.112:0/3704424419 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.304+0000 7f9075c00700 1 --2- 172.21.15.112:0/3704424419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469464a0d0 0x56469464c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.305+0000 7f9075200700 1 -- 172.21.15.112:0/3704424419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469464a0d0 msgr2=0x56469464c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.305+0000 7f9075200700 1 --2- 172.21.15.112:0/3704424419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469464a0d0 0x56469464c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.305+0000 7f9075200700 1 -- 172.21.15.112:0/3704424419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 msgr2=0x564694649b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.305+0000 7f9075200700 1 --2- 172.21.15.112:0/3704424419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 0x564694649b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.305+0000 7f9075200700 1 -- 172.21.15.112:0/3704424419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56469452e760 con 0x56469463e900 2024-04-25T18:50:11.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.305+0000 7f9075c00700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f90580a3db0 0x7f90580a6270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9068009ec0 tx=0x7f9068004570 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:11.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.305+0000 7f9075200700 1 --2- 172.21.15.112:0/3704424419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463e900 0x56469463ece0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9064099400 tx=0x7f9064098e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f712f6d01feaa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:11.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.305+0000 7f9062000700 1 -- 172.21.15.112:0/3704424419 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906409d130 con 0x56469463e900 2024-04-25T18:50:11.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.305+0000 7f9062000700 1 -- 172.21.15.112:0/3704424419 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906409ca30 con 0x56469463e900 2024-04-25T18:50:11.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.305+0000 7f9062000700 1 -- 172.21.15.112:0/3704424419 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90640990a0 con 0x56469463e900 2024-04-25T18:50:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.306+0000 7f9082508b80 1 -- 172.21.15.112:0/3704424419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463e900 msgr2=0x56469463ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.306+0000 7f9082508b80 1 --2- 172.21.15.112:0/3704424419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463e900 0x56469463ece0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9064099400 tx=0x7f9064098e00 comp rx=0 tx=0).stop 2024-04-25T18:50:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.306+0000 7f9082508b80 1 -- 172.21.15.112:0/3704424419 shutdown_connections 2024-04-25T18:50:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.306+0000 7f9082508b80 1 --2- 172.21.15.112:0/3704424419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469464a0d0 0x56469464c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.306+0000 7f9082508b80 1 --2- 172.21.15.112:0/3704424419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463e900 0x56469463ece0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.306+0000 7f9082508b80 1 --2- 172.21.15.112:0/3704424419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 0x564694649b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.306+0000 7f9082508b80 1 -- 172.21.15.112:0/3704424419 >> 172.21.15.112:0/3704424419 conn(0x56469456d680 msgr2=0x56469463adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.306+0000 7f9082508b80 1 -- 172.21.15.112:0/3704424419 shutdown_connections 2024-04-25T18:50:11.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.306+0000 7f9082508b80 1 -- 172.21.15.112:0/3704424419 wait complete. 2024-04-25T18:50:11.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.307+0000 7f9082508b80 1 -- start start 2024-04-25T18:50:11.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.307+0000 7f9082508b80 1 Processor -- start 2024-04-25T18:50:11.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f9082508b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x5646946498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:11.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f9082508b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 0x564694646bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:11.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f9082508b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469464a0d0 0x564694647100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:11.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f9082508b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56469464cbf0 con 0x56469463f220 2024-04-25T18:50:11.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f9082508b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56469454b940 con 0x56469464a0d0 2024-04-25T18:50:11.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f9082508b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56469464cd70 con 0x56469463e900 2024-04-25T18:50:11.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f9082508b80 10 client.0 _ll_register_callbacks cb 0x56469452a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:11.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f906fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 0x564694646bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:11.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f906fe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 0x564694646bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42012/0 (socket says 172.21.15.112:42012) 2024-04-25T18:50:11.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f906fe00700 1 -- 172.21.15.112:0/760177489 learned_addr learned my addr 172.21.15.112:0/760177489 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:11.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f9075c00700 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469464a0d0 0x564694647100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:11.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.308+0000 7f9075200700 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x5646946498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:11.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.309+0000 7f906fe00700 1 -- 172.21.15.112:0/760177489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 msgr2=0x5646946498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.309+0000 7f906fe00700 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x5646946498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.309+0000 7f906fe00700 1 -- 172.21.15.112:0/760177489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469464a0d0 msgr2=0x564694647100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.309+0000 7f906fe00700 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469464a0d0 0x564694647100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.309+0000 7f906fe00700 1 -- 172.21.15.112:0/760177489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646944df9b0 con 0x56469463f220 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.309+0000 7f9075c00700 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469464a0d0 0x564694647100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.309+0000 7f906fe00700 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 0x564694646bc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f905c0098f0 tx=0x7f905c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537fb19bfacd8757 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.309+0000 7f9056a00700 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.309+0000 7f9056a00700 1 -- 172.21.15.112:0/760177489 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c005240 con 0x56469463f220 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.309+0000 7f9082508b80 1 -- 172.21.15.112:0/760177489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f906409ca30 con 0x56469463f220 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.309+0000 7f9056a00700 1 -- 172.21.15.112:0/760177489 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f905c0053e0 con 0x56469463f220 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.310+0000 7f9056a00700 1 -- 172.21.15.112:0/760177489 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f905c0055e0 con 0x56469463f220 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.310+0000 7f9082508b80 10 client.15351 fetch_fsmap learned FSMap version 28 2024-04-25T18:50:11.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.310+0000 7f9082508b80 1 -- 172.21.15.112:0/760177489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564694528bd0 con 0x56469463f220 2024-04-25T18:50:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.310+0000 7f9056a00700 1 -- 172.21.15.112:0/760177489 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906409ca30 con 0x56469463f220 2024-04-25T18:50:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.310+0000 7f9056a00700 1 -- 172.21.15.112:0/760177489 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f905c02a050 con 0x56469463f220 2024-04-25T18:50:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.310+0000 7f9082508b80 10 client.15351 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T18:50:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.310+0000 7f9082508b80 10 client.15351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T18:50:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.311+0000 7f9082508b80 20 client.15351 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.311+0000 7f9082508b80 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564694659a30 0x56469465be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.311+0000 7f9082508b80 4 client.15351 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:50:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.311+0000 7f9082508b80 1 -- 172.21.15.112:0/760177489 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56469459f760 con 0x564694659a30 2024-04-25T18:50:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.311+0000 7f9075200700 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564694659a30 0x56469465be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.312+0000 7f9075200700 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564694659a30 0x56469465be20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e725bbb41ff91d44 server_cookie=fa0535cc700d0535 in_seq=0 out_seq=0 2024-04-25T18:50:11.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:11.312+0000 7f9056a00700 10 client.15351 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:12.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:11 smithi171 ceph-mon[33266]: pgmap v87: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T18:50:12.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:11 smithi161 ceph-mon[32609]: pgmap v87: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T18:50:12.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:11 smithi112 ceph-mon[27864]: pgmap v87: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T18:50:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.465+0000 7f1cd7400640 1 -- 172.21.15.112:0/1369503483 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560cd4bc2320 con 0x560cd4cd9170 2024-04-25T18:50:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cd7400640 10 client.15294 handle_command_reply: tid=0 2024-04-25T18:50:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cff61e740 1 client.15294 shutdown 2024-04-25T18:50:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cff61e740 1 -- 172.21.15.112:0/1369503483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 msgr2=0x560cd4c5cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cff61e740 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c5cf30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40ff1b0 tx=0x7f1ce40046c0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f9056a00700 1 -- 172.21.15.112:0/760177489 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56469459f760 con 0x564694659a30 2024-04-25T18:50:13.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f9056a00700 10 client.15351 handle_command_reply: tid=0 2024-04-25T18:50:13.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f9082508b80 1 client.15351 shutdown 2024-04-25T18:50:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cff61e740 1 -- 172.21.15.112:0/1369503483 shutdown_connections 2024-04-25T18:50:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cff61e740 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cd4c59a00 0x560cd4c5d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cff61e740 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560cd4cd9170 0x560cd4cdb590 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cff61e740 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560cd4c4f140 0x560cd4c5cf30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cff61e740 1 --2- 172.21.15.112:0/1369503483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560cd4c4e820 0x560cd4c61bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cff61e740 1 -- 172.21.15.112:0/1369503483 >> 172.21.15.112:0/1369503483 conn(0x560cd4c4ad30 msgr2=0x560cd4c49e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cd7400640 0 client.15294 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cff61e740 1 -- 172.21.15.112:0/1369503483 shutdown_connections 2024-04-25T18:50:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f1cff61e740 1 -- 172.21.15.112:0/1369503483 wait complete. 2024-04-25T18:50:13.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f9082508b80 1 -- 172.21.15.112:0/760177489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 msgr2=0x564694646bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:13.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.466+0000 7f9082508b80 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 0x564694646bc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f905c0098f0 tx=0x7f905c00f690 comp rx=0 tx=0).stop 2024-04-25T18:50:13.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.467+0000 7f9082508b80 1 -- 172.21.15.112:0/760177489 shutdown_connections 2024-04-25T18:50:13.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.467+0000 7f9082508b80 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564694659a30 0x56469465be20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.467+0000 7f9082508b80 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469464a0d0 0x564694647100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.467+0000 7f1cff61e740 20 client.15294 trim_cache size 0 max 16384 2024-04-25T18:50:13.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.467+0000 7f9082508b80 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x5646946498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.467+0000 7f9082508b80 1 --2- 172.21.15.112:0/760177489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469463f220 0x564694646bc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.467+0000 7f9082508b80 1 -- 172.21.15.112:0/760177489 >> 172.21.15.112:0/760177489 conn(0x56469456d680 msgr2=0x56469463a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:13.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.467+0000 7f9056a00700 0 client.15351 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:13.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.467+0000 7f9082508b80 1 -- 172.21.15.112:0/760177489 shutdown_connections 2024-04-25T18:50:13.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.467+0000 7f9082508b80 1 -- 172.21.15.112:0/760177489 wait complete. 2024-04-25T18:50:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:50:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:50:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:50:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:50:13.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.467+0000 7f9082508b80 20 client.15351 trim_cache size 0 max 16384 2024-04-25T18:50:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 -- 172.21.15.112:0/3085764058 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f1cdc0a3dc0 msgr2=0x7f1cdc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f1cdc0a3dc0 0x7f1cdc0a6280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x560cd4c4c150 tx=0x7f1ce4004730 comp rx=0 tx=0).stop 2024-04-25T18:50:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 -- 172.21.15.112:0/3085764058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf410ebd0 msgr2=0x7f1cf419a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf410ebd0 0x7f1cf419a8b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009fd0 tx=0x7f1cf000ef10 comp rx=0 tx=0).stop 2024-04-25T18:50:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 -- 172.21.15.112:0/3085764058 shutdown_connections 2024-04-25T18:50:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f1cdc0a3dc0 0x7f1cdc0a6280 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf410ebd0 0x7f1cf419a8b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1cf4103050 0x7f1cf41a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 --2- 172.21.15.112:0/3085764058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cf4100990 0x7f1cf41a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 -- 172.21.15.112:0/3085764058 >> 172.21.15.112:0/3085764058 conn(0x7f1cf40773d0 msgr2=0x7f1cf4110070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 -- 172.21.15.112:0/3085764058 shutdown_connections 2024-04-25T18:50:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:13.468+0000 7f1cfc800640 1 -- 172.21.15.112:0/3085764058 wait complete. 2024-04-25T18:50:13.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.489+0000 7f9082508b80 1 Processor -- start 2024-04-25T18:50:13.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.489+0000 7f9082508b80 1 -- start start 2024-04-25T18:50:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.490+0000 7f9082508b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x564694645b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.490+0000 7f9082508b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x5646946616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:13.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.490+0000 7f9082508b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469464a0d0 0x564694661c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.490+0000 7f9082508b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b1db0 con 0x56469464a0d0 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.490+0000 7f9082508b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90701af640 con 0x56469463f220 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.490+0000 7f9082508b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f90701ad490 con 0x56469463e900 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.490+0000 7f906fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x5646946616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.490+0000 7f906fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x5646946616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:39604/0 (socket says 172.21.15.112:39604) 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.490+0000 7f906fe00700 1 -- 172.21.15.112:0/407158147 learned_addr learned my addr 172.21.15.112:0/407158147 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.490+0000 7f9075c00700 1 --2- 172.21.15.112:0/407158147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469464a0d0 0x564694661c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.490+0000 7f9075200700 1 --2- 172.21.15.112:0/407158147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x564694645b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.491+0000 7f906fe00700 1 -- 172.21.15.112:0/407158147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 msgr2=0x564694645b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.491+0000 7f906fe00700 1 --2- 172.21.15.112:0/407158147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x564694645b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.491+0000 7f906fe00700 1 -- 172.21.15.112:0/407158147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469464a0d0 msgr2=0x564694661c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.491+0000 7f906fe00700 1 --2- 172.21.15.112:0/407158147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469464a0d0 0x564694661c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.491+0000 7f906fe00700 1 -- 172.21.15.112:0/407158147 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564694649500 con 0x56469463f220 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.491+0000 7f906fe00700 1 --2- 172.21.15.112:0/407158147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x5646946616d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f905c009810 tx=0x7f905c009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cffb6fb7c8f5fa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.491+0000 7f9061600700 1 -- 172.21.15.112:0/407158147 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c038070 con 0x56469463f220 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.491+0000 7f9061600700 1 -- 172.21.15.112:0/407158147 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905c0058e0 con 0x56469463f220 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.491+0000 7f9061600700 1 -- 172.21.15.112:0/407158147 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c02f030 con 0x56469463f220 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.492+0000 7f9082508b80 1 -- 172.21.15.112:0/407158147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 msgr2=0x5646946616d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.492+0000 7f9082508b80 1 --2- 172.21.15.112:0/407158147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x5646946616d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f905c009810 tx=0x7f905c009290 comp rx=0 tx=0).stop 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.492+0000 7f9082508b80 1 -- 172.21.15.112:0/407158147 shutdown_connections 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.492+0000 7f9082508b80 1 --2- 172.21.15.112:0/407158147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469464a0d0 0x564694661c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.492+0000 7f9082508b80 1 --2- 172.21.15.112:0/407158147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x564694645b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.492+0000 7f9082508b80 1 --2- 172.21.15.112:0/407158147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x5646946616d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.492+0000 7f9082508b80 1 -- 172.21.15.112:0/407158147 >> 172.21.15.112:0/407158147 conn(0x56469456d680 msgr2=0x564694648c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.492+0000 7f9082508b80 1 -- 172.21.15.112:0/407158147 shutdown_connections 2024-04-25T18:50:13.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.492+0000 7f9082508b80 1 -- 172.21.15.112:0/407158147 wait complete. 2024-04-25T18:50:13.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.493+0000 7f9082508b80 1 -- start start 2024-04-25T18:50:13.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.493+0000 7f9082508b80 1 Processor -- start 2024-04-25T18:50:13.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f9082508b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x5646946501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:13.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f9082508b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x564694650700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:13.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f9082508b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469464a0d0 0x5646946bef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:13.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f9082508b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90701adb00 con 0x56469464a0d0 2024-04-25T18:50:13.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f9082508b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90701aef70 con 0x56469463f220 2024-04-25T18:50:13.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f9082508b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f90701ae1d0 con 0x56469463e900 2024-04-25T18:50:13.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f9082508b80 10 client.0 _ll_register_callbacks cb 0x56469452a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:13.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f906fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x564694650700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f906fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x564694650700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:39618/0 (socket says 172.21.15.112:39618) 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f906fe00700 1 -- 172.21.15.112:0/799170498 learned_addr learned my addr 172.21.15.112:0/799170498 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f9075200700 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x5646946501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f9075c00700 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469464a0d0 0x5646946bef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f906fe00700 1 -- 172.21.15.112:0/799170498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 msgr2=0x5646946501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f906fe00700 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x5646946501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f906fe00700 1 -- 172.21.15.112:0/799170498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469464a0d0 msgr2=0x5646946bef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.494+0000 7f906fe00700 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469464a0d0 0x5646946bef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.495+0000 7f906fe00700 1 -- 172.21.15.112:0/799170498 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646946bf4e0 con 0x56469463f220 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.495+0000 7f9075200700 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x5646946501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.495+0000 7f9075c00700 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469464a0d0 0x5646946bef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.495+0000 7f906fe00700 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x564694650700 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f905c005fe0 tx=0x7f905c005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16ede41a5c7dd855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.495+0000 7f9056a00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.495+0000 7f9056a00700 1 -- 172.21.15.112:0/799170498 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c02f030 con 0x56469463f220 2024-04-25T18:50:13.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.495+0000 7f9056a00700 1 -- 172.21.15.112:0/799170498 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f905c0053d0 con 0x56469463f220 2024-04-25T18:50:13.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.495+0000 7f9082508b80 1 -- 172.21.15.112:0/799170498 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f905c0058e0 con 0x56469463f220 2024-04-25T18:50:13.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9056a00700 1 -- 172.21.15.112:0/799170498 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f905c038040 con 0x56469463f220 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9082508b80 10 client.25502 fetch_fsmap learned FSMap version 28 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9082508b80 1 -- 172.21.15.112:0/799170498 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646946c07c0 con 0x56469463f220 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9056a00700 1 -- 172.21.15.112:0/799170498 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905c0058e0 con 0x56469463f220 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9056a00700 1 -- 172.21.15.112:0/799170498 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f905c02a070 con 0x56469463f220 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9082508b80 10 client.25502 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9082508b80 10 client.25502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9082508b80 20 client.25502 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9082508b80 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5646946c0e80 0x5646946c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9082508b80 4 client.25502 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-5aed9937-1-mds.b.json"}] 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9082508b80 1 -- 172.21.15.112:0/799170498 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-5aed9937-1-mds.b.json"}) -- 0x564694599800 con 0x5646946c0e80 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.496+0000 7f9075200700 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5646946c0e80 0x5646946c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.497+0000 7f9056a00700 10 client.25502 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.497+0000 7f9075200700 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5646946c0e80 0x5646946c32a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70312937967e3c server_cookie=249c64719c9bc629 in_seq=0 out_seq=0 2024-04-25T18:50:13.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9056a00700 1 -- 172.21.15.112:0/799170498 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x564694599800 con 0x5646946c0e80 2024-04-25T18:50:13.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9056a00700 10 client.25502 handle_command_reply: tid=0 2024-04-25T18:50:13.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9082508b80 1 client.25502 shutdown 2024-04-25T18:50:13.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9082508b80 1 -- 172.21.15.112:0/799170498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 msgr2=0x564694650700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:13.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9082508b80 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x564694650700 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f905c005fe0 tx=0x7f905c005b40 comp rx=0 tx=0).stop 2024-04-25T18:50:13.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9082508b80 1 -- 172.21.15.112:0/799170498 shutdown_connections 2024-04-25T18:50:13.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9082508b80 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5646946c0e80 0x5646946c32a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9082508b80 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56469464a0d0 0x5646946bef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9082508b80 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56469463e900 0x5646946501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9082508b80 1 --2- 172.21.15.112:0/799170498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56469463f220 0x564694650700 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9056a00700 0 client.25502 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9082508b80 1 -- 172.21.15.112:0/799170498 >> 172.21.15.112:0/799170498 conn(0x56469456d680 msgr2=0x564694639ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.536+0000 7f9082508b80 1 -- 172.21.15.112:0/799170498 shutdown_connections 2024-04-25T18:50:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.537+0000 7f9082508b80 1 -- 172.21.15.112:0/799170498 wait complete. 2024-04-25T18:50:13.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.537+0000 7f9082508b80 20 client.25502 trim_cache size 0 max 16384 2024-04-25T18:50:13.539 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:50:13.539 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/var/run/ceph/ops-5aed9937-1-mds.b.json" 2024-04-25T18:50:13.539 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.538+0000 7f9077c00700 1 -- 172.21.15.112:0/2299975731 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f90580a3db0 msgr2=0x7f90580a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.538+0000 7f9077c00700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f90580a3db0 0x7f90580a6270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9068009ec0 tx=0x7f9068004570 comp rx=0 tx=0).stop 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.538+0000 7f9077c00700 1 -- 172.21.15.112:0/2299975731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f907006afe0 msgr2=0x7f907019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.538+0000 7f9077c00700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f907006afe0 0x7f907019b3d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f906400ec10 tx=0x7f906400ef20 comp rx=0 tx=0).stop 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.538+0000 7f9077c00700 1 -- 172.21.15.112:0/2299975731 shutdown_connections 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.538+0000 7f9077c00700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f90580a3db0 0x7f90580a6270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.538+0000 7f9077c00700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f907010f4e0 0x7f907019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.538+0000 7f9077c00700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f907006afe0 0x7f907019b3d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.538+0000 7f9077c00700 1 --2- 172.21.15.112:0/2299975731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f907006b900 0x7f907019b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.539+0000 7f9077c00700 1 -- 172.21.15.112:0/2299975731 >> 172.21.15.112:0/2299975731 conn(0x7f90700788b0 msgr2=0x7f9070110a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.539+0000 7f9077c00700 1 -- 172.21.15.112:0/2299975731 shutdown_connections 2024-04-25T18:50:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:13.539+0000 7f9077c00700 1 -- 172.21.15.112:0/2299975731 wait complete. 2024-04-25T18:50:14.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:13 smithi171 ceph-mon[33266]: pgmap v88: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:50:14.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:13 smithi161 ceph-mon[32609]: pgmap v88: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:50:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:13 smithi112 ceph-mon[27864]: pgmap v88: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:50:14.570 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-04-25T18:50:14.589 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.gfYYd1TaIs 2024-04-25T18:50:14.589 DEBUG:teuthology.orchestra.run.smithi171:> sudo cp /var/run/ceph/1/ops-5aed9937-1-mds.i.json /tmp/tmp.gfYYd1TaIs 2024-04-25T18:50:14.658 INFO:teuthology.orchestra.run.smithi171.stderr:cp: cannot stat '/var/run/ceph/1/ops-5aed9937-1-mds.i.json': No such file or directory 2024-04-25T18:50:14.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:50:14.659 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't pull ops dump at '/var/run/ceph/1/ops-5aed9937-1-mds.i.json' on rank 2 (i), error: Command failed on smithi171 with status 1: 'sudo cp /var/run/ceph/1/ops-5aed9937-1-mds.i.json /tmp/tmp.gfYYd1TaIs' 2024-04-25T18:50:14.660 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-04-25T18:50:14.718 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.a291xAYfXh 2024-04-25T18:50:14.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /var/run/ceph/1/ops-5aed9937-1-mds.a.json /tmp/tmp.a291xAYfXh 2024-04-25T18:50:14.874 INFO:teuthology.orchestra.run.smithi112.stderr:cp: cannot stat '/var/run/ceph/1/ops-5aed9937-1-mds.a.json': No such file or directory 2024-04-25T18:50:14.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:50:14.878 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't pull ops dump at '/var/run/ceph/1/ops-5aed9937-1-mds.a.json' on rank 1 (a), error: Command failed on smithi112 with status 1: 'sudo cp /var/run/ceph/1/ops-5aed9937-1-mds.a.json /tmp/tmp.a291xAYfXh' 2024-04-25T18:50:14.879 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-04-25T18:50:14.893 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.0YfvSPpDnN 2024-04-25T18:50:14.894 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /var/run/ceph/1/ops-5aed9937-1-mds.b.json /tmp/tmp.0YfvSPpDnN 2024-04-25T18:50:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:cp: cannot stat '/var/run/ceph/1/ops-5aed9937-1-mds.b.json': No such file or directory 2024-04-25T18:50:14.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:50:14.962 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't pull ops dump at '/var/run/ceph/1/ops-5aed9937-1-mds.b.json' on rank 0 (b), error: Command failed on smithi161 with status 1: 'sudo cp /var/run/ceph/1/ops-5aed9937-1-mds.b.json /tmp/tmp.0YfvSPpDnN' 2024-04-25T18:50:14.963 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:50:15.418 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:50:16.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:15 smithi171 ceph-mon[33266]: pgmap v89: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:50:16.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:15 smithi161 ceph-mon[32609]: pgmap v89: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:50:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:15 smithi112 ceph-mon[27864]: pgmap v89: 97 pgs: 97 active+clean; 21 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T18:50:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.840+0000 7fce11600700 1 -- 172.21.15.112:0/3040194382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c10f440 msgr2=0x7fce0c111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.840+0000 7fce11600700 1 --2- 172.21.15.112:0/3040194382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c10f440 0x7fce0c111830 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fce0400b0a0 tx=0x7fce0400b3b0 comp rx=0 tx=0).stop 2024-04-25T18:50:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.840+0000 7fce11600700 1 -- 172.21.15.112:0/3040194382 shutdown_connections 2024-04-25T18:50:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.840+0000 7fce11600700 1 --2- 172.21.15.112:0/3040194382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c10f440 0x7fce0c111830 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.840+0000 7fce11600700 1 --2- 172.21.15.112:0/3040194382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce0c074c70 0x7fce0c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.840+0000 7fce11600700 1 --2- 172.21.15.112:0/3040194382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce0c075590 0x7fce0c10ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.840+0000 7fce11600700 1 -- 172.21.15.112:0/3040194382 >> 172.21.15.112:0/3040194382 conn(0x7fce0c079ba0 msgr2=0x7fce0c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.840+0000 7fce11600700 1 -- 172.21.15.112:0/3040194382 shutdown_connections 2024-04-25T18:50:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.840+0000 7fce11600700 1 -- 172.21.15.112:0/3040194382 wait complete. 2024-04-25T18:50:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.841+0000 7fce11600700 1 Processor -- start 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.841+0000 7fce11600700 1 -- start start 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.842+0000 7fce11600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce0c074c70 0x7fce0c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.842+0000 7fce11600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c075590 0x7fce0c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.842+0000 7fce11600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce0c10f440 0x7fce0c19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.842+0000 7fce11600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c114df0 con 0x7fce0c075590 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.842+0000 7fce11600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c114c70 con 0x7fce0c074c70 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.842+0000 7fce11600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c114f70 con 0x7fce0c10f440 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.842+0000 7fce0aa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce0c074c70 0x7fce0c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.842+0000 7fce0a000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c075590 0x7fce0c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.842+0000 7fce0aa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce0c074c70 0x7fce0c19b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:39624/0 (socket says 172.21.15.112:39624) 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.842+0000 7fce0aa00700 1 -- 172.21.15.112:0/504618683 learned_addr learned my addr 172.21.15.112:0/504618683 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.842+0000 7fce0b400700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce0c10f440 0x7fce0c19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.843+0000 7fce0aa00700 1 -- 172.21.15.112:0/504618683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce0c10f440 msgr2=0x7fce0c19fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.843+0000 7fce0aa00700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce0c10f440 0x7fce0c19fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.843+0000 7fce0aa00700 1 -- 172.21.15.112:0/504618683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c075590 msgr2=0x7fce0c19b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.843+0000 7fce0aa00700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c075590 0x7fce0c19b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.843+0000 7fce0aa00700 1 -- 172.21.15.112:0/504618683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c1a0280 con 0x7fce0c074c70 2024-04-25T18:50:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.843+0000 7fce0aa00700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce0c074c70 0x7fce0c19b2c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00e9c0 tx=0x7fcdfc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:17.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.843+0000 7fce03400700 1 -- 172.21.15.112:0/504618683 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc00cbd0 con 0x7fce0c074c70 2024-04-25T18:50:17.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.843+0000 7fce11600700 1 -- 172.21.15.112:0/504618683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c1a0570 con 0x7fce0c074c70 2024-04-25T18:50:17.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.843+0000 7fce03400700 1 -- 172.21.15.112:0/504618683 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdfc00ee60 con 0x7fce0c074c70 2024-04-25T18:50:17.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.843+0000 7fce03400700 1 -- 172.21.15.112:0/504618683 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0055c0 con 0x7fce0c074c70 2024-04-25T18:50:17.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.843+0000 7fce11600700 1 -- 172.21.15.112:0/504618683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c1a7df0 con 0x7fce0c074c70 2024-04-25T18:50:17.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.845+0000 7fce11600700 1 -- 172.21.15.112:0/504618683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0c069730 con 0x7fce0c074c70 2024-04-25T18:50:17.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.846+0000 7fce03400700 1 -- 172.21.15.112:0/504618683 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fcdfc004750 con 0x7fce0c074c70 2024-04-25T18:50:17.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.848+0000 7fce03400700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcdf00a3db0 0x7fcdf00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.848+0000 7fce03400700 1 -- 172.21.15.112:0/504618683 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fcdfc0d0b20 con 0x7fce0c074c70 2024-04-25T18:50:17.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.848+0000 7fce0a000700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcdf00a3db0 0x7fcdf00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:17.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.849+0000 7fce0a000700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcdf00a3db0 0x7fcdf00a6270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009680 tx=0x7fcdf8009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:17.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:17.852+0000 7fce03400700 1 -- 172.21.15.112:0/504618683 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcdfc09d160 con 0x7fce0c074c70 2024-04-25T18:50:17.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:17 smithi112 ceph-mon[27864]: pgmap v90: 97 pgs: 97 active+clean; 21 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 86 KiB/s wr, 36 op/s 2024-04-25T18:50:17.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:17 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:50:18.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.150+0000 7fce11600700 1 -- 172.21.15.112:0/504618683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce0c02df80 con 0x7fce0c074c70 2024-04-25T18:50:18.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.151+0000 7fce03400700 1 -- 172.21.15.112:0/504618683 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11894 (secure 0 0 0) 0x7fcdfc09ca60 con 0x7fce0c074c70 2024-04-25T18:50:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:50:18.153 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:50:16.419511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25459":{"gid":25459,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3485388562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3485388562},{"type":"v1","addr":"172.21.15.112:6839","nonce":3485388562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24493,24511,24512]},"id":1}]} 2024-04-25T18:50:18.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.154+0000 7fce11600700 1 -- 172.21.15.112:0/504618683 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcdf00a3db0 msgr2=0x7fcdf00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:18.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.154+0000 7fce11600700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcdf00a3db0 0x7fcdf00a6270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009680 tx=0x7fcdf8009260 comp rx=0 tx=0).stop 2024-04-25T18:50:18.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.154+0000 7fce11600700 1 -- 172.21.15.112:0/504618683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce0c074c70 msgr2=0x7fce0c19b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:18.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.154+0000 7fce11600700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce0c074c70 0x7fce0c19b2c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00e9c0 tx=0x7fcdfc00ecd0 comp rx=0 tx=0).stop 2024-04-25T18:50:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.154+0000 7fce11600700 1 -- 172.21.15.112:0/504618683 shutdown_connections 2024-04-25T18:50:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.154+0000 7fce11600700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcdf00a3db0 0x7fcdf00a6270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.154+0000 7fce11600700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce0c10f440 0x7fce0c19fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.154+0000 7fce11600700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce0c074c70 0x7fce0c19b2c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.154+0000 7fce11600700 1 --2- 172.21.15.112:0/504618683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c075590 0x7fce0c19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.154+0000 7fce11600700 1 -- 172.21.15.112:0/504618683 >> 172.21.15.112:0/504618683 conn(0x7fce0c079ba0 msgr2=0x7fce0c0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.155+0000 7fce11600700 1 -- 172.21.15.112:0/504618683 shutdown_connections 2024-04-25T18:50:18.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:18.155+0000 7fce11600700 1 -- 172.21.15.112:0/504618683 wait complete. 2024-04-25T18:50:18.157 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-04-25T18:50:18.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:17 smithi171 ceph-mon[33266]: pgmap v90: 97 pgs: 97 active+clean; 21 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 86 KiB/s wr, 36 op/s 2024-04-25T18:50:18.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:17 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:50:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:17 smithi161 ceph-mon[32609]: pgmap v90: 97 pgs: 97 active+clean; 21 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 86 KiB/s wr, 36 op/s 2024-04-25T18:50:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:17 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:50:19.123 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:18 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/504618683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:50:19.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:18 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/504618683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:50:19.186 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24512 quiesce db --set-id 5aed9937 --await-for 42.0' 2024-04-25T18:50:19.186 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24512 quiesce db --set-id 5aed9937 --await-for 42.0 2024-04-25T18:50:19.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:18 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/504618683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:50:19.692 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:50:20.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:19 smithi112 ceph-mon[27864]: pgmap v91: 97 pgs: 97 active+clean; 21 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 85 KiB/s wr, 33 op/s 2024-04-25T18:50:20.122 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:19 smithi161 ceph-mon[32609]: pgmap v91: 97 pgs: 97 active+clean; 21 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 85 KiB/s wr, 33 op/s 2024-04-25T18:50:20.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:19 smithi171 ceph-mon[33266]: pgmap v91: 97 pgs: 97 active+clean; 21 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 85 KiB/s wr, 33 op/s 2024-04-25T18:50:21.889 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:21 smithi112 ceph-mon[27864]: pgmap v92: 97 pgs: 97 active+clean; 27 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 671 KiB/s wr, 56 op/s 2024-04-25T18:50:22.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:21 smithi171 ceph-mon[33266]: pgmap v92: 97 pgs: 97 active+clean; 27 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 671 KiB/s wr, 56 op/s 2024-04-25T18:50:22.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.212+0000 7f65f8800700 1 -- 172.21.15.112:0/865932622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f0104ec0 msgr2=0x7f65f01052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:22.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.212+0000 7f65f8800700 1 --2- 172.21.15.112:0/865932622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f0104ec0 0x7f65f01052a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f65e40099c0 tx=0x7f65e4009cd0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.212+0000 7f65f8800700 1 -- 172.21.15.112:0/865932622 shutdown_connections 2024-04-25T18:50:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.212+0000 7f65f8800700 1 --2- 172.21.15.112:0/865932622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f0109fa0 0x7f65f0111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.212+0000 7f65f8800700 1 --2- 172.21.15.112:0/865932622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f0104ec0 0x7f65f01052a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.212+0000 7f65f8800700 1 --2- 172.21.15.112:0/865932622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f0105870 0x7f65f01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.212+0000 7f65f8800700 1 -- 172.21.15.112:0/865932622 >> 172.21.15.112:0/865932622 conn(0x7f65f0100750 msgr2=0x7f65f0102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.213+0000 7f65f8800700 1 -- 172.21.15.112:0/865932622 shutdown_connections 2024-04-25T18:50:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.213+0000 7f65f8800700 1 -- 172.21.15.112:0/865932622 wait complete. 2024-04-25T18:50:22.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.214+0000 7f65f8800700 1 Processor -- start 2024-04-25T18:50:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.214+0000 7f65f8800700 1 -- start start 2024-04-25T18:50:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.214+0000 7f65f8800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f0104ec0 0x7f65f019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.214+0000 7f65f8800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f0105870 0x7f65f019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.214+0000 7f65f8800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f0109fa0 0x7f65f019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.214+0000 7f65f8800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65f0114ea0 con 0x7f65f0104ec0 2024-04-25T18:50:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.214+0000 7f65f8800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65f0114d20 con 0x7f65f0105870 2024-04-25T18:50:22.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.214+0000 7f65f8800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f65f0115020 con 0x7f65f0109fa0 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.215+0000 7f65f5400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f0105870 0x7f65f019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.215+0000 7f65f5400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f0105870 0x7f65f019b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48696/0 (socket says 172.21.15.112:48696) 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.215+0000 7f65f5400700 1 -- 172.21.15.112:0/3245749930 learned_addr learned my addr 172.21.15.112:0/3245749930 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.215+0000 7f65f5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f0104ec0 0x7f65f019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.215+0000 7f65f6800700 1 --2- 172.21.15.112:0/3245749930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f0109fa0 0x7f65f019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.215+0000 7f65f5400700 1 -- 172.21.15.112:0/3245749930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f0109fa0 msgr2=0x7f65f019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.215+0000 7f65f5400700 1 --2- 172.21.15.112:0/3245749930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f0109fa0 0x7f65f019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.215+0000 7f65f5400700 1 -- 172.21.15.112:0/3245749930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f0104ec0 msgr2=0x7f65f019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.215+0000 7f65f5400700 1 --2- 172.21.15.112:0/3245749930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f0104ec0 0x7f65f019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.215+0000 7f65f5400700 1 -- 172.21.15.112:0/3245749930 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f01a03a0 con 0x7f65f0105870 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.216+0000 7f65f5400700 1 --2- 172.21.15.112:0/3245749930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f0105870 0x7f65f019b8d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f65e000e970 tx=0x7f65e000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.216+0000 7f65eaa00700 1 -- 172.21.15.112:0/3245749930 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0004a30 con 0x7f65f0105870 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.216+0000 7f65eaa00700 1 -- 172.21.15.112:0/3245749930 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e000cc90 con 0x7f65f0105870 2024-04-25T18:50:22.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.216+0000 7f65f8800700 1 -- 172.21.15.112:0/3245749930 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f01a0690 con 0x7f65f0105870 2024-04-25T18:50:22.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.216+0000 7f65eaa00700 1 -- 172.21.15.112:0/3245749930 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0005610 con 0x7f65f0105870 2024-04-25T18:50:22.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.216+0000 7f65f8800700 1 -- 172.21.15.112:0/3245749930 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f01a7f10 con 0x7f65f0105870 2024-04-25T18:50:22.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.220+0000 7f65eaa00700 1 -- 172.21.15.112:0/3245749930 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f65e000c7e0 con 0x7f65f0105870 2024-04-25T18:50:22.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:21 smithi161 ceph-mon[32609]: pgmap v92: 97 pgs: 97 active+clean; 27 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 671 KiB/s wr, 56 op/s 2024-04-25T18:50:22.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.221+0000 7f660324ab80 1 Processor -- start 2024-04-25T18:50:22.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.221+0000 7f660324ab80 1 -- start start 2024-04-25T18:50:22.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f660324ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f660324ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f660324ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f660324ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f92932cf90 con 0x55f9293d9d40 2024-04-25T18:50:22.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f660324ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f9292d3830 con 0x55f9293e54d0 2024-04-25T18:50:22.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f660324ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f929328f50 con 0x55f9293da660 2024-04-25T18:50:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f65f6800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f65f5e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f65f5e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293e4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:39342/0 (socket says 172.21.15.112:39342) 2024-04-25T18:50:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f65eaa00700 1 --2- 172.21.15.112:0/3245749930 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f65d80a3db0 0x7f65d80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f65f5e00700 1 -- 172.21.15.112:0/1935561704 learned_addr learned my addr 172.21.15.112:0/1935561704 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f65f6800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293da120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55736/0 (socket says 172.21.15.112:55736) 2024-04-25T18:50:22.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.222+0000 7f65f5400700 1 --2- 172.21.15.112:0/1935561704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.223+0000 7f65f6800700 1 --2- 172.21.15.112:0/3245749930 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f65d80a3db0 0x7f65d80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.223+0000 7f65eaa00700 1 -- 172.21.15.112:0/3245749930 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f65e00d0a10 con 0x7f65f0105870 2024-04-25T18:50:22.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.223+0000 7f65f5400700 1 -- 172.21.15.112:0/1935561704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 msgr2=0x55f9293e4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:22.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.223+0000 7f65f5400700 1 --2- 172.21.15.112:0/1935561704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293e4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.223+0000 7f65f5400700 1 -- 172.21.15.112:0/1935561704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 msgr2=0x55f9293da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.223+0000 7f65f5400700 1 --2- 172.21.15.112:0/1935561704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293da120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.223+0000 7f65f5400700 1 -- 172.21.15.112:0/1935561704 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9292c9780 con 0x55f9293e54d0 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.223+0000 7f65f5400700 1 --2- 172.21.15.112:0/1935561704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293e78c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65e00ce100 tx=0x7f65e0004bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd45c640aabebde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.223+0000 7f65df400700 1 -- 172.21.15.112:0/1935561704 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e009c720 con 0x55f9293e54d0 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.223+0000 7f65df400700 1 -- 172.21.15.112:0/1935561704 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e000cad0 con 0x55f9293e54d0 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.224+0000 7f65df400700 1 -- 172.21.15.112:0/1935561704 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e00a55c0 con 0x55f9293e54d0 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.224+0000 7f660324ab80 1 -- 172.21.15.112:0/1935561704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 msgr2=0x55f9293e78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.224+0000 7f660324ab80 1 --2- 172.21.15.112:0/1935561704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293e78c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65e00ce100 tx=0x7f65e0004bd0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.224+0000 7f65f6800700 1 --2- 172.21.15.112:0/3245749930 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f65d80a3db0 0x7f65d80a6270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009ec0 tx=0x7f65ec004b40 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.224+0000 7f660324ab80 1 -- 172.21.15.112:0/1935561704 shutdown_connections 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.224+0000 7f660324ab80 1 --2- 172.21.15.112:0/1935561704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293e78c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.224+0000 7f660324ab80 1 --2- 172.21.15.112:0/1935561704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.224+0000 7f660324ab80 1 --2- 172.21.15.112:0/1935561704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293e4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.224+0000 7f660324ab80 1 -- 172.21.15.112:0/1935561704 >> 172.21.15.112:0/1935561704 conn(0x55f929309050 msgr2=0x55f9293d6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:22.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.225+0000 7f660324ab80 1 -- 172.21.15.112:0/1935561704 shutdown_connections 2024-04-25T18:50:22.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.225+0000 7f660324ab80 1 -- 172.21.15.112:0/1935561704 wait complete. 2024-04-25T18:50:22.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.225+0000 7f660324ab80 1 -- start start 2024-04-25T18:50:22.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.225+0000 7f660324ab80 1 Processor -- start 2024-04-25T18:50:22.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.226+0000 7f660324ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293e4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.226+0000 7f660324ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293e1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.226+0000 7f660324ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293e2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.226+0000 7f660324ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9293e7ea0 con 0x55f9293d9d40 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.226+0000 7f660324ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f9292e6960 con 0x55f9293e54d0 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.226+0000 7f660324ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f9293e8020 con 0x55f9293da660 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.226+0000 7f660324ab80 10 client.0 _ll_register_callbacks cb 0x55f9292c5820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.226+0000 7f65f6800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.226+0000 7f65f6800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293e2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48714/0 (socket says 172.21.15.112:48714) 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.226+0000 7f65f6800700 1 -- 172.21.15.112:0/1005187476 learned_addr learned my addr 172.21.15.112:0/1005187476 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.226+0000 7f65f5400700 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293e1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.227+0000 7f65f5e00700 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293e4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.227+0000 7f65f6800700 1 -- 172.21.15.112:0/1005187476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 msgr2=0x55f9293e1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:22.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.227+0000 7f65f6800700 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293e1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.227+0000 7f65f6800700 1 -- 172.21.15.112:0/1005187476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 msgr2=0x55f9293e4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.227+0000 7f65f6800700 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293e4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.227+0000 7f65f6800700 1 -- 172.21.15.112:0/1005187476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92927a9d0 con 0x55f9293e54d0 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.227+0000 7f65f6800700 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293e2530 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65ec018ca0 tx=0x7f65ec004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81c07a8680588f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.227+0000 7f65d3e00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.227+0000 7f65d3e00700 1 -- 172.21.15.112:0/1005187476 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec003cd0 con 0x55f9293e54d0 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f65d3e00700 1 -- 172.21.15.112:0/1005187476 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f65ec0079c0 con 0x55f9293e54d0 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f660324ab80 1 -- 172.21.15.112:0/1005187476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f92933a780 con 0x55f9293e54d0 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f65d3e00700 1 -- 172.21.15.112:0/1005187476 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f65ec007bc0 con 0x55f9293e54d0 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f660324ab80 10 client.25520 fetch_fsmap learned FSMap version 30 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f660324ab80 1 -- 172.21.15.112:0/1005187476 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f9292c3bf0 con 0x55f9293e54d0 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f65d3e00700 1 -- 172.21.15.112:0/1005187476 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f92933a780 con 0x55f9293e54d0 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f65d3e00700 1 -- 172.21.15.112:0/1005187476 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f65ec01a470 con 0x55f9293e54d0 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f660324ab80 10 client.25520 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f660324ab80 10 client.25520 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-25T18:50:22.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f660324ab80 20 client.25520 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:22.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f660324ab80 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f9293f4cd0 0x55f9293f7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f660324ab80 4 client.25520 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:50:22.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.228+0000 7f660324ab80 1 -- 172.21.15.112:0/1005187476 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f929334820 con 0x55f9293f4cd0 2024-04-25T18:50:22.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.229+0000 7f65f5e00700 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f9293f4cd0 0x55f9293f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.229+0000 7f65f5e00700 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f9293f4cd0 0x55f9293f7100 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1212f68c6da57e4e server_cookie=3591676ae3f4602d in_seq=0 out_seq=0 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.229+0000 7f65d3e00700 10 client.25520 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f65d3e00700 1 -- 172.21.15.112:0/1005187476 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f929334820 con 0x55f9293f4cd0 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f65d3e00700 10 client.25520 handle_command_reply: tid=0 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f660324ab80 1 client.25520 shutdown 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f660324ab80 1 -- 172.21.15.112:0/1005187476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 msgr2=0x55f9293e2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f660324ab80 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293e2530 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65ec018ca0 tx=0x7f65ec004ad0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f660324ab80 1 -- 172.21.15.112:0/1005187476 shutdown_connections 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f660324ab80 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f9293f4cd0 0x55f9293f7100 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f660324ab80 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293e2530 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f660324ab80 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293e4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f65d3e00700 0 client.25520 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:22.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f660324ab80 1 --2- 172.21.15.112:0/1005187476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293e1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.231+0000 7f660324ab80 1 -- 172.21.15.112:0/1005187476 >> 172.21.15.112:0/1005187476 conn(0x55f929309050 msgr2=0x55f9293d5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:22.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.232+0000 7f660324ab80 1 -- 172.21.15.112:0/1005187476 shutdown_connections 2024-04-25T18:50:22.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.232+0000 7f660324ab80 1 -- 172.21.15.112:0/1005187476 wait complete. 2024-04-25T18:50:22.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.232+0000 7f660324ab80 20 client.25520 trim_cache size 0 max 16384 2024-04-25T18:50:22.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.254+0000 7f660324ab80 1 Processor -- start 2024-04-25T18:50:22.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.254+0000 7f660324ab80 1 -- start start 2024-04-25T18:50:22.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.254+0000 7f660324ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293d9d40 0x55f9293f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.254+0000 7f660324ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293da660 0x55f9293fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.254+0000 7f660324ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293e54d0 0x55f9293fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.254+0000 7f660324ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b1fb0 con 0x55f9293da660 2024-04-25T18:50:22.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.254+0000 7f660324ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01af7e0 con 0x55f9293d9d40 2024-04-25T18:50:22.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.254+0000 7f660324ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01ad630 con 0x55f9293e54d0 2024-04-25T18:50:22.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f5e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293d9d40 0x55f9293f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f5e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293d9d40 0x55f9293f0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48728/0 (socket says 172.21.15.112:48728) 2024-04-25T18:50:22.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f5e00700 1 -- 172.21.15.112:0/768227250 learned_addr learned my addr 172.21.15.112:0/768227250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:22.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f6800700 1 --2- 172.21.15.112:0/768227250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293e54d0 0x55f9293fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f5400700 1 --2- 172.21.15.112:0/768227250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293da660 0x55f9293fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f5e00700 1 -- 172.21.15.112:0/768227250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293e54d0 msgr2=0x55f9293fcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:22.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f5e00700 1 --2- 172.21.15.112:0/768227250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293e54d0 0x55f9293fcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f5e00700 1 -- 172.21.15.112:0/768227250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293da660 msgr2=0x55f9293fc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:22.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f5e00700 1 --2- 172.21.15.112:0/768227250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293da660 0x55f9293fc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f5e00700 1 -- 172.21.15.112:0/768227250 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9293dace0 con 0x55f9293d9d40 2024-04-25T18:50:22.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f6800700 1 --2- 172.21.15.112:0/768227250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293e54d0 0x55f9293fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:50:22.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.255+0000 7f65f5e00700 1 --2- 172.21.15.112:0/768227250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293d9d40 0x55f9293f0d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65e40098e0 tx=0x7f65e401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fe1a9de912a54f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:22.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.256+0000 7f65dea00700 1 -- 172.21.15.112:0/768227250 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e40056a0 con 0x55f9293d9d40 2024-04-25T18:50:22.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.256+0000 7f65dea00700 1 -- 172.21.15.112:0/768227250 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e402f070 con 0x55f9293d9d40 2024-04-25T18:50:22.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.256+0000 7f65dea00700 1 -- 172.21.15.112:0/768227250 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e40296a0 con 0x55f9293d9d40 2024-04-25T18:50:22.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.256+0000 7f660324ab80 1 -- 172.21.15.112:0/768227250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293d9d40 msgr2=0x55f9293f0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:22.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.256+0000 7f660324ab80 1 --2- 172.21.15.112:0/768227250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293d9d40 0x55f9293f0d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65e40098e0 tx=0x7f65e401a040 comp rx=0 tx=0).stop 2024-04-25T18:50:22.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.256+0000 7f660324ab80 1 -- 172.21.15.112:0/768227250 shutdown_connections 2024-04-25T18:50:22.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.256+0000 7f660324ab80 1 --2- 172.21.15.112:0/768227250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293e54d0 0x55f9293fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.256+0000 7f660324ab80 1 --2- 172.21.15.112:0/768227250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293d9d40 0x55f9293f0d60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.256+0000 7f660324ab80 1 --2- 172.21.15.112:0/768227250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293da660 0x55f9293fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.256+0000 7f660324ab80 1 -- 172.21.15.112:0/768227250 >> 172.21.15.112:0/768227250 conn(0x55f929309050 msgr2=0x55f9293e40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:22.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.256+0000 7f660324ab80 1 -- 172.21.15.112:0/768227250 shutdown_connections 2024-04-25T18:50:22.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.257+0000 7f660324ab80 1 -- 172.21.15.112:0/768227250 wait complete. 2024-04-25T18:50:22.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.257+0000 7f660324ab80 1 -- start start 2024-04-25T18:50:22.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.257+0000 7f660324ab80 1 Processor -- start 2024-04-25T18:50:22.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f660324ab80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293ea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f660324ab80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f660324ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f660324ab80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01adca0 con 0x55f9293d9d40 2024-04-25T18:50:22.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f660324ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01af110 con 0x55f9293e54d0 2024-04-25T18:50:22.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f660324ab80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01ae370 con 0x55f9293da660 2024-04-25T18:50:22.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f660324ab80 10 client.0 _ll_register_callbacks cb 0x55f9292c5820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:22.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f65f5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293ea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f65f6800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f65f6800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293eacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48740/0 (socket says 172.21.15.112:48740) 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f65f6800700 1 -- 172.21.15.112:0/3100098368 learned_addr learned my addr 172.21.15.112:0/3100098368 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f65f5e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293ea240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55758/0 (socket says 172.21.15.112:55758) 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.258+0000 7f65f5400700 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f65f6800700 1 -- 172.21.15.112:0/3100098368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 msgr2=0x55f9293ea780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f65f6800700 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293ea780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f65f6800700 1 -- 172.21.15.112:0/3100098368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 msgr2=0x55f9293ea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f65f6800700 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293ea240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f65f6800700 1 -- 172.21.15.112:0/3100098368 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92945b380 con 0x55f9293e54d0 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f65f6800700 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293eacc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65ec019010 tx=0x7f65ec007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a5e4d65e0dffb5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f65d3e00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f65d3e00700 1 -- 172.21.15.112:0/3100098368 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec032020 con 0x55f9293e54d0 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f65d3e00700 1 -- 172.21.15.112:0/3100098368 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f65ec01a680 con 0x55f9293e54d0 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f660324ab80 1 -- 172.21.15.112:0/3100098368 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e402f070 con 0x55f9293e54d0 2024-04-25T18:50:22.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f65d3e00700 1 -- 172.21.15.112:0/3100098368 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f65ec01e070 con 0x55f9293e54d0 2024-04-25T18:50:22.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f660324ab80 10 client.25526 fetch_fsmap learned FSMap version 30 2024-04-25T18:50:22.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.259+0000 7f660324ab80 1 -- 172.21.15.112:0/3100098368 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f92945c670 con 0x55f9293e54d0 2024-04-25T18:50:22.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.260+0000 7f65d3e00700 1 -- 172.21.15.112:0/3100098368 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e402f070 con 0x55f9293e54d0 2024-04-25T18:50:22.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.260+0000 7f65d3e00700 1 -- 172.21.15.112:0/3100098368 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f65ec026420 con 0x55f9293e54d0 2024-04-25T18:50:22.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.260+0000 7f660324ab80 10 client.25526 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:50:22.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.260+0000 7f660324ab80 10 client.25526 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-25T18:50:22.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.260+0000 7f660324ab80 20 client.25526 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:22.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.260+0000 7f660324ab80 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f92945ce60 0x55f92945f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:22.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.260+0000 7f660324ab80 4 client.25526 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5aed9937", "await_for": 42.0}] 2024-04-25T18:50:22.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.260+0000 7f660324ab80 1 -- 172.21.15.112:0/3100098368 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5aed9937", "await_for": 42.0}) -- 0x55f9291ad350 con 0x55f92945ce60 2024-04-25T18:50:22.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.260+0000 7f65f5e00700 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f92945ce60 0x55f92945f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:22.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f65d3e00700 10 client.25526 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:22.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f65f5e00700 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f92945ce60 0x55f92945f280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6cbcadd1fba8393 server_cookie=b06b264fd396c547 in_seq=0 out_seq=0 2024-04-25T18:50:22.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f65d3e00700 1 -- 172.21.15.112:0/3100098368 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: -110 ) ==== 8+0+590 (crc 0 0 0) 0x55f9291ad350 con 0x55f92945ce60 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f65d3e00700 10 client.25526 handle_command_reply: tid=0 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f660324ab80 1 client.25526 shutdown 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f660324ab80 1 -- 172.21.15.112:0/3100098368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 msgr2=0x55f9293eacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f660324ab80 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293eacc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f65ec019010 tx=0x7f65ec007f90 comp rx=0 tx=0).stop 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f660324ab80 1 -- 172.21.15.112:0/3100098368 shutdown_connections 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f660324ab80 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f92945ce60 0x55f92945f280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f660324ab80 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f9293e54d0 0x55f9293eacc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f65d3e00700 0 client.25526 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f660324ab80 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9293d9d40 0x55f9293ea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f660324ab80 1 --2- 172.21.15.112:0/3100098368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f9293da660 0x55f9293ea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.261+0000 7f660324ab80 1 -- 172.21.15.112:0/3100098368 >> 172.21.15.112:0/3100098368 conn(0x55f929309050 msgr2=0x55f9293dbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.262+0000 7f660324ab80 1 -- 172.21.15.112:0/3100098368 shutdown_connections 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.262+0000 7f660324ab80 1 -- 172.21.15.112:0/3100098368 wait complete. 2024-04-25T18:50:22.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:50:22.262+0000 7f660324ab80 20 client.25526 trim_cache size 0 max 16384 2024-04-25T18:50:22.266 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:50:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 30, 2024-04-25T18:50:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "leader": 24512, 2024-04-25T18:50:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "set_version": 6, 2024-04-25T18:50:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "sets": { 2024-04-25T18:50:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "5aed9937": { 2024-04-25T18:50:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "version": 6, 2024-04-25T18:50:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "age_ref": 9.1, 2024-04-25T18:50:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "state": { 2024-04-25T18:50:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "TIMEDOUT", 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 0.0 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "timeout": 60.0, 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "expiration": 80.8, 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "members": { 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "file:/": { 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "excluded": false, 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "state": { 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "QUIESCING", 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 60.0 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:50:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:50:22.268 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:50:22.268 INFO:teuthology.orchestra.run.smithi112.stderr:Error ETIMEDOUT: 2024-04-25T18:50:22.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:22 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:50:23.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:22 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:50:23.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:22 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:50:23.407 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-04-25T18:50:23.407 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't quiesce root with rc: 110 (ETIMEDOUT), stdout: { "epoch": 30, "leader": 24512, "set_version": 6, "sets": { "5aed9937": { "version": 6, "age_ref": 9.1, "state": { "name": "TIMEDOUT", "age": 0.0 }, "timeout": 60.0, "expiration": 80.8, "members": { "file:/": { "excluded": false, "state": { "name": "QUIESCING", "age": 60.0 } } } } } } 2024-04-25T18:50:23.407 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa/tasks/quiescer.py", line 346, in _run self.do_quiesce(d) File "/home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa/tasks/quiescer.py", line 301, in do_quiesce raise RuntimeError(f"Error quiescing set '{set_id}': {rcinfo}") RuntimeError: Error quiescing set '5aed9937': 110 (ETIMEDOUT) 2024-04-25T18:50:24.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:23 smithi171 ceph-mon[33266]: pgmap v93: 97 pgs: 97 active+clean; 27 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 671 KiB/s wr, 54 op/s 2024-04-25T18:50:24.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:23 smithi161 ceph-mon[32609]: pgmap v93: 97 pgs: 97 active+clean; 27 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 671 KiB/s wr, 54 op/s 2024-04-25T18:50:24.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:23 smithi112 ceph-mon[27864]: pgmap v93: 97 pgs: 97 active+clean; 27 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 671 KiB/s wr, 54 op/s 2024-04-25T18:50:25.491 INFO:tasks.workunit.client.0.smithi112.stderr:Updating files: 7% (321/4290) Updating files: 8% (344/4290) Updating files: 9% (387/4290) Updating files: 10% (429/4290) Updating files: 11% (472/4290) Updating files: 11% (490/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 17% (769/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 29% (1247/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 32% (1381/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 44% (1919/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 54% (2344/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 66% (2839/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 77% (3314/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 90% (3874/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 93% (3990/4290) Updating files: 93% (4025/4290) Updating files: 94% (4033/4290) Updating files: 95% (4076/4290) Updating files: 95% (4094/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-25T18:50:25.586 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd xfstests-dev 2024-04-25T18:50:25.586 INFO:tasks.workunit.client.0.smithi112.stderr:+ make -j4 2024-04-25T18:50:25.613 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize -cfi 2024-04-25T18:50:26.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:25 smithi171 ceph-mon[33266]: pgmap v94: 97 pgs: 97 active+clean; 27 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 671 KiB/s wr, 54 op/s 2024-04-25T18:50:26.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:25 smithi161 ceph-mon[32609]: pgmap v94: 97 pgs: 97 active+clean; 27 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 671 KiB/s wr, 54 op/s 2024-04-25T18:50:26.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:25 smithi112 ceph-mon[27864]: pgmap v94: 97 pgs: 97 active+clean; 27 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 671 KiB/s wr, 54 op/s 2024-04-25T18:50:26.516 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-25T18:50:26.516 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './config.guess' 2024-04-25T18:50:26.530 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './config.sub' 2024-04-25T18:50:26.546 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './install-sh' 2024-04-25T18:50:26.562 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file './ltmain.sh' 2024-04-25T18:50:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.608+0000 7fe041e00640 1 -- 172.21.15.112:0/1738251696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe03c1080e0 msgr2=0x7fe03c1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.608+0000 7fe041e00640 1 --2- 172.21.15.112:0/1738251696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe03c1080e0 0x7fe03c1084c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009900 tx=0x7fe02c02a480 comp rx=0 tx=0).stop 2024-04-25T18:50:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.608+0000 7fe041e00640 1 -- 172.21.15.112:0/1738251696 shutdown_connections 2024-04-25T18:50:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.608+0000 7fe041e00640 1 --2- 172.21.15.112:0/1738251696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe03c06b120 0x7fe03c10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.608+0000 7fe041e00640 1 --2- 172.21.15.112:0/1738251696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe03c06a780 0x7fe03c06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.608+0000 7fe041e00640 1 --2- 172.21.15.112:0/1738251696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe03c1080e0 0x7fe03c1084c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.608+0000 7fe041e00640 1 -- 172.21.15.112:0/1738251696 >> 172.21.15.112:0/1738251696 conn(0x7fe03c076c50 msgr2=0x7fe03c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.608+0000 7fe041e00640 1 -- 172.21.15.112:0/1738251696 shutdown_connections 2024-04-25T18:50:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.608+0000 7fe041e00640 1 -- 172.21.15.112:0/1738251696 wait complete. 2024-04-25T18:50:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.609+0000 7fe041e00640 1 Processor -- start 2024-04-25T18:50:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.609+0000 7fe041e00640 1 -- start start 2024-04-25T18:50:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.609+0000 7fe041e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe03c06a780 0x7fe03c1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.609+0000 7fe041e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe03c06b120 0x7fe03c1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.609+0000 7fe041e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe03c1080e0 0x7fe03c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.609+0000 7fe041e00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c112200 con 0x7fe03c06b120 2024-04-25T18:50:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.609+0000 7fe041e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c112080 con 0x7fe03c1080e0 2024-04-25T18:50:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.609+0000 7fe041e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c112380 con 0x7fe03c06a780 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.610+0000 7fe03a000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe03c06b120 0x7fe03c1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.610+0000 7fe03b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe03c1080e0 0x7fe03c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.610+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe03c06a780 0x7fe03c1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.610+0000 7fe03a000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe03c06b120 0x7fe03c1a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55774/0 (socket says 172.21.15.112:55774) 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.610+0000 7fe03a000640 1 -- 172.21.15.112:0/2649282815 learned_addr learned my addr 172.21.15.112:0/2649282815 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.610+0000 7fe03b400640 1 -- 172.21.15.112:0/2649282815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe03c06a780 msgr2=0x7fe03c1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.610+0000 7fe03b400640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe03c06a780 0x7fe03c1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.610+0000 7fe03b400640 1 -- 172.21.15.112:0/2649282815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe03c06b120 msgr2=0x7fe03c1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.610+0000 7fe03b400640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe03c06b120 0x7fe03c1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.610+0000 7fe03b400640 1 -- 172.21.15.112:0/2649282815 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03c19af50 con 0x7fe03c1080e0 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.610+0000 7fe03b400640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe03c1080e0 0x7fe03c19a690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe034009fd0 tx=0x7fe03400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.611+0000 7fe040a00640 1 -- 172.21.15.112:0/2649282815 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03400ce20 con 0x7fe03c1080e0 2024-04-25T18:50:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.611+0000 7fe040a00640 1 -- 172.21.15.112:0/2649282815 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe034005020 con 0x7fe03c1080e0 2024-04-25T18:50:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.611+0000 7fe040a00640 1 -- 172.21.15.112:0/2649282815 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034010690 con 0x7fe03c1080e0 2024-04-25T18:50:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.611+0000 7fe041e00640 1 -- 172.21.15.112:0/2649282815 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03c19b240 con 0x7fe03c1080e0 2024-04-25T18:50:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.611+0000 7fe041e00640 1 -- 172.21.15.112:0/2649282815 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03c1a7550 con 0x7fe03c1080e0 2024-04-25T18:50:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.614+0000 7fe044bfe740 1 Processor -- start 2024-04-25T18:50:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.614+0000 7fe040a00640 1 -- 172.21.15.112:0/2649282815 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fe0340054d0 con 0x7fe03c1080e0 2024-04-25T18:50:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.614+0000 7fe044bfe740 1 -- start start 2024-04-25T18:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.615+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c928c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.615+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c929140 0x55938c9334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.615+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c935df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.615+0000 7fe044bfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55938c88e900 con 0x55938c929140 2024-04-25T18:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.615+0000 7fe044bfe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55938c834ef0 con 0x55938c933a00 2024-04-25T18:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.615+0000 7fe044bfe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55938c88a8c0 con 0x55938c928820 2024-04-25T18:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.615+0000 7fe03a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c935df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.615+0000 7fe03a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c935df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48768/0 (socket says 172.21.15.112:48768) 2024-04-25T18:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.615+0000 7fe03a000640 1 -- 172.21.15.112:0/1156293636 learned_addr learned my addr 172.21.15.112:0/1156293636 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.615+0000 7fe03a000640 1 --2- 172.21.15.112:0/1156293636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c928c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.615+0000 7fe03aa00640 1 --2- 172.21.15.112:0/1156293636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c929140 0x55938c9334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe03a000640 1 -- 172.21.15.112:0/1156293636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 msgr2=0x55938c928c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe03a000640 1 --2- 172.21.15.112:0/1156293636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c928c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe03a000640 1 -- 172.21.15.112:0/1156293636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c929140 msgr2=0x55938c9334c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe03a000640 1 --2- 172.21.15.112:0/1156293636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c929140 0x55938c9334c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe03a000640 1 -- 172.21.15.112:0/1156293636 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55938c5a1210 con 0x55938c933a00 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe040a00640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fe0200a3e40 0x7fe0200a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe040a00640 1 -- 172.21.15.112:0/2649282815 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fe0340d0e10 con 0x7fe03c1080e0 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe03a000640 1 --2- 172.21.15.112:0/1156293636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c935df0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe0280118d0 tx=0x7fe0280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b8b4e09bf351164 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe03aa00640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fe0200a3e40 0x7fe0200a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe027e00640 1 -- 172.21.15.112:0/1156293636 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028005020 con 0x55938c933a00 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe027e00640 1 -- 172.21.15.112:0/1156293636 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0280047a0 con 0x55938c933a00 2024-04-25T18:50:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.616+0000 7fe027e00640 1 -- 172.21.15.112:0/1156293636 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02800f9c0 con 0x55938c933a00 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.617+0000 7fe03aa00640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fe0200a3e40 0x7fe0200a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe02c005ab0 tx=0x7fe02c005a20 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.617+0000 7fe044bfe740 1 -- 172.21.15.112:0/1156293636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 msgr2=0x55938c935df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.617+0000 7fe044bfe740 1 --2- 172.21.15.112:0/1156293636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c935df0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe0280118d0 tx=0x7fe0280046c0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.617+0000 7fe044bfe740 1 -- 172.21.15.112:0/1156293636 shutdown_connections 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.617+0000 7fe044bfe740 1 --2- 172.21.15.112:0/1156293636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c935df0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.617+0000 7fe044bfe740 1 --2- 172.21.15.112:0/1156293636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c929140 0x55938c9334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.617+0000 7fe044bfe740 1 --2- 172.21.15.112:0/1156293636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c928c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.617+0000 7fe044bfe740 1 -- 172.21.15.112:0/1156293636 >> 172.21.15.112:0/1156293636 conn(0x55938c924d30 msgr2=0x55938c925110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.617+0000 7fe044bfe740 1 -- 172.21.15.112:0/1156293636 shutdown_connections 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.617+0000 7fe044bfe740 1 -- 172.21.15.112:0/1156293636 wait complete. 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.618+0000 7fe044bfe740 1 -- start start 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.618+0000 7fe044bfe740 1 Processor -- start 2024-04-25T18:50:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.618+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c9309b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c929140 0x55938c930ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c931430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe044bfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55938c936630 con 0x55938c929140 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe044bfe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55938c848310 con 0x55938c933a00 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe044bfe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55938c9367b0 con 0x55938c928820 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe044bfe740 10 client.0 _ll_register_callbacks cb 0x55938c816cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe03b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c931430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe03a000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c929140 0x55938c930ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe03b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c931430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48774/0 (socket says 172.21.15.112:48774) 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe03b400640 1 -- 172.21.15.112:0/3060499624 learned_addr learned my addr 172.21.15.112:0/3060499624 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe03aa00640 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c9309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe03b400640 1 -- 172.21.15.112:0/3060499624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 msgr2=0x55938c9309b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe03b400640 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c9309b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe03b400640 1 -- 172.21.15.112:0/3060499624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c929140 msgr2=0x55938c930ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe03b400640 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c929140 0x55938c930ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe03b400640 1 -- 172.21.15.112:0/3060499624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55938c66d620 con 0x55938c933a00 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.619+0000 7fe03a000640 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c929140 0x55938c930ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.620+0000 7fe03b400640 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c931430 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe0340a5490 tx=0x7fe03409cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b8d8aa604debc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.620+0000 7fe024c00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.620+0000 7fe024c00640 1 -- 172.21.15.112:0/3060499624 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0340b6060 con 0x55938c933a00 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.620+0000 7fe024c00640 1 -- 172.21.15.112:0/3060499624 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fe0340993c0 con 0x55938c933a00 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.620+0000 7fe044bfe740 1 -- 172.21.15.112:0/3060499624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0280047a0 con 0x55938c933a00 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.620+0000 7fe024c00640 1 -- 172.21.15.112:0/3060499624 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0340a5530 con 0x55938c933a00 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.620+0000 7fe044bfe740 10 client.25538 fetch_fsmap learned FSMap version 30 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.620+0000 7fe044bfe740 1 -- 172.21.15.112:0/3060499624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55938c942950 con 0x55938c933a00 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.620+0000 7fe024c00640 1 -- 172.21.15.112:0/3060499624 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0280047a0 con 0x55938c933a00 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.621+0000 7fe024c00640 1 -- 172.21.15.112:0/3060499624 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe0340bd070 con 0x55938c933a00 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.621+0000 7fe044bfe740 10 client.25538 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.621+0000 7fe044bfe740 10 client.25538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:50:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.621+0000 7fe044bfe740 20 client.25538 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.621+0000 7fe044bfe740 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55938c943e10 0x55938c946200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.621+0000 7fe044bfe740 4 client.25538 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:50:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.621+0000 7fe044bfe740 1 -- 172.21.15.112:0/3060499624 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55938c8a28b0 con 0x55938c943e10 2024-04-25T18:50:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.621+0000 7fe03aa00640 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55938c943e10 0x55938c946200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.622+0000 7fe03aa00640 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55938c943e10 0x55938c946200 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47f2f4709f518a6b server_cookie=f2e0f4eed260e55b in_seq=0 out_seq=0 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.622+0000 7fe024c00640 10 client.25538 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe024c00640 1 -- 172.21.15.112:0/3060499624 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55938c8a28b0 con 0x55938c943e10 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe024c00640 10 client.25538 handle_command_reply: tid=0 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe044bfe740 1 client.25538 shutdown 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe044bfe740 1 -- 172.21.15.112:0/3060499624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 msgr2=0x55938c931430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe044bfe740 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c931430 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe0340a5490 tx=0x7fe03409cd50 comp rx=0 tx=0).stop 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe044bfe740 1 -- 172.21.15.112:0/3060499624 shutdown_connections 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe044bfe740 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c931430 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe044bfe740 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55938c943e10 0x55938c946200 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe044bfe740 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c929140 0x55938c930ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe044bfe740 1 --2- 172.21.15.112:0/3060499624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c9309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe044bfe740 1 -- 172.21.15.112:0/3060499624 >> 172.21.15.112:0/3060499624 conn(0x55938c924d30 msgr2=0x55938c92aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe024c00640 0 client.25538 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe044bfe740 1 -- 172.21.15.112:0/3060499624 shutdown_connections 2024-04-25T18:50:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.623+0000 7fe044bfe740 1 -- 172.21.15.112:0/3060499624 wait complete. 2024-04-25T18:50:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.624+0000 7fe044bfe740 20 client.25538 trim_cache size 0 max 16384 2024-04-25T18:50:26.640 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-25T18:50:26.640 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-25T18:50:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.643+0000 7fe044bfe740 1 Processor -- start 2024-04-25T18:50:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.643+0000 7fe044bfe740 1 -- start start 2024-04-25T18:50:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c928820 0x55938c923e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c929140 0x55938c92b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c931080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe044bfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1b2740 con 0x55938c928820 2024-04-25T18:50:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe044bfe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1affb0 con 0x55938c933a00 2024-04-25T18:50:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe044bfe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1ade00 con 0x55938c929140 2024-04-25T18:50:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c928820 0x55938c923e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c928820 0x55938c923e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55806/0 (socket says 172.21.15.112:55806) 2024-04-25T18:50:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe03aa00640 1 -- 172.21.15.112:0/913658668 learned_addr learned my addr 172.21.15.112:0/913658668 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe03a000640 1 --2- 172.21.15.112:0/913658668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c929140 0x55938c92b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe03b400640 1 --2- 172.21.15.112:0/913658668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c931080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe03aa00640 1 -- 172.21.15.112:0/913658668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c929140 msgr2=0x55938c92b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe03aa00640 1 --2- 172.21.15.112:0/913658668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c929140 0x55938c92b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe03aa00640 1 -- 172.21.15.112:0/913658668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 msgr2=0x55938c931080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe03aa00640 1 --2- 172.21.15.112:0/913658668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c931080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.644+0000 7fe03aa00640 1 -- 172.21.15.112:0/913658668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55938c920690 con 0x55938c928820 2024-04-25T18:50:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.645+0000 7fe03aa00640 1 --2- 172.21.15.112:0/913658668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c928820 0x55938c923e90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009900 tx=0x7fe02c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a687a47b8c7a48f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.645+0000 7fe027400640 1 -- 172.21.15.112:0/913658668 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c035040 con 0x55938c928820 2024-04-25T18:50:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.645+0000 7fe027400640 1 -- 172.21.15.112:0/913658668 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe02c05a070 con 0x55938c928820 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.645+0000 7fe027400640 1 -- 172.21.15.112:0/913658668 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c004e90 con 0x55938c928820 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.646+0000 7fe044bfe740 1 -- 172.21.15.112:0/913658668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c928820 msgr2=0x55938c923e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.646+0000 7fe044bfe740 1 --2- 172.21.15.112:0/913658668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c928820 0x55938c923e90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009900 tx=0x7fe02c03f000 comp rx=0 tx=0).stop 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.646+0000 7fe044bfe740 1 -- 172.21.15.112:0/913658668 shutdown_connections 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.646+0000 7fe044bfe740 1 --2- 172.21.15.112:0/913658668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c933a00 0x55938c931080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.646+0000 7fe044bfe740 1 --2- 172.21.15.112:0/913658668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c929140 0x55938c92b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.646+0000 7fe044bfe740 1 --2- 172.21.15.112:0/913658668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c928820 0x55938c923e90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.646+0000 7fe044bfe740 1 -- 172.21.15.112:0/913658668 >> 172.21.15.112:0/913658668 conn(0x55938c924d30 msgr2=0x55938c930970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.646+0000 7fe044bfe740 1 -- 172.21.15.112:0/913658668 shutdown_connections 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.646+0000 7fe044bfe740 1 -- 172.21.15.112:0/913658668 wait complete. 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.647+0000 7fe044bfe740 1 -- start start 2024-04-25T18:50:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.647+0000 7fe044bfe740 1 Processor -- start 2024-04-25T18:50:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c939750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c929140 0x55938c939c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe044bfe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c933a00 0x55938c93a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe044bfe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1ae470 con 0x55938c933a00 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe044bfe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1af8e0 con 0x55938c929140 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe044bfe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c1aeb40 con 0x55938c928820 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe044bfe740 10 client.0 _ll_register_callbacks cb 0x7fe03409c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe03b400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c933a00 0x55938c93a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe03b400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c933a00 0x55938c93a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55816/0 (socket says 172.21.15.112:55816) 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe03b400640 1 -- 172.21.15.112:0/638641872 learned_addr learned my addr 172.21.15.112:0/638641872 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe03a000640 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c929140 0x55938c939c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe03aa00640 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c939750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe03b400640 1 -- 172.21.15.112:0/638641872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 msgr2=0x55938c939750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe03b400640 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c939750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe03b400640 1 -- 172.21.15.112:0/638641872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c929140 msgr2=0x55938c939c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe03b400640 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c929140 0x55938c939c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe03b400640 1 -- 172.21.15.112:0/638641872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55938c9a95e0 con 0x55938c933a00 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.648+0000 7fe03a000640 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c929140 0x55938c939c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.649+0000 7fe03b400640 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c933a00 0x55938c93a1d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe03409d600 tx=0x7fe034005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4812929a568d86d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.649+0000 7fe024c00640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.649+0000 7fe024c00640 1 -- 172.21.15.112:0/638641872 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03409cb20 con 0x55938c933a00 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.649+0000 7fe024c00640 1 -- 172.21.15.112:0/638641872 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fe03407bb70 con 0x55938c933a00 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.649+0000 7fe044bfe740 1 -- 172.21.15.112:0/638641872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe02c05a070 con 0x55938c933a00 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.649+0000 7fe024c00640 1 -- 172.21.15.112:0/638641872 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe03407bd70 con 0x55938c933a00 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.649+0000 7fe044bfe740 10 client.15399 fetch_fsmap learned FSMap version 30 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.649+0000 7fe044bfe740 1 -- 172.21.15.112:0/638641872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55938c9aa8d0 con 0x55938c933a00 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.649+0000 7fe024c00640 1 -- 172.21.15.112:0/638641872 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe02c05a070 con 0x55938c933a00 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.650+0000 7fe024c00640 1 -- 172.21.15.112:0/638641872 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe03409bc10 con 0x55938c933a00 2024-04-25T18:50:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.650+0000 7fe044bfe740 10 client.15399 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.650+0000 7fe044bfe740 10 client.15399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.650+0000 7fe044bfe740 20 client.15399 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.650+0000 7fe044bfe740 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55938c9ab080 0x55938c9ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.650+0000 7fe044bfe740 4 client.15399 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.650+0000 7fe044bfe740 1 -- 172.21.15.112:0/638641872 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55938c89c320 con 0x55938c9ab080 2024-04-25T18:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.650+0000 7fe03aa00640 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55938c9ab080 0x55938c9ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.651+0000 7fe03aa00640 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55938c9ab080 0x55938c9ad4a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=886488cadf11c321 server_cookie=790a691350ca253 in_seq=0 out_seq=0 2024-04-25T18:50:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:26.651+0000 7fe024c00640 10 client.15399 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:26.659 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-25T18:50:26.673 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-25T18:50:26.689 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-25T18:50:26.705 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-25T18:50:26.723 INFO:tasks.workunit.client.0.smithi112.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-25T18:50:26.727 INFO:tasks.workunit.client.0.smithi112.stdout:cp include/install-sh . 2024-04-25T18:50:26.987 INFO:tasks.workunit.client.0.smithi112.stdout:aclocal -I m4 2024-04-25T18:50:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe024c00640 1 -- 172.21.15.112:0/638641872 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55938c89c320 con 0x55938c9ab080 2024-04-25T18:50:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe024c00640 10 client.15399 handle_command_reply: tid=0 2024-04-25T18:50:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 1 client.15399 shutdown 2024-04-25T18:50:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 1 -- 172.21.15.112:0/638641872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c933a00 msgr2=0x55938c93a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c933a00 0x55938c93a1d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe03409d600 tx=0x7fe034005870 comp rx=0 tx=0).stop 2024-04-25T18:50:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 1 -- 172.21.15.112:0/638641872 shutdown_connections 2024-04-25T18:50:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55938c9ab080 0x55938c9ad4a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55938c933a00 0x55938c93a1d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55938c929140 0x55938c939c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe024c00640 0 client.15399 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 1 --2- 172.21.15.112:0/638641872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55938c928820 0x55938c939750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 1 -- 172.21.15.112:0/638641872 >> 172.21.15.112:0/638641872 conn(0x55938c924d30 msgr2=0x55938c923d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 1 -- 172.21.15.112:0/638641872 shutdown_connections 2024-04-25T18:50:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 1 -- 172.21.15.112:0/638641872 wait complete. 2024-04-25T18:50:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.332+0000 7fe044bfe740 20 client.15399 trim_cache size 0 max 16384 2024-04-25T18:50:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:50:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:50:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:50:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:50:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 -- 172.21.15.112:0/2649282815 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fe0200a3e40 msgr2=0x7fe0200a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fe0200a3e40 0x7fe0200a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe02c005ab0 tx=0x7fe02c005a20 comp rx=0 tx=0).stop 2024-04-25T18:50:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 -- 172.21.15.112:0/2649282815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe03c1080e0 msgr2=0x7fe03c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe03c1080e0 0x7fe03c19a690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe034009fd0 tx=0x7fe03400ef10 comp rx=0 tx=0).stop 2024-04-25T18:50:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 -- 172.21.15.112:0/2649282815 shutdown_connections 2024-04-25T18:50:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fe0200a3e40 0x7fe0200a6300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe03c1080e0 0x7fe03c19a690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe03c06b120 0x7fe03c1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 --2- 172.21.15.112:0/2649282815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe03c06a780 0x7fe03c1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 -- 172.21.15.112:0/2649282815 >> 172.21.15.112:0/2649282815 conn(0x7fe03c076c50 msgr2=0x7fe03c10d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 -- 172.21.15.112:0/2649282815 shutdown_connections 2024-04-25T18:50:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:27.333+0000 7fe041e00640 1 -- 172.21.15.112:0/2649282815 wait complete. 2024-04-25T18:50:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:27 smithi171 ceph-mon[33266]: pgmap v95: 97 pgs: 97 active+clean; 37 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 254 op/s 2024-04-25T18:50:28.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:27 smithi161 ceph-mon[32609]: pgmap v95: 97 pgs: 97 active+clean; 37 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 254 op/s 2024-04-25T18:50:28.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:27 smithi112 ceph-mon[27864]: pgmap v95: 97 pgs: 97 active+clean; 37 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 254 op/s 2024-04-25T18:50:28.649 INFO:tasks.workunit.client.0.smithi112.stdout:autoheader 2024-04-25T18:50:29.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:29 smithi112 ceph-mon[27864]: pgmap v96: 97 pgs: 97 active+clean; 37 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-04-25T18:50:30.081 INFO:tasks.workunit.client.0.smithi112.stdout:autoconf 2024-04-25T18:50:30.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:29 smithi171 ceph-mon[33266]: pgmap v96: 97 pgs: 97 active+clean; 37 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-04-25T18:50:30.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:29 smithi161 ceph-mon[32609]: pgmap v96: 97 pgs: 97 active+clean; 37 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-04-25T18:50:30.326 INFO:tasks.workunit.client.0.smithi112.stdout:./configure \ 2024-04-25T18:50:30.327 INFO:tasks.workunit.client.0.smithi112.stdout: --libexecdir=/usr/lib \ 2024-04-25T18:50:30.327 INFO:tasks.workunit.client.0.smithi112.stdout: --exec_prefix=/var/lib 2024-04-25T18:50:30.643 INFO:tasks.workunit.client.0.smithi112.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-25T18:50:30.643 INFO:tasks.workunit.client.0.smithi112.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-25T18:50:30.646 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to print strings... printf 2024-04-25T18:50:30.647 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc... gcc 2024-04-25T18:50:30.818 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether the C compiler works... yes 2024-04-25T18:50:30.818 INFO:tasks.workunit.client.0.smithi112.stdout:checking for C compiler default output file name... a.out 2024-04-25T18:50:30.901 INFO:tasks.workunit.client.0.smithi112.stdout:checking for suffix of executables... 2024-04-25T18:50:30.991 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are cross compiling... no 2024-04-25T18:50:31.066 INFO:tasks.workunit.client.0.smithi112.stdout:checking for suffix of object files... o 2024-04-25T18:50:31.122 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T18:50:31.180 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc accepts -g... yes 2024-04-25T18:50:31.328 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T18:50:31.338 INFO:tasks.workunit.client.0.smithi112.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-25T18:50:31.347 INFO:tasks.workunit.client.0.smithi112.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T18:50:31.351 INFO:tasks.workunit.client.0.smithi112.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T18:50:31.355 INFO:tasks.workunit.client.0.smithi112.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-25T18:50:31.368 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-25T18:50:31.374 INFO:tasks.workunit.client.0.smithi112.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-25T18:50:31.476 INFO:tasks.workunit.client.0.smithi112.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-25T18:50:32.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:31 smithi171 ceph-mon[33266]: pgmap v97: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 949 op/s 2024-04-25T18:50:32.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:31 smithi161 ceph-mon[32609]: pgmap v97: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 949 op/s 2024-04-25T18:50:32.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:31 smithi112 ceph-mon[27864]: pgmap v97: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 949 op/s 2024-04-25T18:50:32.564 INFO:tasks.workunit.client.0.smithi112.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-25T18:50:32.564 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether ln -s works... yes 2024-04-25T18:50:32.595 INFO:tasks.workunit.client.0.smithi112.stdout:checking the maximum length of command line arguments... 1572864 2024-04-25T18:50:32.606 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-25T18:50:32.606 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-25T18:50:32.606 INFO:tasks.workunit.client.0.smithi112.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-25T18:50:32.607 INFO:tasks.workunit.client.0.smithi112.stdout:checking for objdump... objdump 2024-04-25T18:50:32.608 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to recognize dependent libraries... pass_all 2024-04-25T18:50:32.610 INFO:tasks.workunit.client.0.smithi112.stdout:checking for dlltool... no 2024-04-25T18:50:32.610 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-25T18:50:32.611 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ar... ar 2024-04-25T18:50:32.782 INFO:tasks.workunit.client.0.smithi112.stdout:checking for archiver @FILE support... @ 2024-04-25T18:50:32.783 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strip... strip 2024-04-25T18:50:32.784 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ranlib... ranlib 2024-04-25T18:50:32.785 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gawk... gawk 2024-04-25T18:50:33.013 INFO:tasks.workunit.client.0.smithi112.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-25T18:50:33.019 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sysroot... no 2024-04-25T18:50:33.043 INFO:tasks.workunit.client.0.smithi112.stdout:checking for a working dd... /usr/bin/dd 2024-04-25T18:50:33.066 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-25T18:50:33.144 INFO:tasks.workunit.client.0.smithi112.stdout:checking for mt... no 2024-04-25T18:50:33.172 INFO:tasks.workunit.client.0.smithi112.stdout:checking if : is a manifest tool... no 2024-04-25T18:50:33.400 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T18:50:33.786 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ANSI C header files... yes 2024-04-25T18:50:33.866 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/types.h... yes 2024-04-25T18:50:33.958 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/stat.h... yes 2024-04-25T18:50:34.050 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdlib.h... yes 2024-04-25T18:50:34.143 INFO:tasks.workunit.client.0.smithi112.stdout:checking for string.h... yes 2024-04-25T18:50:34.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:33 smithi171 ceph-mon[33266]: pgmap v98: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.7 MiB/s wr, 925 op/s 2024-04-25T18:50:34.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:33 smithi161 ceph-mon[32609]: pgmap v98: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.7 MiB/s wr, 925 op/s 2024-04-25T18:50:34.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:33 smithi112 ceph-mon[27864]: pgmap v98: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.7 MiB/s wr, 925 op/s 2024-04-25T18:50:34.240 INFO:tasks.workunit.client.0.smithi112.stdout:checking for memory.h... yes 2024-04-25T18:50:34.332 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strings.h... yes 2024-04-25T18:50:34.423 INFO:tasks.workunit.client.0.smithi112.stdout:checking for inttypes.h... yes 2024-04-25T18:50:34.516 INFO:tasks.workunit.client.0.smithi112.stdout:checking for stdint.h... yes 2024-04-25T18:50:34.614 INFO:tasks.workunit.client.0.smithi112.stdout:checking for unistd.h... yes 2024-04-25T18:50:34.706 INFO:tasks.workunit.client.0.smithi112.stdout:checking for dlfcn.h... yes 2024-04-25T18:50:34.724 INFO:tasks.workunit.client.0.smithi112.stdout:checking for objdir... .libs 2024-04-25T18:50:34.987 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-25T18:50:34.988 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-25T18:50:35.072 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-25T18:50:35.173 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc static flag -static works... no 2024-04-25T18:50:35.318 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc supports -c -o file.o... yes 2024-04-25T18:50:35.318 INFO:tasks.workunit.client.0.smithi112.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-25T18:50:35.342 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-25T18:50:35.449 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether -lc should be explicitly linked in... no 2024-04-25T18:50:35.605 INFO:tasks.workunit.client.0.smithi112.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-25T18:50:35.605 INFO:tasks.workunit.client.0.smithi112.stdout:checking how to hardcode library paths into programs... immediate 2024-04-25T18:50:35.609 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether stripping libraries is possible... yes 2024-04-25T18:50:35.609 INFO:tasks.workunit.client.0.smithi112.stdout:checking if libtool supports shared libraries... yes 2024-04-25T18:50:35.609 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether to build shared libraries... yes 2024-04-25T18:50:35.609 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether to build static libraries... yes 2024-04-25T18:50:35.628 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc... (cached) gcc 2024-04-25T18:50:35.701 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T18:50:35.702 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T18:50:35.702 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T18:50:35.703 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gmake... /usr/bin/gmake 2024-04-25T18:50:35.703 INFO:tasks.workunit.client.0.smithi112.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-25T18:50:35.704 INFO:tasks.workunit.client.0.smithi112.stdout:checking for tar... /bin/tar 2024-04-25T18:50:35.704 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gzip... /bin/gzip 2024-04-25T18:50:35.754 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether gcc -MM is supported... yes 2024-04-25T18:50:35.755 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sort... /bin/sort 2024-04-25T18:50:35.755 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether ln -s works... yes 2024-04-25T18:50:35.756 INFO:tasks.workunit.client.0.smithi112.stdout:checking for rpm... /bin/rpm 2024-04-25T18:50:35.769 INFO:tasks.workunit.client.0.smithi112.stdout:checking for rpmbuild... no 2024-04-25T18:50:35.858 INFO:tasks.workunit.client.0.smithi112.stdout:checking assert.h usability... yes 2024-04-25T18:50:35.895 INFO:tasks.workunit.client.0.smithi112.stdout:checking assert.h presence... yes 2024-04-25T18:50:35.895 INFO:tasks.workunit.client.0.smithi112.stdout:checking for assert.h... yes 2024-04-25T18:50:36.006 INFO:tasks.workunit.client.0.smithi112.stdout:checking bstring.h usability... no 2024-04-25T18:50:36.073 INFO:tasks.workunit.client.0.smithi112.stdout:checking bstring.h presence... no 2024-04-25T18:50:36.073 INFO:tasks.workunit.client.0.smithi112.stdout:checking for bstring.h... no 2024-04-25T18:50:36.161 INFO:tasks.workunit.client.0.smithi112.stdout:checking libgen.h usability... yes 2024-04-25T18:50:36.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:35 smithi171 ceph-mon[33266]: pgmap v99: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.7 MiB/s wr, 925 op/s 2024-04-25T18:50:36.200 INFO:tasks.workunit.client.0.smithi112.stdout:checking libgen.h presence... yes 2024-04-25T18:50:36.200 INFO:tasks.workunit.client.0.smithi112.stdout:checking for libgen.h... yes 2024-04-25T18:50:36.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:35 smithi161 ceph-mon[32609]: pgmap v99: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.7 MiB/s wr, 925 op/s 2024-04-25T18:50:36.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:35 smithi112 ceph-mon[27864]: pgmap v99: 97 pgs: 97 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1.7 MiB/s wr, 925 op/s 2024-04-25T18:50:36.333 INFO:tasks.workunit.client.0.smithi112.stdout:checking dirent.h usability... yes 2024-04-25T18:50:36.369 INFO:tasks.workunit.client.0.smithi112.stdout:checking dirent.h presence... yes 2024-04-25T18:50:36.369 INFO:tasks.workunit.client.0.smithi112.stdout:checking for dirent.h... yes 2024-04-25T18:50:36.454 INFO:tasks.workunit.client.0.smithi112.stdout:checking errno.h usability... yes 2024-04-25T18:50:36.482 INFO:tasks.workunit.client.0.smithi112.stdout:checking errno.h presence... yes 2024-04-25T18:50:36.483 INFO:tasks.workunit.client.0.smithi112.stdout:checking for errno.h... yes 2024-04-25T18:50:36.568 INFO:tasks.workunit.client.0.smithi112.stdout:checking malloc.h usability... yes 2024-04-25T18:50:36.608 INFO:tasks.workunit.client.0.smithi112.stdout:checking malloc.h presence... yes 2024-04-25T18:50:36.608 INFO:tasks.workunit.client.0.smithi112.stdout:checking for malloc.h... yes 2024-04-25T18:50:36.710 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid.h usability... no 2024-04-25T18:50:36.768 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid.h presence... no 2024-04-25T18:50:36.768 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid.h... no 2024-04-25T18:50:36.860 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid/uuid.h usability... yes 2024-04-25T18:50:36.897 INFO:tasks.workunit.client.0.smithi112.stdout:checking uuid/uuid.h presence... yes 2024-04-25T18:50:36.897 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid/uuid.h... yes 2024-04-25T18:50:36.995 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/uuid.h usability... no 2024-04-25T18:50:37.052 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/uuid.h presence... no 2024-04-25T18:50:37.052 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/uuid.h... no 2024-04-25T18:50:37.146 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/file.h usability... yes 2024-04-25T18:50:37.179 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/file.h presence... yes 2024-04-25T18:50:37.179 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/file.h... yes 2024-04-25T18:50:37.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:36 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:37.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:36 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:37.292 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fcntl.h usability... yes 2024-04-25T18:50:37.330 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fcntl.h presence... yes 2024-04-25T18:50:37.330 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/fcntl.h... yes 2024-04-25T18:50:37.428 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/syssgi.h usability... no 2024-04-25T18:50:37.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:36 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:37.487 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/syssgi.h presence... no 2024-04-25T18:50:37.487 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/syssgi.h... no 2024-04-25T18:50:37.605 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/param.h usability... yes 2024-04-25T18:50:37.642 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/param.h presence... yes 2024-04-25T18:50:37.642 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/param.h... yes 2024-04-25T18:50:37.678 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/stat.h... (cached) yes 2024-04-25T18:50:37.762 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/statvfs.h usability... yes 2024-04-25T18:50:37.797 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/statvfs.h presence... yes 2024-04-25T18:50:37.797 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/statvfs.h... yes 2024-04-25T18:50:37.897 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/time.h usability... yes 2024-04-25T18:50:37.936 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/time.h presence... yes 2024-04-25T18:50:37.936 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/time.h... yes 2024-04-25T18:50:38.039 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/ioctl.h usability... yes 2024-04-25T18:50:38.078 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/ioctl.h presence... yes 2024-04-25T18:50:38.078 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/ioctl.h... yes 2024-04-25T18:50:38.196 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/wait.h usability... yes 2024-04-25T18:50:38.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:37 smithi161 ceph-mon[32609]: pgmap v100: 97 pgs: 97 active+clean; 32 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.8 MiB/s wr, 934 op/s 2024-04-25T18:50:38.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:37 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:38.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:37 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:38.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:37 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:50:38.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:37 smithi112 ceph-mon[27864]: pgmap v100: 97 pgs: 97 active+clean; 32 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.8 MiB/s wr, 934 op/s 2024-04-25T18:50:38.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:37 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:38.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:37 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:38.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:37 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:50:38.240 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/wait.h presence... yes 2024-04-25T18:50:38.240 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/wait.h... yes 2024-04-25T18:50:38.274 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/types.h... (cached) yes 2024-04-25T18:50:38.291 INFO:tasks.workunit.client.0.smithi112.stdout:checking for strings.h... (cached) yes 2024-04-25T18:50:38.380 INFO:tasks.workunit.client.0.smithi112.stdout:checking err.h usability... yes 2024-04-25T18:50:38.418 INFO:tasks.workunit.client.0.smithi112.stdout:checking err.h presence... yes 2024-04-25T18:50:38.418 INFO:tasks.workunit.client.0.smithi112.stdout:checking for err.h... yes 2024-04-25T18:50:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:37 smithi171 ceph-mon[33266]: pgmap v100: 97 pgs: 97 active+clean; 32 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.8 MiB/s wr, 934 op/s 2024-04-25T18:50:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:37 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:37 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:37 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:50:38.517 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/falloc.h usability... yes 2024-04-25T18:50:38.554 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/falloc.h presence... yes 2024-04-25T18:50:38.554 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/falloc.h... yes 2024-04-25T18:50:38.666 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-25T18:50:38.729 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-25T18:50:38.729 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-25T18:50:38.830 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-25T18:50:38.889 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-25T18:50:38.889 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-25T18:50:38.987 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/platform_defs.h usability... no 2024-04-25T18:50:39.053 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/platform_defs.h presence... no 2024-04-25T18:50:39.053 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/platform_defs.h... no 2024-04-25T18:50:39.149 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfs/ioctl.h usability... no 2024-04-25T18:50:39.211 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfs/ioctl.h presence... no 2024-04-25T18:50:39.211 INFO:tasks.workunit.client.0.smithi112.stdout:checking for btrfs/ioctl.h... no 2024-04-25T18:50:39.311 INFO:tasks.workunit.client.0.smithi112.stdout:checking cifs/ioctl.h usability... no 2024-04-25T18:50:39.376 INFO:tasks.workunit.client.0.smithi112.stdout:checking cifs/ioctl.h presence... no 2024-04-25T18:50:39.377 INFO:tasks.workunit.client.0.smithi112.stdout:checking for cifs/ioctl.h... no 2024-04-25T18:50:39.477 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/mman.h usability... yes 2024-04-25T18:50:39.516 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/mman.h presence... yes 2024-04-25T18:50:39.516 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/mman.h... yes 2024-04-25T18:50:39.627 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/ext4.h usability... no 2024-04-25T18:50:39.694 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/ext4.h presence... no 2024-04-25T18:50:39.694 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/ext4.h... no 2024-04-25T18:50:39.762 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/xfs_log_format.h... no 2024-04-25T18:50:39.879 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid_compare... no 2024-04-25T18:50:39.970 INFO:tasks.workunit.client.0.smithi112.stdout:checking for uuid_compare in -luuid... yes 2024-04-25T18:50:40.089 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/xfs.h... yes 2024-04-25T18:50:40.158 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/libxfs.h... no 2024-04-25T18:50:40.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:39 smithi161 ceph-mon[32609]: pgmap v101: 97 pgs: 97 active+clean; 32 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 750 KiB/s wr, 734 op/s 2024-04-25T18:50:40.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:39 smithi112 ceph-mon[27864]: pgmap v101: 97 pgs: 97 active+clean; 32 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 750 KiB/s wr, 734 op/s 2024-04-25T18:50:40.239 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-25T18:50:40.325 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/xqm.h... yes 2024-04-25T18:50:40.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:39 smithi171 ceph-mon[33266]: pgmap v101: 97 pgs: 97 active+clean; 32 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 750 KiB/s wr, 734 op/s 2024-04-25T18:50:40.439 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-25T18:50:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.467+0000 7f42b2800640 1 -- 172.21.15.112:0/1047444063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac104d50 msgr2=0x7f42ac108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.467+0000 7f42b2800640 1 --2- 172.21.15.112:0/1047444063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac104d50 0x7f42ac108d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4298009900 tx=0x7f429802a410 comp rx=0 tx=0).stop 2024-04-25T18:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.467+0000 7f42b2800640 1 -- 172.21.15.112:0/1047444063 shutdown_connections 2024-04-25T18:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.467+0000 7f42b2800640 1 --2- 172.21.15.112:0/1047444063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ac109470 0x7f42ac111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.467+0000 7f42b2800640 1 --2- 172.21.15.112:0/1047444063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac104d50 0x7f42ac108d90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.467+0000 7f42b2800640 1 --2- 172.21.15.112:0/1047444063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ac1043a0 0x7f42ac104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.467+0000 7f42b2800640 1 -- 172.21.15.112:0/1047444063 >> 172.21.15.112:0/1047444063 conn(0x7f42ac0ffc40 msgr2=0x7f42ac102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.467+0000 7f42b2800640 1 -- 172.21.15.112:0/1047444063 shutdown_connections 2024-04-25T18:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.468+0000 7f42b2800640 1 -- 172.21.15.112:0/1047444063 wait complete. 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42b2800640 1 Processor -- start 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42b2800640 1 -- start start 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42b2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ac1043a0 0x7f42ac1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42b2800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ac109470 0x7f42ac1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42b2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac19a740 0x7f42ac19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42b2800640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac114400 con 0x7f42ac109470 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42b2800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac114280 con 0x7f42ac19a740 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42b2800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac114580 con 0x7f42ac1043a0 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42ab400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ac1043a0 0x7f42ac1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42aaa00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ac109470 0x7f42ac1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42ab400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ac1043a0 0x7f42ac1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:32862/0 (socket says 172.21.15.112:32862) 2024-04-25T18:50:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.469+0000 7f42ab400640 1 -- 172.21.15.112:0/2417884433 learned_addr learned my addr 172.21.15.112:0/2417884433 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.470+0000 7f42abe00640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac19a740 0x7f42ac19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.470+0000 7f42abe00640 1 -- 172.21.15.112:0/2417884433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ac1043a0 msgr2=0x7f42ac1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.470+0000 7f42abe00640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ac1043a0 0x7f42ac1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.470+0000 7f42abe00640 1 -- 172.21.15.112:0/2417884433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ac109470 msgr2=0x7f42ac1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.470+0000 7f42abe00640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ac109470 0x7f42ac1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.470+0000 7f42abe00640 1 -- 172.21.15.112:0/2417884433 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac19b340 con 0x7f42ac19a740 2024-04-25T18:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.470+0000 7f42ab400640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ac1043a0 0x7f42ac1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.470+0000 7f42abe00640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac19a740 0x7f42ac19abc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f42a4009fd0 tx=0x7f42a400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.471+0000 7f42a3e00640 1 -- 172.21.15.112:0/2417884433 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a400ce20 con 0x7f42ac19a740 2024-04-25T18:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.471+0000 7f42a3e00640 1 -- 172.21.15.112:0/2417884433 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42a4005020 con 0x7f42ac19a740 2024-04-25T18:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.471+0000 7f42b2800640 1 -- 172.21.15.112:0/2417884433 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ac1a7420 con 0x7f42ac19a740 2024-04-25T18:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.471+0000 7f42a3e00640 1 -- 172.21.15.112:0/2417884433 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4010690 con 0x7f42ac19a740 2024-04-25T18:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.471+0000 7f42b2800640 1 -- 172.21.15.112:0/2417884433 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ac1a77d0 con 0x7f42ac19a740 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.474+0000 7f42b5623740 1 Processor -- start 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.474+0000 7f42a3e00640 1 -- 172.21.15.112:0/2417884433 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f42a40054d0 con 0x7f42ac19a740 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.474+0000 7f42b5623740 1 -- start start 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.474+0000 7f42b5623740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fee820 0x55e0d9feec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.474+0000 7f42b5623740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 0x55e0d9ff94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42b5623740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ffbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42b5623740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e0d9f54900 con 0x55e0d9fee820 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42b5623740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e0d9efaef0 con 0x55e0d9ff9a00 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42b5623740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e0d9f508c0 con 0x55e0d9fef140 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42aaa00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fee820 0x55e0d9feec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42ab400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 0x55e0d9ff94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42aaa00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fee820 0x55e0d9feec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55988/0 (socket says 172.21.15.112:55988) 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42aaa00640 1 -- 172.21.15.112:0/858428150 learned_addr learned my addr 172.21.15.112:0/858428150 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42aaa00640 1 --2- 172.21.15.112:0/858428150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ffbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42ab400640 1 -- 172.21.15.112:0/858428150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 msgr2=0x55e0d9ffbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42ab400640 1 --2- 172.21.15.112:0/858428150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ffbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42ab400640 1 -- 172.21.15.112:0/858428150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fee820 msgr2=0x55e0d9feec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42ab400640 1 --2- 172.21.15.112:0/858428150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fee820 0x55e0d9feec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42ab400640 1 -- 172.21.15.112:0/858428150 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0d9c67210 con 0x55e0d9fef140 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.475+0000 7f42aaa00640 1 --2- 172.21.15.112:0/858428150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fee820 0x55e0d9feec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.476+0000 7f42a3e00640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f42900a3dc0 0x7f42900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.476+0000 7f42ab400640 1 --2- 172.21.15.112:0/858428150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 0x55e0d9ff94c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f429c0114f0 tx=0x7f429c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14452f95c359d9ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.476+0000 7f42a3e00640 1 -- 172.21.15.112:0/2417884433 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f42a40d0e90 con 0x7f42ac19a740 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.476+0000 7f42aaa00640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f42900a3dc0 0x7f42900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.476+0000 7f42a0c00640 1 -- 172.21.15.112:0/858428150 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c01e070 con 0x55e0d9fef140 2024-04-25T18:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.476+0000 7f42a0c00640 1 -- 172.21.15.112:0/858428150 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f429c0035f0 con 0x55e0d9fef140 2024-04-25T18:50:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.477+0000 7f42b5623740 1 -- 172.21.15.112:0/858428150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 msgr2=0x55e0d9ff94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.477+0000 7f42aaa00640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f42900a3dc0 0x7f42900a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4298000c00 tx=0x7f4298005e50 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.477+0000 7f42b5623740 1 --2- 172.21.15.112:0/858428150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 0x55e0d9ff94c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f429c0114f0 tx=0x7f429c0119b0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.477+0000 7f42b5623740 1 -- 172.21.15.112:0/858428150 shutdown_connections 2024-04-25T18:50:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.477+0000 7f42b5623740 1 --2- 172.21.15.112:0/858428150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ffbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.477+0000 7f42b5623740 1 --2- 172.21.15.112:0/858428150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 0x55e0d9ff94c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.477+0000 7f42b5623740 1 --2- 172.21.15.112:0/858428150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fee820 0x55e0d9feec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.477+0000 7f42b5623740 1 -- 172.21.15.112:0/858428150 >> 172.21.15.112:0/858428150 conn(0x55e0d9fead30 msgr2=0x55e0d9feb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.477+0000 7f42b5623740 1 -- 172.21.15.112:0/858428150 shutdown_connections 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.477+0000 7f42b5623740 1 -- 172.21.15.112:0/858428150 wait complete. 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.478+0000 7f42b5623740 1 -- start start 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.478+0000 7f42b5623740 1 Processor -- start 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.478+0000 7f42b5623740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9fee820 0x55e0d9ff67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.478+0000 7f42b5623740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 0x55e0d9ff6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.478+0000 7f42b5623740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ff7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.478+0000 7f42b5623740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e0d9ffc630 con 0x55e0d9ff9a00 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.478+0000 7f42b5623740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e0d9f0e310 con 0x55e0d9fee820 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.478+0000 7f42b5623740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e0d9ffc7b0 con 0x55e0d9fef140 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.478+0000 7f42b5623740 10 client.0 _ll_register_callbacks cb 0x55e0d9edccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f42aaa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 0x55e0d9ff6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f42ab400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9fee820 0x55e0d9ff67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f42aaa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 0x55e0d9ff6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:32890/0 (socket says 172.21.15.112:32890) 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f42aaa00640 1 -- 172.21.15.112:0/228558825 learned_addr learned my addr 172.21.15.112:0/228558825 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f42abe00640 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ff7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f42ab400640 1 -- 172.21.15.112:0/228558825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 msgr2=0x55e0d9ff6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f42ab400640 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 0x55e0d9ff6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f42ab400640 1 -- 172.21.15.112:0/228558825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9ff9a00 msgr2=0x55e0d9ff7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f42ab400640 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ff7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f42ab400640 1 -- 172.21.15.112:0/228558825 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0d9d33620 con 0x55e0d9fee820 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f42ab400640 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9fee820 0x55e0d9ff67e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f429c000c00 tx=0x7f429c00fcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b4c320ae1e1cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.479+0000 7f4295600640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.480+0000 7f4295600640 1 -- 172.21.15.112:0/228558825 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c005c50 con 0x55e0d9fee820 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.480+0000 7f42b5623740 1 -- 172.21.15.112:0/228558825 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f429c0035f0 con 0x55e0d9fee820 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.480+0000 7f4295600640 1 -- 172.21.15.112:0/228558825 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f429c025b90 con 0x55e0d9fee820 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.480+0000 7f4295600640 1 -- 172.21.15.112:0/228558825 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f429c025db0 con 0x55e0d9fee820 2024-04-25T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.480+0000 7f42b5623740 10 client.25556 fetch_fsmap learned FSMap version 30 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.480+0000 7f42b5623740 1 -- 172.21.15.112:0/228558825 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e0da008770 con 0x55e0d9fee820 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.480+0000 7f4295600640 1 -- 172.21.15.112:0/228558825 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f429c0035f0 con 0x55e0d9fee820 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.480+0000 7f4295600640 1 -- 172.21.15.112:0/228558825 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f429c02d020 con 0x55e0d9fee820 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.481+0000 7f42b5623740 10 client.25556 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.481+0000 7f42b5623740 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.481+0000 7f42b5623740 20 client.25556 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.481+0000 7f42b5623740 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e0da009ac0 0x55e0da00beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.481+0000 7f42b5623740 4 client.25556 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.481+0000 7f42b5623740 1 -- 172.21.15.112:0/228558825 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0d9f688b0 con 0x55e0da009ac0 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.481+0000 7f42aaa00640 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e0da009ac0 0x55e0da00beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.481+0000 7f42aaa00640 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e0da009ac0 0x55e0da00beb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989aee0be49b5b8c server_cookie=a67e1182a36a543 in_seq=0 out_seq=0 2024-04-25T18:50:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.481+0000 7f4295600640 10 client.25556 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f4295600640 1 -- 172.21.15.112:0/228558825 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0d9f688b0 con 0x55e0da009ac0 2024-04-25T18:50:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f4295600640 10 client.25556 handle_command_reply: tid=0 2024-04-25T18:50:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f42b5623740 1 client.25556 shutdown 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f42b5623740 1 -- 172.21.15.112:0/228558825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9fee820 msgr2=0x55e0d9ff67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f42b5623740 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9fee820 0x55e0d9ff67e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f429c000c00 tx=0x7f429c00fcd0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f42b5623740 1 -- 172.21.15.112:0/228558825 shutdown_connections 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f42b5623740 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e0da009ac0 0x55e0da00beb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f42b5623740 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ff7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f42b5623740 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fef140 0x55e0d9ff6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f42b5623740 1 --2- 172.21.15.112:0/228558825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9fee820 0x55e0d9ff67e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f4295600640 0 client.25556 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.482+0000 7f42b5623740 1 -- 172.21.15.112:0/228558825 >> 172.21.15.112:0/228558825 conn(0x55e0d9fead30 msgr2=0x55e0d9ff0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.483+0000 7f42b5623740 1 -- 172.21.15.112:0/228558825 shutdown_connections 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.483+0000 7f42b5623740 1 -- 172.21.15.112:0/228558825 wait complete. 2024-04-25T18:50:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.483+0000 7f42b5623740 20 client.25556 trim_cache size 0 max 16384 2024-04-25T18:50:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.499+0000 7f42b5623740 1 Processor -- start 2024-04-25T18:50:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.499+0000 7f42b5623740 1 -- start start 2024-04-25T18:50:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.499+0000 7f42b5623740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fee820 0x55e0da005540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.499+0000 7f42b5623740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 0x55e0d9ff60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.499+0000 7f42b5623740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 0x55e0da011720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.499+0000 7f42b5623740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac1ae290 con 0x55e0d9fef140 2024-04-25T18:50:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.499+0000 7f42b5623740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac1abb00 con 0x55e0d9ff9a00 2024-04-25T18:50:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.499+0000 7f42b5623740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac1a9950 con 0x55e0d9fee820 2024-04-25T18:50:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42abe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 0x55e0da011720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42abe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 0x55e0da011720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:54042/0 (socket says 172.21.15.112:54042) 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42abe00640 1 -- 172.21.15.112:0/2203680518 learned_addr learned my addr 172.21.15.112:0/2203680518 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42aaa00640 1 --2- 172.21.15.112:0/2203680518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 0x55e0d9ff60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42ab400640 1 --2- 172.21.15.112:0/2203680518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fee820 0x55e0da005540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42abe00640 1 -- 172.21.15.112:0/2203680518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fee820 msgr2=0x55e0da005540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42abe00640 1 --2- 172.21.15.112:0/2203680518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fee820 0x55e0da005540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42abe00640 1 -- 172.21.15.112:0/2203680518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 msgr2=0x55e0d9ff60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42abe00640 1 --2- 172.21.15.112:0/2203680518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 0x55e0d9ff60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42abe00640 1 -- 172.21.15.112:0/2203680518 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0d9ff4d30 con 0x55e0d9ff9a00 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42ab400640 1 --2- 172.21.15.112:0/2203680518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fee820 0x55e0da005540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.500+0000 7f42abe00640 1 --2- 172.21.15.112:0/2203680518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 0x55e0da011720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f42a409cad0 tx=0x7f42a40bbf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1284f3d5639ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.501+0000 7f4297e00640 1 -- 172.21.15.112:0/2203680518 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a40b5070 con 0x55e0d9ff9a00 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.501+0000 7f4297e00640 1 -- 172.21.15.112:0/2203680518 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42a40b6260 con 0x55e0d9ff9a00 2024-04-25T18:50:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.501+0000 7f4297e00640 1 -- 172.21.15.112:0/2203680518 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a40a55b0 con 0x55e0d9ff9a00 2024-04-25T18:50:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.501+0000 7f42b5623740 1 -- 172.21.15.112:0/2203680518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 msgr2=0x55e0da011720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.501+0000 7f42b5623740 1 --2- 172.21.15.112:0/2203680518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 0x55e0da011720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f42a409cad0 tx=0x7f42a40bbf90 comp rx=0 tx=0).stop 2024-04-25T18:50:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.502+0000 7f42b5623740 1 -- 172.21.15.112:0/2203680518 shutdown_connections 2024-04-25T18:50:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.502+0000 7f42b5623740 1 --2- 172.21.15.112:0/2203680518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9ff9a00 0x55e0da011720 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.502+0000 7f42b5623740 1 --2- 172.21.15.112:0/2203680518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 0x55e0d9ff60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.502+0000 7f42b5623740 1 --2- 172.21.15.112:0/2203680518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9fee820 0x55e0da005540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.502+0000 7f42b5623740 1 -- 172.21.15.112:0/2203680518 >> 172.21.15.112:0/2203680518 conn(0x55e0d9fead30 msgr2=0x55e0d9ff8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.502+0000 7f42b5623740 1 -- 172.21.15.112:0/2203680518 shutdown_connections 2024-04-25T18:50:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.502+0000 7f42b5623740 1 -- 172.21.15.112:0/2203680518 wait complete. 2024-04-25T18:50:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.502+0000 7f42b5623740 1 -- start start 2024-04-25T18:50:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.502+0000 7f42b5623740 1 Processor -- start 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42b5623740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9fee820 0x55e0da001e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42b5623740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 0x55e0d9ffa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42b5623740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ffaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42b5623740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac1a9fc0 con 0x55e0d9fef140 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42b5623740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac1ab430 con 0x55e0d9fee820 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42b5623740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac1aa690 con 0x55e0d9ff9a00 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42b5623740 10 client.0 _ll_register_callbacks cb 0x7f429c009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42aaa00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 0x55e0d9ffa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42aaa00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 0x55e0d9ffa560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56016/0 (socket says 172.21.15.112:56016) 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42aaa00640 1 -- 172.21.15.112:0/2758820250 learned_addr learned my addr 172.21.15.112:0/2758820250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42ab400640 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9fee820 0x55e0da001e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42abe00640 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ffaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42aaa00640 1 -- 172.21.15.112:0/2758820250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9ff9a00 msgr2=0x55e0d9ffaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42aaa00640 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ffaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42aaa00640 1 -- 172.21.15.112:0/2758820250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9fee820 msgr2=0x55e0da001e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42aaa00640 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9fee820 0x55e0da001e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.503+0000 7f42aaa00640 1 -- 172.21.15.112:0/2758820250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0d9ffb2d0 con 0x55e0d9fef140 2024-04-25T18:50:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.504+0000 7f42aaa00640 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 0x55e0d9ffa560 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f429802aa00 tx=0x7f429802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee495d5a6c2a2dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.504+0000 7f4295600640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:50:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.504+0000 7f4295600640 1 -- 172.21.15.112:0/2758820250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298035040 con 0x55e0d9fef140 2024-04-25T18:50:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.504+0000 7f4295600640 1 -- 172.21.15.112:0/2758820250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f4298057750 con 0x55e0d9fef140 2024-04-25T18:50:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.504+0000 7f42b5623740 1 -- 172.21.15.112:0/2758820250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42a40b6260 con 0x55e0d9fef140 2024-04-25T18:50:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.504+0000 7f4295600640 1 -- 172.21.15.112:0/2758820250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4298067050 con 0x55e0d9fef140 2024-04-25T18:50:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.504+0000 7f42b5623740 10 client.15417 fetch_fsmap learned FSMap version 30 2024-04-25T18:50:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.505+0000 7f42b5623740 1 -- 172.21.15.112:0/2758820250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e0da070030 con 0x55e0d9fef140 2024-04-25T18:50:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.505+0000 7f4295600640 1 -- 172.21.15.112:0/2758820250 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42a40b6260 con 0x55e0d9fef140 2024-04-25T18:50:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.505+0000 7f4295600640 1 -- 172.21.15.112:0/2758820250 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4298054050 con 0x55e0d9fef140 2024-04-25T18:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.505+0000 7f42b5623740 10 client.15417 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.505+0000 7f42b5623740 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.505+0000 7f42b5623740 20 client.15417 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.505+0000 7f42b5623740 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e0da0707f0 0x55e0da072c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.505+0000 7f42b5623740 4 client.15417 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.505+0000 7f42b5623740 1 -- 172.21.15.112:0/2758820250 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0d9f62320 con 0x55e0da0707f0 2024-04-25T18:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.505+0000 7f42ab400640 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e0da0707f0 0x55e0da072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.506+0000 7f42ab400640 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e0da0707f0 0x55e0da072c10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82fb2907e29b0490 server_cookie=3daa9b2620b64f2f in_seq=0 out_seq=0 2024-04-25T18:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:40.506+0000 7f4295600640 10 client.15417 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:40.513 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/handle.h usability... yes 2024-04-25T18:50:40.542 INFO:tasks.workunit.client.0.smithi112.stdout:checking xfs/handle.h presence... yes 2024-04-25T18:50:40.542 INFO:tasks.workunit.client.0.smithi112.stdout:checking for xfs/handle.h... yes 2024-04-25T18:50:40.639 INFO:tasks.workunit.client.0.smithi112.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-25T18:50:40.712 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/acl.h usability... yes 2024-04-25T18:50:40.750 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/acl.h presence... yes 2024-04-25T18:50:40.750 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/acl.h... yes 2024-04-25T18:50:40.840 INFO:tasks.workunit.client.0.smithi112.stdout:checking acl/libacl.h usability... yes 2024-04-25T18:50:40.877 INFO:tasks.workunit.client.0.smithi112.stdout:checking acl/libacl.h presence... yes 2024-04-25T18:50:40.877 INFO:tasks.workunit.client.0.smithi112.stdout:checking for acl/libacl.h... yes 2024-04-25T18:50:40.974 INFO:tasks.workunit.client.0.smithi112.stdout:checking for acl_init in -lacl... yes 2024-04-25T18:50:41.089 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm-ndbm.h usability... no 2024-04-25T18:50:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.210+0000 7f4295600640 1 -- 172.21.15.112:0/2758820250 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0d9f62320 con 0x55e0da0707f0 2024-04-25T18:50:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.210+0000 7f4295600640 10 client.15417 handle_command_reply: tid=0 2024-04-25T18:50:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.210+0000 7f42b5623740 1 client.15417 shutdown 2024-04-25T18:50:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f42b5623740 1 -- 172.21.15.112:0/2758820250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 msgr2=0x55e0d9ffa560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f42b5623740 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 0x55e0d9ffa560 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f429802aa00 tx=0x7f429802a410 comp rx=0 tx=0).stop 2024-04-25T18:50:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f42b5623740 1 -- 172.21.15.112:0/2758820250 shutdown_connections 2024-04-25T18:50:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f42b5623740 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0d9ff9a00 0x55e0d9ffaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f42b5623740 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55e0da0707f0 0x55e0da072c10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f42b5623740 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d9fef140 0x55e0d9ffa560 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f42b5623740 1 --2- 172.21.15.112:0/2758820250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e0d9fee820 0x55e0da001e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f4295600640 0 client.15417 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f42b5623740 1 -- 172.21.15.112:0/2758820250 >> 172.21.15.112:0/2758820250 conn(0x55e0d9fead30 msgr2=0x55e0d9ff6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f42b5623740 1 -- 172.21.15.112:0/2758820250 shutdown_connections 2024-04-25T18:50:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f42b5623740 1 -- 172.21.15.112:0/2758820250 wait complete. 2024-04-25T18:50:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.211+0000 7f42b5623740 20 client.15417 trim_cache size 0 max 16384 2024-04-25T18:50:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:50:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:50:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:50:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 -- 172.21.15.112:0/2417884433 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f42900a3dc0 msgr2=0x7f42900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f42900a3dc0 0x7f42900a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4298000c00 tx=0x7f4298005e50 comp rx=0 tx=0).stop 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 -- 172.21.15.112:0/2417884433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac19a740 msgr2=0x7f42ac19abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac19a740 0x7f42ac19abc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f42a4009fd0 tx=0x7f42a400ef10 comp rx=0 tx=0).stop 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 -- 172.21.15.112:0/2417884433 shutdown_connections 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f42900a3dc0 0x7f42900a6280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac19a740 0x7f42ac19abc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42ac109470 0x7f42ac1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 --2- 172.21.15.112:0/2417884433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ac1043a0 0x7f42ac1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 -- 172.21.15.112:0/2417884433 >> 172.21.15.112:0/2417884433 conn(0x7f42ac0ffc40 msgr2=0x7f42ac100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 -- 172.21.15.112:0/2417884433 shutdown_connections 2024-04-25T18:50:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:41.213+0000 7f42b2800640 1 -- 172.21.15.112:0/2417884433 wait complete. 2024-04-25T18:50:41.255 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm-ndbm.h presence... no 2024-04-25T18:50:41.255 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gdbm-ndbm.h... no 2024-04-25T18:50:41.324 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm/ndbm.h usability... no 2024-04-25T18:50:41.379 INFO:tasks.workunit.client.0.smithi112.stdout:checking gdbm/ndbm.h presence... no 2024-04-25T18:50:41.380 INFO:tasks.workunit.client.0.smithi112.stdout:checking for gdbm/ndbm.h... no 2024-04-25T18:50:41.469 INFO:tasks.workunit.client.0.smithi112.stdout:checking ndbm.h usability... no 2024-04-25T18:50:41.531 INFO:tasks.workunit.client.0.smithi112.stdout:checking ndbm.h presence... no 2024-04-25T18:50:41.531 INFO:tasks.workunit.client.0.smithi112.stdout:checking for ndbm.h... no 2024-04-25T18:50:41.606 INFO:tasks.workunit.client.0.smithi112.stdout:checking libaio.h usability... yes 2024-04-25T18:50:41.648 INFO:tasks.workunit.client.0.smithi112.stdout:checking libaio.h presence... yes 2024-04-25T18:50:41.648 INFO:tasks.workunit.client.0.smithi112.stdout:checking for libaio.h... yes 2024-04-25T18:50:41.745 INFO:tasks.workunit.client.0.smithi112.stdout:checking liburing.h usability... no 2024-04-25T18:50:41.808 INFO:tasks.workunit.client.0.smithi112.stdout:checking liburing.h presence... no 2024-04-25T18:50:41.808 INFO:tasks.workunit.client.0.smithi112.stdout:checking for liburing.h... no 2024-04-25T18:50:41.896 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/capability.h usability... no 2024-04-25T18:50:41.962 INFO:tasks.workunit.client.0.smithi112.stdout:checking sys/capability.h presence... no 2024-04-25T18:50:41.962 INFO:tasks.workunit.client.0.smithi112.stdout:checking for sys/capability.h... no 2024-04-25T18:50:42.033 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fiemap.h usability... yes 2024-04-25T18:50:42.071 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fiemap.h presence... yes 2024-04-25T18:50:42.071 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/fiemap.h... yes 2024-04-25T18:50:42.163 INFO:tasks.workunit.client.0.smithi112.stdout:checking for fallocate... yes 2024-04-25T18:50:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:41 smithi161 ceph-mon[32609]: pgmap v102: 97 pgs: 97 active+clean; 34 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 948 KiB/s wr, 745 op/s 2024-04-25T18:50:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:41 smithi112 ceph-mon[27864]: pgmap v102: 97 pgs: 97 active+clean; 34 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 948 KiB/s wr, 745 op/s 2024-04-25T18:50:42.273 INFO:tasks.workunit.client.0.smithi112.stdout:checking for open_by_handle_at... yes 2024-04-25T18:50:42.353 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fs.h usability... yes 2024-04-25T18:50:42.391 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/fs.h presence... yes 2024-04-25T18:50:42.391 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/fs.h... yes 2024-04-25T18:50:42.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:41 smithi171 ceph-mon[33266]: pgmap v102: 97 pgs: 97 active+clean; 34 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 948 KiB/s wr, 745 op/s 2024-04-25T18:50:42.493 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfsutil.h usability... no 2024-04-25T18:50:42.555 INFO:tasks.workunit.client.0.smithi112.stdout:checking btrfsutil.h presence... no 2024-04-25T18:50:42.555 INFO:tasks.workunit.client.0.smithi112.stdout:checking for btrfsutil.h... no 2024-04-25T18:50:42.654 INFO:tasks.workunit.client.0.smithi112.stdout:checking for copy_file_range... yes 2024-04-25T18:50:42.751 INFO:tasks.workunit.client.0.smithi112.stdout:checking for SEEK_DATA... yes 2024-04-25T18:50:42.872 INFO:tasks.workunit.client.0.smithi112.stdout:checking for BMV_OF_SHARED... yes 2024-04-25T18:50:42.973 INFO:tasks.workunit.client.0.smithi112.stdout:checking for nftw... yes 2024-04-25T18:50:43.086 INFO:tasks.workunit.client.0.smithi112.stdout:checking for RLIMIT_NOFILE... yes 2024-04-25T18:50:43.186 INFO:tasks.workunit.client.0.smithi112.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-25T18:50:43.310 INFO:tasks.workunit.client.0.smithi112.stdout:checking for FICLONE... yes 2024-04-25T18:50:43.408 INFO:tasks.workunit.client.0.smithi112.stdout:checking for renameat2... yes 2024-04-25T18:50:43.507 INFO:tasks.workunit.client.0.smithi112.stdout:checking for reallocarray... yes 2024-04-25T18:50:43.646 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct mount_attr... yes 2024-04-25T18:50:43.790 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-25T18:50:43.930 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-25T18:50:44.069 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-25T18:50:44.208 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-25T18:50:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:43 smithi161 ceph-mon[32609]: pgmap v103: 97 pgs: 97 active+clean; 34 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 312 KiB/s wr, 19 op/s 2024-04-25T18:50:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:43 smithi112 ceph-mon[27864]: pgmap v103: 97 pgs: 97 active+clean; 34 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 312 KiB/s wr, 19 op/s 2024-04-25T18:50:44.348 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-25T18:50:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:43 smithi171 ceph-mon[33266]: pgmap v103: 97 pgs: 97 active+clean; 34 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 312 KiB/s wr, 19 op/s 2024-04-25T18:50:44.484 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-25T18:50:44.620 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-25T18:50:44.710 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs.h usability... yes 2024-04-25T18:50:44.748 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs.h presence... yes 2024-04-25T18:50:44.748 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/btrfs.h... yes 2024-04-25T18:50:44.849 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-25T18:50:44.889 INFO:tasks.workunit.client.0.smithi112.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-25T18:50:44.889 INFO:tasks.workunit.client.0.smithi112.stdout:checking for linux/btrfs_tree.h... yes 2024-04-25T18:50:44.978 INFO:tasks.workunit.client.0.smithi112.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-25T18:50:45.053 INFO:tasks.workunit.client.0.smithi112.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-25T18:50:45.094 INFO:tasks.workunit.client.0.smithi112.stdout:configure: creating ./config.status 2024-04-25T18:50:46.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:45 smithi161 ceph-mon[32609]: pgmap v104: 97 pgs: 97 active+clean; 34 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 312 KiB/s wr, 19 op/s 2024-04-25T18:50:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:45 smithi112 ceph-mon[27864]: pgmap v104: 97 pgs: 97 active+clean; 34 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 312 KiB/s wr, 19 op/s 2024-04-25T18:50:46.359 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: creating include/builddefs 2024-04-25T18:50:46.410 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: creating include/config.h 2024-04-25T18:50:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:45 smithi171 ceph-mon[33266]: pgmap v104: 97 pgs: 97 active+clean; 34 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 312 KiB/s wr, 19 op/s 2024-04-25T18:50:46.445 INFO:tasks.workunit.client.0.smithi112.stdout:config.status: executing libtool commands 2024-04-25T18:50:46.609 INFO:tasks.workunit.client.0.smithi112.stdout:Building include 2024-04-25T18:50:46.610 INFO:tasks.workunit.client.0.smithi112.stdout:Building lib 2024-04-25T18:50:46.610 INFO:tasks.workunit.client.0.smithi112.stdout:Building tests 2024-04-25T18:50:46.620 INFO:tasks.workunit.client.0.smithi112.stdout: [LTDEP] 2024-04-25T18:50:46.635 INFO:tasks.workunit.client.0.smithi112.stdout:Building btrfs 2024-04-25T18:50:46.635 INFO:tasks.workunit.client.0.smithi112.stdout:Building ceph 2024-04-25T18:50:46.635 INFO:tasks.workunit.client.0.smithi112.stdout:Building cifs 2024-04-25T18:50:46.653 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-04-25T18:50:46.654 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-04-25T18:50:46.688 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-04-25T18:50:46.706 INFO:tasks.workunit.client.0.smithi112.stdout:Building ext4 2024-04-25T18:50:46.709 INFO:tasks.workunit.client.0.smithi112.stdout:Building f2fs 2024-04-25T18:50:46.727 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-04-25T18:50:46.744 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-04-25T18:50:46.791 INFO:tasks.workunit.client.0.smithi112.stdout:Building generic 2024-04-25T18:50:46.887 INFO:tasks.workunit.client.0.smithi112.stdout:Building nfs 2024-04-25T18:50:46.903 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-04-25T18:50:46.968 INFO:tasks.workunit.client.0.smithi112.stdout:Building ocfs2 2024-04-25T18:50:46.982 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-04-25T18:50:47.003 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-04-25T18:50:47.028 INFO:tasks.workunit.client.0.smithi112.stdout:Building overlay 2024-04-25T18:50:47.057 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-04-25T18:50:47.061 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dataascii.lo 2024-04-25T18:50:47.066 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] databin.lo 2024-04-25T18:50:47.203 INFO:tasks.workunit.client.0.smithi112.stdout:Building perf 2024-04-25T18:50:47.219 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-04-25T18:50:47.257 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] datapid.lo 2024-04-25T18:50:47.308 INFO:tasks.workunit.client.0.smithi112.stdout:Building selftest 2024-04-25T18:50:47.322 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-04-25T18:50:47.361 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] file_lock.lo 2024-04-25T18:50:47.364 INFO:tasks.workunit.client.0.smithi112.stdout:Building shared 2024-04-25T18:50:47.380 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-04-25T18:50:47.395 INFO:tasks.workunit.client.0.smithi112.stdout:Building tmpfs 2024-04-25T18:50:47.410 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-04-25T18:50:47.442 INFO:tasks.workunit.client.0.smithi112.stdout:Building udf 2024-04-25T18:50:47.456 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-04-25T18:50:47.460 INFO:tasks.workunit.client.0.smithi112.stdout:Building xfs 2024-04-25T18:50:47.492 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] forker.lo 2024-04-25T18:50:47.495 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] pattern.lo 2024-04-25T18:50:47.649 INFO:tasks.workunit.client.0.smithi112.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-04-25T18:50:47.696 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] open_flags.lo 2024-04-25T18:50:47.750 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] random_range.lo 2024-04-25T18:50:47.801 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] string_to_tokens.lo 2024-04-25T18:50:47.966 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] str_to_bytes.lo 2024-04-25T18:50:48.007 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] tlibio.lo 2024-04-25T18:50:48.013 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] write_log.lo 2024-04-25T18:50:48.117 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] random.lo 2024-04-25T18:50:48.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:47 smithi161 ceph-mon[32609]: pgmap v105: 97 pgs: 97 active+clean; 27 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 534 KiB/s wr, 104 op/s 2024-04-25T18:50:48.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:47 smithi112 ceph-mon[27864]: pgmap v105: 97 pgs: 97 active+clean; 27 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 534 KiB/s wr, 104 op/s 2024-04-25T18:50:48.233 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:47 smithi171 ceph-mon[33266]: pgmap v105: 97 pgs: 97 active+clean; 27 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 534 KiB/s wr, 104 op/s 2024-04-25T18:50:48.391 INFO:tasks.workunit.client.0.smithi112.stdout: [LD] 2024-04-25T18:50:48.676 INFO:tasks.workunit.client.0.smithi112.stdout:Building ltp 2024-04-25T18:50:48.676 INFO:tasks.workunit.client.0.smithi112.stdout:Building src 2024-04-25T18:50:48.676 INFO:tasks.workunit.client.0.smithi112.stdout:Building m4 2024-04-25T18:50:48.677 INFO:tasks.workunit.client.0.smithi112.stdout:Building common 2024-04-25T18:50:48.690 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-04-25T18:50:48.711 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-04-25T18:50:49.133 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] doio 2024-04-25T18:50:49.133 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsstress 2024-04-25T18:50:49.135 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsx 2024-04-25T18:50:50.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:49 smithi161 ceph-mon[32609]: pgmap v106: 97 pgs: 97 active+clean; 27 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 420 KiB/s wr, 95 op/s 2024-04-25T18:50:50.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:49 smithi112 ceph-mon[27864]: pgmap v106: 97 pgs: 97 active+clean; 27 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 420 KiB/s wr, 95 op/s 2024-04-25T18:50:50.246 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] growfiles 2024-04-25T18:50:50.285 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] iogen 2024-04-25T18:50:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:49 smithi171 ceph-mon[33266]: pgmap v106: 97 pgs: 97 active+clean; 27 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 420 KiB/s wr, 95 op/s 2024-04-25T18:50:50.903 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-stress 2024-04-25T18:50:52.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:51 smithi161 ceph-mon[32609]: pgmap v107: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-04-25T18:50:52.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:51 smithi112 ceph-mon[27864]: pgmap v107: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-04-25T18:50:52.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:51 smithi171 ceph-mon[33266]: pgmap v107: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-04-25T18:50:53.307 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-04-25T18:50:53.307 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-04-25T18:50:53.309 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-04-25T18:50:53.311 INFO:tasks.workunit.client.0.smithi112.stdout: [DEP] 2024-04-25T18:50:53.873 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dirstress 2024-04-25T18:50:53.874 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fill 2024-04-25T18:50:53.876 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fill2 2024-04-25T18:50:53.877 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] getpagesize 2024-04-25T18:50:54.145 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] holes 2024-04-25T18:50:54.168 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] lstat64 2024-04-25T18:50:54.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:53 smithi112 ceph-mon[27864]: pgmap v108: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 900 KiB/s wr, 207 op/s 2024-04-25T18:50:54.275 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] nametest 2024-04-25T18:50:54.297 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] permname 2024-04-25T18:50:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.376+0000 7f1785600640 1 -- 172.21.15.112:0/348039639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17801031e0 msgr2=0x7f17801035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.376+0000 7f1785600640 1 --2- 172.21.15.112:0/348039639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17801031e0 0x7f17801035c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1770009900 tx=0x7f177002a480 comp rx=0 tx=0).stop 2024-04-25T18:50:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.376+0000 7f1785600640 1 -- 172.21.15.112:0/348039639 shutdown_connections 2024-04-25T18:50:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.376+0000 7f1785600640 1 --2- 172.21.15.112:0/348039639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f178010c5b0 0x7f178010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.376+0000 7f1785600640 1 --2- 172.21.15.112:0/348039639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1780103b00 0x7f178010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.376+0000 7f1785600640 1 --2- 172.21.15.112:0/348039639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17801031e0 0x7f17801035c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.376+0000 7f1785600640 1 -- 172.21.15.112:0/348039639 >> 172.21.15.112:0/348039639 conn(0x7f1780077f60 msgr2=0x7f1780078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.376+0000 7f1785600640 1 -- 172.21.15.112:0/348039639 shutdown_connections 2024-04-25T18:50:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.376+0000 7f1785600640 1 -- 172.21.15.112:0/348039639 wait complete. 2024-04-25T18:50:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.377+0000 7f1785600640 1 Processor -- start 2024-04-25T18:50:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.378+0000 7f1785600640 1 -- start start 2024-04-25T18:50:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.378+0000 7f1785600640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17801031e0 0x7f17801a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.378+0000 7f1785600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1780103b00 0x7f17801a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.378+0000 7f1785600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f178010c5b0 0x7f178019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.378+0000 7f1785600640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1780112230 con 0x7f17801031e0 2024-04-25T18:50:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.378+0000 7f1785600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f17801120b0 con 0x7f178010c5b0 2024-04-25T18:50:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.378+0000 7f1785600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f17801123b0 con 0x7f1780103b00 2024-04-25T18:50:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.378+0000 7f177e000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1780103b00 0x7f17801a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.379+0000 7f177ea00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17801031e0 0x7f17801a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.379+0000 7f177e000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1780103b00 0x7f17801a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:43988/0 (socket says 172.21.15.112:43988) 2024-04-25T18:50:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.379+0000 7f177e000640 1 -- 172.21.15.112:0/3881861738 learned_addr learned my addr 172.21.15.112:0/3881861738 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.379+0000 7f177f400640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f178010c5b0 0x7f178019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.379+0000 7f177e000640 1 -- 172.21.15.112:0/3881861738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f178010c5b0 msgr2=0x7f178019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.379+0000 7f177e000640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f178010c5b0 0x7f178019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.379+0000 7f177e000640 1 -- 172.21.15.112:0/3881861738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17801031e0 msgr2=0x7f17801a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.379+0000 7f177e000640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17801031e0 0x7f17801a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.379+0000 7f177e000640 1 -- 172.21.15.112:0/3881861738 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178019af60 con 0x7f1780103b00 2024-04-25T18:50:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.379+0000 7f177e000640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1780103b00 0x7f17801a0b60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f176c00d950 tx=0x7f176c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.380+0000 7f1777400640 1 -- 172.21.15.112:0/3881861738 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c017030 con 0x7f1780103b00 2024-04-25T18:50:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.380+0000 7f1777400640 1 -- 172.21.15.112:0/3881861738 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176c004020 con 0x7f1780103b00 2024-04-25T18:50:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.380+0000 7f1777400640 1 -- 172.21.15.112:0/3881861738 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c0047a0 con 0x7f1780103b00 2024-04-25T18:50:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.380+0000 7f1785600640 1 -- 172.21.15.112:0/3881861738 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178019b250 con 0x7f1780103b00 2024-04-25T18:50:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.381+0000 7f1785600640 1 -- 172.21.15.112:0/3881861738 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17801a7470 con 0x7f1780103b00 2024-04-25T18:50:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.385+0000 7f178841c740 1 Processor -- start 2024-04-25T18:50:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.385+0000 7f178841c740 1 -- start start 2024-04-25T18:50:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.385+0000 7f178841c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 0x561b95e39c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.385+0000 7f178841c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.385+0000 7f178841c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e44a00 0x561b95e46df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.385+0000 7f178841c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561b95d9f900 con 0x561b95e39820 2024-04-25T18:50:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.385+0000 7f178841c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561b95d45ef0 con 0x561b95e3a140 2024-04-25T18:50:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.385+0000 7f178841c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561b95d9b8c0 con 0x561b95e44a00 2024-04-25T18:50:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.386+0000 7f177f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 0x561b95e39c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.386+0000 7f177f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 0x561b95e39c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52886/0 (socket says 172.21.15.112:52886) 2024-04-25T18:50:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.386+0000 7f177f400640 1 -- 172.21.15.112:0/639192156 learned_addr learned my addr 172.21.15.112:0/639192156 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.386+0000 7f177ea00640 1 --2- 172.21.15.112:0/639192156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.387+0000 7f177f400640 1 -- 172.21.15.112:0/639192156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e44a00 msgr2=0x561b95e46df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T18:50:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.387+0000 7f177f400640 1 --2- 172.21.15.112:0/639192156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e44a00 0x561b95e46df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.387+0000 7f177f400640 1 -- 172.21.15.112:0/639192156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 msgr2=0x561b95e444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.387+0000 7f1777400640 1 -- 172.21.15.112:0/3881861738 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f176c009a30 con 0x7f1780103b00 2024-04-25T18:50:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.387+0000 7f177f400640 1 --2- 172.21.15.112:0/639192156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.387+0000 7f177f400640 1 -- 172.21.15.112:0/639192156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b95ab2210 con 0x561b95e39820 2024-04-25T18:50:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.387+0000 7f177ea00640 1 --2- 172.21.15.112:0/639192156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:50:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.388+0000 7f177f400640 1 --2- 172.21.15.112:0/639192156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 0x561b95e39c00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1778009f40 tx=0x7f1778010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cf624deb4101b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.388+0000 7f176b400640 1 -- 172.21.15.112:0/639192156 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778016070 con 0x561b95e39820 2024-04-25T18:50:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.388+0000 7f176b400640 1 -- 172.21.15.112:0/639192156 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1778004d20 con 0x561b95e39820 2024-04-25T18:50:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.388+0000 7f176b400640 1 -- 172.21.15.112:0/639192156 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778012850 con 0x561b95e39820 2024-04-25T18:50:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.389+0000 7f1777400640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17640a3e40 0x7f17640a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.390+0000 7f1777400640 1 -- 172.21.15.112:0/3881861738 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f176c0d0140 con 0x7f1780103b00 2024-04-25T18:50:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.390+0000 7f177ea00640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17640a3e40 0x7f17640a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.391+0000 7f178841c740 1 -- 172.21.15.112:0/639192156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 msgr2=0x561b95e39c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.391+0000 7f178841c740 1 --2- 172.21.15.112:0/639192156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 0x561b95e39c00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1778009f40 tx=0x7f1778010800 comp rx=0 tx=0).stop 2024-04-25T18:50:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.391+0000 7f178841c740 1 -- 172.21.15.112:0/639192156 shutdown_connections 2024-04-25T18:50:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.391+0000 7f178841c740 1 --2- 172.21.15.112:0/639192156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e44a00 0x561b95e46df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.391+0000 7f178841c740 1 --2- 172.21.15.112:0/639192156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.391+0000 7f178841c740 1 --2- 172.21.15.112:0/639192156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 0x561b95e39c00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.391+0000 7f178841c740 1 -- 172.21.15.112:0/639192156 >> 172.21.15.112:0/639192156 conn(0x561b95e35d30 msgr2=0x561b95e36110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.391+0000 7f177ea00640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17640a3e40 0x7f17640a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1770005ab0 tx=0x7f1770005a20 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.391+0000 7f178841c740 1 -- 172.21.15.112:0/639192156 shutdown_connections 2024-04-25T18:50:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.391+0000 7f178841c740 1 -- 172.21.15.112:0/639192156 wait complete. 2024-04-25T18:50:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.392+0000 7f178841c740 1 -- start start 2024-04-25T18:50:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.392+0000 7f178841c740 1 Processor -- start 2024-04-25T18:50:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f178841c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 0x561b95e41a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f178841c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e41f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f178841c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e44a00 0x561b95e424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f178841c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561b95e47630 con 0x561b95e39820 2024-04-25T18:50:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f178841c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561b95d59310 con 0x561b95e3a140 2024-04-25T18:50:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f178841c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561b95e477b0 con 0x561b95e44a00 2024-04-25T18:50:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f178841c740 10 client.0 _ll_register_callbacks cb 0x561b95d27cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f177ea00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 0x561b95e41a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f177ea00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 0x561b95e41a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52898/0 (socket says 172.21.15.112:52898) 2024-04-25T18:50:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f177ea00640 1 -- 172.21.15.112:0/2627043851 learned_addr learned my addr 172.21.15.112:0/2627043851 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f177e000640 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e41f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.393+0000 7f177f400640 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e44a00 0x561b95e424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.394+0000 7f177e000640 1 -- 172.21.15.112:0/2627043851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e44a00 msgr2=0x561b95e424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.394+0000 7f177e000640 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e44a00 0x561b95e424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.394+0000 7f177e000640 1 -- 172.21.15.112:0/2627043851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 msgr2=0x561b95e41a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.394+0000 7f177e000640 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 0x561b95e41a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.394+0000 7f177e000640 1 -- 172.21.15.112:0/2627043851 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b95b7e620 con 0x561b95e3a140 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.394+0000 7f177e000640 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e41f70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f176c0cd890 tx=0x7f176c09bb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf98e4d1df79af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.394+0000 7f175fe00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.394+0000 7f175fe00640 1 -- 172.21.15.112:0/2627043851 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c0a0490 con 0x561b95e3a140 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.394+0000 7f175fe00640 1 -- 172.21.15.112:0/2627043851 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f176c0a0120 con 0x561b95e3a140 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.394+0000 7f178841c740 1 -- 172.21.15.112:0/2627043851 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1778004d20 con 0x561b95e3a140 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.395+0000 7f175fe00640 1 -- 172.21.15.112:0/2627043851 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f176c0a4d50 con 0x561b95e3a140 2024-04-25T18:50:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.395+0000 7f178841c740 10 client.25571 fetch_fsmap learned FSMap version 30 2024-04-25T18:50:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.395+0000 7f178841c740 1 -- 172.21.15.112:0/2627043851 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561b95e538f0 con 0x561b95e3a140 2024-04-25T18:50:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.395+0000 7f175fe00640 1 -- 172.21.15.112:0/2627043851 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1778004d20 con 0x561b95e3a140 2024-04-25T18:50:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.396+0000 7f175fe00640 1 -- 172.21.15.112:0/2627043851 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f176c0b0b80 con 0x561b95e3a140 2024-04-25T18:50:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.396+0000 7f178841c740 10 client.25571 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:50:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.396+0000 7f178841c740 10 client.25571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:50:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.396+0000 7f178841c740 20 client.25571 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.396+0000 7f178841c740 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561b95e54db0 0x561b95e571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.396+0000 7f178841c740 4 client.25571 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:50:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.396+0000 7f178841c740 1 -- 172.21.15.112:0/2627043851 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b95db38b0 con 0x561b95e54db0 2024-04-25T18:50:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.396+0000 7f177ea00640 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561b95e54db0 0x561b95e571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.397+0000 7f175fe00640 10 client.25571 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.397+0000 7f177ea00640 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561b95e54db0 0x561b95e571d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1407e40bd3fa65ab server_cookie=cdb10002882eabb3 in_seq=0 out_seq=0 2024-04-25T18:50:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.399+0000 7f175fe00640 1 -- 172.21.15.112:0/2627043851 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b95db38b0 con 0x561b95e54db0 2024-04-25T18:50:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.399+0000 7f175fe00640 10 client.25571 handle_command_reply: tid=0 2024-04-25T18:50:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.399+0000 7f178841c740 1 client.25571 shutdown 2024-04-25T18:50:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f178841c740 1 -- 172.21.15.112:0/2627043851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 msgr2=0x561b95e41f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f178841c740 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e41f70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f176c0cd890 tx=0x7f176c09bb30 comp rx=0 tx=0).stop 2024-04-25T18:50:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f178841c740 1 -- 172.21.15.112:0/2627043851 shutdown_connections 2024-04-25T18:50:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f178841c740 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e44a00 0x561b95e424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f178841c740 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e41f70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f178841c740 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561b95e54db0 0x561b95e571d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f178841c740 1 --2- 172.21.15.112:0/2627043851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e39820 0x561b95e41a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f178841c740 1 -- 172.21.15.112:0/2627043851 >> 172.21.15.112:0/2627043851 conn(0x561b95e35d30 msgr2=0x561b95e3bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f175fe00640 0 client.25571 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f178841c740 1 -- 172.21.15.112:0/2627043851 shutdown_connections 2024-04-25T18:50:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f178841c740 1 -- 172.21.15.112:0/2627043851 wait complete. 2024-04-25T18:50:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.400+0000 7f178841c740 20 client.25571 trim_cache size 0 max 16384 2024-04-25T18:50:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.420+0000 7f178841c740 1 Processor -- start 2024-04-25T18:50:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.421+0000 7f178841c740 1 -- start start 2024-04-25T18:50:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.421+0000 7f178841c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 0x561b95e504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.421+0000 7f178841c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.421+0000 7f178841c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e44a00 0x561b95e5cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.421+0000 7f178841c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17801b27b0 con 0x561b95e44a00 2024-04-25T18:50:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.421+0000 7f178841c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f17801b0020 con 0x561b95e3a140 2024-04-25T18:50:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.421+0000 7f178841c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f17801ade70 con 0x561b95e39820 2024-04-25T18:50:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.421+0000 7f177f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e44a00 0x561b95e5cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.421+0000 7f177e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.422+0000 7f177e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:32872/0 (socket says 172.21.15.112:32872) 2024-04-25T18:50:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.422+0000 7f177e000640 1 -- 172.21.15.112:0/2504177703 learned_addr learned my addr 172.21.15.112:0/2504177703 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.422+0000 7f177ea00640 1 --2- 172.21.15.112:0/2504177703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 0x561b95e504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.422+0000 7f177e000640 1 -- 172.21.15.112:0/2504177703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 msgr2=0x561b95e504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.422+0000 7f177e000640 1 --2- 172.21.15.112:0/2504177703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 0x561b95e504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.422+0000 7f177e000640 1 -- 172.21.15.112:0/2504177703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e44a00 msgr2=0x561b95e5cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.422+0000 7f177e000640 1 --2- 172.21.15.112:0/2504177703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e44a00 0x561b95e5cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.422+0000 7f177e000640 1 -- 172.21.15.112:0/2504177703 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b95e36ac0 con 0x561b95e3a140 2024-04-25T18:50:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.422+0000 7f177f400640 1 --2- 172.21.15.112:0/2504177703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e44a00 0x561b95e5cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:50:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.423+0000 7f177e000640 1 --2- 172.21.15.112:0/2504177703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e422d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f176c0cde90 tx=0x7f176c09c260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97aa5a3203680f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.424+0000 7f176aa00640 1 -- 172.21.15.112:0/2504177703 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c0a0490 con 0x561b95e3a140 2024-04-25T18:50:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.424+0000 7f176aa00640 1 -- 172.21.15.112:0/2504177703 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176c09c7e0 con 0x561b95e3a140 2024-04-25T18:50:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.424+0000 7f176aa00640 1 -- 172.21.15.112:0/2504177703 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c097f90 con 0x561b95e3a140 2024-04-25T18:50:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.424+0000 7f178841c740 1 -- 172.21.15.112:0/2504177703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 msgr2=0x561b95e422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.424+0000 7f178841c740 1 --2- 172.21.15.112:0/2504177703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e422d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f176c0cde90 tx=0x7f176c09c260 comp rx=0 tx=0).stop 2024-04-25T18:50:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.425+0000 7f178841c740 1 -- 172.21.15.112:0/2504177703 shutdown_connections 2024-04-25T18:50:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.425+0000 7f178841c740 1 --2- 172.21.15.112:0/2504177703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e44a00 0x561b95e5cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.425+0000 7f178841c740 1 --2- 172.21.15.112:0/2504177703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e3a140 0x561b95e422d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.425+0000 7f178841c740 1 --2- 172.21.15.112:0/2504177703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 0x561b95e504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.425+0000 7f178841c740 1 -- 172.21.15.112:0/2504177703 >> 172.21.15.112:0/2504177703 conn(0x561b95e35d30 msgr2=0x561b95e34e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.425+0000 7f178841c740 1 -- 172.21.15.112:0/2504177703 shutdown_connections 2024-04-25T18:50:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.425+0000 7f178841c740 1 -- 172.21.15.112:0/2504177703 wait complete. 2024-04-25T18:50:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.426+0000 7f178841c740 1 -- start start 2024-04-25T18:50:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.426+0000 7f178841c740 1 Processor -- start 2024-04-25T18:50:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.426+0000 7f178841c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 0x561b95e4ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.426+0000 7f178841c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e3a140 0x561b95e4cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.426+0000 7f178841c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e44a00 0x561b95e49f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.426+0000 7f178841c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17801ae4e0 con 0x561b95e3a140 2024-04-25T18:50:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.426+0000 7f178841c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f17801af950 con 0x561b95e44a00 2024-04-25T18:50:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.426+0000 7f178841c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f17801aebb0 con 0x561b95e39820 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f178841c740 10 client.0 _ll_register_callbacks cb 0x7f176c09bc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f177ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 0x561b95e4ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f177ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 0x561b95e4ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:44016/0 (socket says 172.21.15.112:44016) 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f177ea00640 1 -- 172.21.15.112:0/1139151424 learned_addr learned my addr 172.21.15.112:0/1139151424 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f177f400640 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e44a00 0x561b95e49f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f177e000640 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e3a140 0x561b95e4cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f177ea00640 1 -- 172.21.15.112:0/1139151424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e44a00 msgr2=0x561b95e49f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f177ea00640 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e44a00 0x561b95e49f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f177ea00640 1 -- 172.21.15.112:0/1139151424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e3a140 msgr2=0x561b95e4cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f177ea00640 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e3a140 0x561b95e4cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f177ea00640 1 -- 172.21.15.112:0/1139151424 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b95e4a790 con 0x561b95e39820 2024-04-25T18:50:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.427+0000 7f177e000640 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e3a140 0x561b95e4cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:50:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.428+0000 7f177ea00640 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 0x561b95e4ca00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x561b95e57a60 tx=0x7f1770009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=590a9b3f2a83c0ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.428+0000 7f175fe00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:50:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.428+0000 7f175fe00640 1 -- 172.21.15.112:0/1139151424 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770035040 con 0x561b95e39820 2024-04-25T18:50:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.428+0000 7f178841c740 1 -- 172.21.15.112:0/1139151424 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f176c09c7e0 con 0x561b95e39820 2024-04-25T18:50:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.428+0000 7f175fe00640 1 -- 172.21.15.112:0/1139151424 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f1770005510 con 0x561b95e39820 2024-04-25T18:50:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.428+0000 7f175fe00640 1 -- 172.21.15.112:0/1139151424 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1770061050 con 0x561b95e39820 2024-04-25T18:50:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.428+0000 7f178841c740 10 client.25591 fetch_fsmap learned FSMap version 30 2024-04-25T18:50:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.428+0000 7f178841c740 1 -- 172.21.15.112:0/1139151424 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561b95ebaff0 con 0x561b95e39820 2024-04-25T18:50:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.429+0000 7f175fe00640 1 -- 172.21.15.112:0/1139151424 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176c09c7e0 con 0x561b95e39820 2024-04-25T18:50:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.429+0000 7f175fe00640 1 -- 172.21.15.112:0/1139151424 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1770004980 con 0x561b95e39820 2024-04-25T18:50:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.429+0000 7f178841c740 10 client.25591 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:50:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.429+0000 7f178841c740 10 client.25591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:50:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.429+0000 7f178841c740 20 client.25591 populate_metadata read hostname 'smithi112' 2024-04-25T18:50:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.429+0000 7f178841c740 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561b95ebb780 0x561b95ebdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.429+0000 7f178841c740 4 client.25591 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:50:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.429+0000 7f178841c740 1 -- 172.21.15.112:0/1139151424 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b95dad320 con 0x561b95ebb780 2024-04-25T18:50:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.430+0000 7f177e000640 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561b95ebb780 0x561b95ebdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:54.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:53 smithi171 ceph-mon[33266]: pgmap v108: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 900 KiB/s wr, 207 op/s 2024-04-25T18:50:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.430+0000 7f175fe00640 10 client.25591 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.430+0000 7f177e000640 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561b95ebb780 0x561b95ebdba0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1608f63d7498fe88 server_cookie=77d6792f64041966 in_seq=0 out_seq=0 2024-04-25T18:50:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:53 smithi161 ceph-mon[32609]: pgmap v108: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 900 KiB/s wr, 207 op/s 2024-04-25T18:50:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.499+0000 7f175fe00640 1 -- 172.21.15.112:0/1139151424 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b95dad320 con 0x561b95ebb780 2024-04-25T18:50:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.499+0000 7f175fe00640 10 client.25591 handle_command_reply: tid=0 2024-04-25T18:50:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.499+0000 7f178841c740 1 client.25591 shutdown 2024-04-25T18:50:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.499+0000 7f178841c740 1 -- 172.21.15.112:0/1139151424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 msgr2=0x561b95e4ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.499+0000 7f178841c740 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 0x561b95e4ca00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x561b95e57a60 tx=0x7f1770009260 comp rx=0 tx=0).stop 2024-04-25T18:50:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.500+0000 7f178841c740 1 -- 172.21.15.112:0/1139151424 shutdown_connections 2024-04-25T18:50:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.500+0000 7f178841c740 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b95e44a00 0x561b95e49f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.500+0000 7f178841c740 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561b95ebb780 0x561b95ebdba0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.500+0000 7f178841c740 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561b95e3a140 0x561b95e4cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.500+0000 7f178841c740 1 --2- 172.21.15.112:0/1139151424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b95e39820 0x561b95e4ca00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.500+0000 7f178841c740 1 -- 172.21.15.112:0/1139151424 >> 172.21.15.112:0/1139151424 conn(0x561b95e35d30 msgr2=0x561b95e3b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.500+0000 7f175fe00640 0 client.25591 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:50:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.502+0000 7f178841c740 1 -- 172.21.15.112:0/1139151424 shutdown_connections 2024-04-25T18:50:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.502+0000 7f178841c740 1 -- 172.21.15.112:0/1139151424 wait complete. 2024-04-25T18:50:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.502+0000 7f178841c740 20 client.25591 trim_cache size 0 max 16384 2024-04-25T18:50:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:50:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:50:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:50:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:50:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.503+0000 7f1785600640 1 -- 172.21.15.112:0/3881861738 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17640a3e40 msgr2=0x7f17640a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.503+0000 7f1785600640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17640a3e40 0x7f17640a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1770005ab0 tx=0x7f1770005a20 comp rx=0 tx=0).stop 2024-04-25T18:50:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.503+0000 7f1785600640 1 -- 172.21.15.112:0/3881861738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1780103b00 msgr2=0x7f17801a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.503+0000 7f1785600640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1780103b00 0x7f17801a0b60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f176c00d950 tx=0x7f176c00de10 comp rx=0 tx=0).stop 2024-04-25T18:50:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.503+0000 7f1785600640 1 -- 172.21.15.112:0/3881861738 shutdown_connections 2024-04-25T18:50:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.503+0000 7f1785600640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17640a3e40 0x7f17640a6300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.503+0000 7f1785600640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f178010c5b0 0x7f178019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.503+0000 7f1785600640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1780103b00 0x7f17801a0b60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.503+0000 7f1785600640 1 --2- 172.21.15.112:0/3881861738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17801031e0 0x7f17801a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.503+0000 7f1785600640 1 -- 172.21.15.112:0/3881861738 >> 172.21.15.112:0/3881861738 conn(0x7f1780077f60 msgr2=0x7f1780108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.504+0000 7f1785600640 1 -- 172.21.15.112:0/3881861738 shutdown_connections 2024-04-25T18:50:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:50:54.504+0000 7f1785600640 1 -- 172.21.15.112:0/3881861738 wait complete. 2024-04-25T18:50:54.554 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] randholes 2024-04-25T18:50:54.571 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] runas 2024-04-25T18:50:54.710 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] truncfile 2024-04-25T18:50:54.734 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] usemem 2024-04-25T18:50:54.890 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mmapcat 2024-04-25T18:50:55.005 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] append_reader 2024-04-25T18:50:55.018 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] append_writer 2024-04-25T18:50:55.039 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dirperf 2024-04-25T18:50:55.161 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] metaperf 2024-04-25T18:50:55.281 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] devzero 2024-04-25T18:50:55.308 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] feature 2024-04-25T18:50:55.431 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] alloc 2024-04-25T18:50:55.595 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fault 2024-04-25T18:50:55.620 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fstest 2024-04-25T18:50:55.652 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_access_root 2024-04-25T18:50:55.826 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] godown 2024-04-25T18:50:55.904 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] resvtest 2024-04-25T18:50:55.930 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] writemod 2024-04-25T18:50:56.013 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] writev_on_pagefault 2024-04-25T18:50:56.141 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] makeextents 2024-04-25T18:50:56.200 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] itrash 2024-04-25T18:50:56.242 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] rename 2024-04-25T18:50:56.319 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] multi_open_unlink 2024-04-25T18:50:56.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:55 smithi171 ceph-mon[33266]: pgmap v109: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 900 KiB/s wr, 207 op/s 2024-04-25T18:50:56.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:55 smithi161 ceph-mon[32609]: pgmap v109: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 900 KiB/s wr, 207 op/s 2024-04-25T18:50:56.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:55 smithi112 ceph-mon[27864]: pgmap v109: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 900 KiB/s wr, 207 op/s 2024-04-25T18:50:56.475 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] unwritten_sync 2024-04-25T18:50:56.490 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] genhashnames 2024-04-25T18:50:56.498 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_holes 2024-04-25T18:50:56.618 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_writev 2024-04-25T18:50:56.772 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_truncate_cmtime 2024-04-25T18:50:56.802 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dirhash_collide 2024-04-25T18:50:56.817 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_rename_overwrite 2024-04-25T18:50:56.915 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] holetest 2024-04-25T18:50:57.051 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_truncate_self 2024-04-25T18:50:57.078 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] af_unix 2024-04-25T18:50:57.132 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_stale_pmd 2024-04-25T18:50:57.305 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_cow_race 2024-04-25T18:50:57.311 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_fallocate 2024-04-25T18:50:57.391 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsync-err 2024-04-25T18:50:57.399 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_write_ro 2024-04-25T18:50:57.593 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-25T18:50:57.602 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-25T18:50:57.688 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_ofd_locks 2024-04-25T18:50:57.717 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_collision 2024-04-25T18:50:57.882 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mmap-write-concurrent 2024-04-25T18:50:57.893 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_get_file_time 2024-04-25T18:50:58.033 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_create_short_dirs 2024-04-25T18:50:58.097 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_create_long_dirs 2024-04-25T18:50:58.169 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_enospc 2024-04-25T18:50:58.200 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_writev_overlap 2024-04-25T18:50:58.275 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:57 smithi112 ceph-mon[27864]: pgmap v110: 97 pgs: 97 active+clean; 29 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 961 KiB/s wr, 267 op/s 2024-04-25T18:50:58.343 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] checkpoint_journal 2024-04-25T18:50:58.413 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mmap-rw-fault 2024-04-25T18:50:58.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:57 smithi171 ceph-mon[33266]: pgmap v110: 97 pgs: 97 active+clean; 29 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 961 KiB/s wr, 267 op/s 2024-04-25T18:50:58.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:57 smithi161 ceph-mon[32609]: pgmap v110: 97 pgs: 97 active+clean; 29 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 961 KiB/s wr, 267 op/s 2024-04-25T18:50:58.520 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] allocstale 2024-04-25T18:50:58.529 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_cow_memory_failure 2024-04-25T18:50:58.634 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fake-dump-rootino 2024-04-25T18:50:58.756 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dio-buf-fault 2024-04-25T18:50:58.829 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] rewinddir-test 2024-04-25T18:50:58.844 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] readdir-while-renames 2024-04-25T18:50:58.936 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] xfsctl 2024-04-25T18:50:59.038 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bstat 2024-04-25T18:50:59.125 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mtab 2024-04-25T18:50:59.138 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] getdevicesize 2024-04-25T18:50:59.286 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] preallo_rw_pattern_reader 2024-04-25T18:50:59.408 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] preallo_rw_pattern_writer 2024-04-25T18:50:59.418 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] ftrunc 2024-04-25T18:50:59.454 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] trunc 2024-04-25T18:50:59.632 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fs_perms 2024-04-25T18:50:59.728 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] testx 2024-04-25T18:50:59.741 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] looptest 2024-04-25T18:50:59.778 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] locktest 2024-04-25T18:50:59.970 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] unwritten_mmap 2024-04-25T18:51:00.024 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bulkstat_unlink_test 2024-04-25T18:51:00.152 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] deduperace 2024-04-25T18:51:00.303 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bulkstat_unlink_test_modified 2024-04-25T18:51:00.356 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_dir_offset 2024-04-25T18:51:00.392 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_futimens 2024-04-25T18:51:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:50:59 smithi171 ceph-mon[33266]: pgmap v111: 97 pgs: 97 active+clean; 29 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 739 KiB/s wr, 182 op/s 2024-04-25T18:51:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:50:59 smithi161 ceph-mon[32609]: pgmap v111: 97 pgs: 97 active+clean; 29 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 739 KiB/s wr, 182 op/s 2024-04-25T18:51:00.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:50:59 smithi112 ceph-mon[27864]: pgmap v111: 97 pgs: 97 active+clean; 29 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 739 KiB/s wr, 182 op/s 2024-04-25T18:51:00.532 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_immutable 2024-04-25T18:51:00.682 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] stale_handle 2024-04-25T18:51:00.708 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] pwrite_mmap_blocked 2024-04-25T18:51:00.733 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_dir_offset2 2024-04-25T18:51:01.031 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] seek_sanity_test 2024-04-25T18:51:01.040 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] seek_copy_test 2024-04-25T18:51:01.126 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_readdir_1 2024-04-25T18:51:01.385 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_readdir_2 2024-04-25T18:51:01.412 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fsync-tester 2024-04-25T18:51:01.506 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] nsexec 2024-04-25T18:51:01.658 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] cloner 2024-04-25T18:51:01.713 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] renameat2 2024-04-25T18:51:01.806 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_getcwd 2024-04-25T18:51:01.854 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] e4compact 2024-04-25T18:51:01.993 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] test-nextquota 2024-04-25T18:51:02.030 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] punch-alternating 2024-04-25T18:51:02.099 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-25T18:51:02.202 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] listxattr 2024-04-25T18:51:02.298 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dio-interleaved 2024-04-25T18:51:02.360 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_dir_type 2024-04-25T18:51:02.402 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] dio-invalidate-cache 2024-04-25T18:51:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:01 smithi171 ceph-mon[33266]: pgmap v112: 97 pgs: 97 active+clean; 30 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 786 KiB/s wr, 195 op/s 2024-04-25T18:51:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:01 smithi161 ceph-mon[32609]: pgmap v112: 97 pgs: 97 active+clean; 30 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 786 KiB/s wr, 195 op/s 2024-04-25T18:51:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:01 smithi112 ceph-mon[27864]: pgmap v112: 97 pgs: 97 active+clean; 30 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 786 KiB/s wr, 195 op/s 2024-04-25T18:51:02.488 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] stat_test 2024-04-25T18:51:02.589 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_encrypted_d_revalidate 2024-04-25T18:51:02.661 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] attr_replace_test 2024-04-25T18:51:02.753 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] swapon 2024-04-25T18:51:02.893 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mkswap 2024-04-25T18:51:02.953 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_attr_corruption 2024-04-25T18:51:02.978 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_open_tmpfiles 2024-04-25T18:51:03.079 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fscrypt-crypt-util 2024-04-25T18:51:03.189 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] bulkstat_null_ocount 2024-04-25T18:51:03.221 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] splice-test 2024-04-25T18:51:03.323 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] chprojid_fail 2024-04-25T18:51:03.474 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] detached_mounts_propagation 2024-04-25T18:51:03.534 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] ext4_resize 2024-04-25T18:51:03.625 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_readdir_3 2024-04-25T18:51:03.792 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] splice2pipe 2024-04-25T18:51:03.829 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] uuid_ioctl 2024-04-25T18:51:03.970 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-25T18:51:03.981 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fiemap-fault 2024-04-25T18:51:04.092 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fiemap-tester 2024-04-25T18:51:04.104 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_stripealign 2024-04-25T18:51:04.258 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] open_by_handle 2024-04-25T18:51:04.272 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_mmap_dio 2024-04-25T18:51:04.388 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] xfsfind 2024-04-25T18:51:04.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:03 smithi171 ceph-mon[33266]: pgmap v113: 97 pgs: 97 active+clean; 30 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 73 op/s 2024-04-25T18:51:04.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:03 smithi161 ceph-mon[32609]: pgmap v113: 97 pgs: 97 active+clean; 30 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 73 op/s 2024-04-25T18:51:04.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:03 smithi112 ceph-mon[27864]: pgmap v113: 97 pgs: 97 active+clean; 30 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 73 op/s 2024-04-25T18:51:04.528 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] t_reflink_read_race 2024-04-25T18:51:04.577 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] fssum 2024-04-25T18:51:04.667 INFO:tasks.workunit.client.0.smithi112.stdout:Building log-writes 2024-04-25T18:51:04.681 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] replay-log 2024-04-25T18:51:04.779 INFO:tasks.workunit.client.0.smithi112.stdout:Building perf 2024-04-25T18:51:04.787 INFO:tasks.workunit.client.0.smithi112.stdout:Building aio-dio-regress 2024-04-25T18:51:04.800 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aiocp 2024-04-25T18:51:04.855 INFO:tasks.workunit.client.0.smithi112.stdout:Building vfs 2024-04-25T18:51:04.867 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] vfstest 2024-04-25T18:51:05.184 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] mount-idmapped 2024-04-25T18:51:05.189 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-25T18:51:05.214 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-append-write-read-race 2024-04-25T18:51:05.491 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-cow-race 2024-04-25T18:51:05.511 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-cycle-write 2024-04-25T18:51:05.811 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-eof-race 2024-04-25T18:51:05.873 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-extend-stat 2024-04-25T18:51:05.880 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-fcntl-race 2024-04-25T18:51:06.136 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-hole-filling-race 2024-04-25T18:51:06.164 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-invalidate-failure 2024-04-25T18:51:06.184 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-invalidate-readahead 2024-04-25T18:51:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:05 smithi171 ceph-mon[33266]: pgmap v114: 97 pgs: 97 active+clean; 30 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 73 op/s 2024-04-25T18:51:06.436 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aiodio_sparse2 2024-04-25T18:51:06.453 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-subblock-eof-read 2024-04-25T18:51:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:05 smithi161 ceph-mon[32609]: pgmap v114: 97 pgs: 97 active+clean; 30 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 73 op/s 2024-04-25T18:51:06.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:05 smithi112 ceph-mon[27864]: pgmap v114: 97 pgs: 97 active+clean; 30 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 73 op/s 2024-04-25T18:51:06.493 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-dio-write-verify 2024-04-25T18:51:06.726 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-25T18:51:06.807 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-25T18:51:06.827 INFO:tasks.workunit.client.0.smithi112.stdout: [CC] aio-last-ref-held-by-io 2024-04-25T18:51:07.534 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd .. 2024-04-25T18:51:07.534 INFO:tasks.workunit.client.0.smithi112.stderr:+ cp xfstests-dev/ltp/fsx . 2024-04-25T18:51:07.541 INFO:tasks.workunit.client.0.smithi112.stderr:+ OPTIONS=-z 2024-04-25T18:51:07.541 INFO:tasks.workunit.client.0.smithi112.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-04-25T18:51:07.543 INFO:tasks.workunit.client.0.smithi112.stdout:Seed set to 1 2024-04-25T18:51:07.581 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T18:51:07.581 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T18:51:07.583 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T18:51:07.583 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T18:51:07.584 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T18:51:07.586 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T18:51:07.589 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T18:51:07.589 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-04-25T18:51:07.589 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-04-25T18:51:07.590 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x81d54 2024-04-25T18:51:07.595 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xb86d7 2024-04-25T18:51:07.607 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xe627e 2024-04-25T18:51:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.638+0000 7f6da5000640 1 -- 172.21.15.112:0/1540142807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6da006b380 msgr2=0x7f6da0106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.638+0000 7f6da5000640 1 --2- 172.21.15.112:0/1540142807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6da006b380 0x7f6da0106bb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009900 tx=0x7f6d8c02a3f0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.638+0000 7f6da5000640 1 -- 172.21.15.112:0/1540142807 shutdown_connections 2024-04-25T18:51:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.638+0000 7f6da5000640 1 --2- 172.21.15.112:0/1540142807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da0107290 0x7f6da010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.638+0000 7f6da5000640 1 --2- 172.21.15.112:0/1540142807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6da006b380 0x7f6da0106bb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.638+0000 7f6da5000640 1 --2- 172.21.15.112:0/1540142807 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6da006a9d0 0x7f6da006adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.638+0000 7f6da5000640 1 -- 172.21.15.112:0/1540142807 >> 172.21.15.112:0/1540142807 conn(0x7f6da0077dc0 msgr2=0x7f6da00781d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.638+0000 7f6da5000640 1 -- 172.21.15.112:0/1540142807 shutdown_connections 2024-04-25T18:51:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.638+0000 7f6da5000640 1 -- 172.21.15.112:0/1540142807 wait complete. 2024-04-25T18:51:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.639+0000 7f6da5000640 1 Processor -- start 2024-04-25T18:51:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.639+0000 7f6da5000640 1 -- start start 2024-04-25T18:51:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6da5000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6da006a9d0 0x7f6da01a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6da5000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da0107290 0x7f6da01a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6da5000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6da019a6b0 0x7f6da019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6da5000640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01121b0 con 0x7f6da0107290 2024-04-25T18:51:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6da5000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6da0112030 con 0x7f6da006a9d0 2024-04-25T18:51:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6da5000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6da0112330 con 0x7f6da019a6b0 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6d9de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6da006a9d0 0x7f6da01a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6d9d400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da0107290 0x7f6da01a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6d9de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6da006a9d0 0x7f6da01a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:36618/0 (socket says 172.21.15.112:36618) 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6d9e800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6da019a6b0 0x7f6da019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6d9de00640 1 -- 172.21.15.112:0/1332214298 learned_addr learned my addr 172.21.15.112:0/1332214298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6d9de00640 1 -- 172.21.15.112:0/1332214298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6da019a6b0 msgr2=0x7f6da019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6d9de00640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6da019a6b0 0x7f6da019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6d9de00640 1 -- 172.21.15.112:0/1332214298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da0107290 msgr2=0x7f6da01a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6d9de00640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da0107290 0x7f6da01a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.640+0000 7f6d9de00640 1 -- 172.21.15.112:0/1332214298 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da019b2b0 con 0x7f6da006a9d0 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.641+0000 7f6d9de00640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6da006a9d0 0x7f6da01a05f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000ea60 tx=0x7f6d9000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.641+0000 7f6d96a00640 1 -- 172.21.15.112:0/1332214298 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000cde0 con 0x7f6da006a9d0 2024-04-25T18:51:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.641+0000 7f6d96a00640 1 -- 172.21.15.112:0/1332214298 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d900035f0 con 0x7f6da006a9d0 2024-04-25T18:51:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.641+0000 7f6d96a00640 1 -- 172.21.15.112:0/1332214298 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90010690 con 0x7f6da006a9d0 2024-04-25T18:51:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.641+0000 7f6da5000640 1 -- 172.21.15.112:0/1332214298 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da01a7390 con 0x7f6da006a9d0 2024-04-25T18:51:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.641+0000 7f6da5000640 1 -- 172.21.15.112:0/1332214298 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da01a7740 con 0x7f6da006a9d0 2024-04-25T18:51:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.644+0000 7f6da7e2c740 1 Processor -- start 2024-04-25T18:51:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.644+0000 7f6da7e2c740 1 -- start start 2024-04-25T18:51:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.644+0000 7f6d96a00640 1 -- 172.21.15.112:0/1332214298 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f6d900037a0 con 0x7f6da006a9d0 2024-04-25T18:51:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.644+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 0x556260fbbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.644+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fc64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.644+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fc6a00 0x556260fc8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556260f21900 con 0x556260fc6a00 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556260ec7ef0 con 0x556260fbc140 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556260f1d8c0 con 0x556260fbb820 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6d9e800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 0x556260fbbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6d9de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fc64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6d9e800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 0x556260fbbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:41770/0 (socket says 172.21.15.112:41770) 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6d9e800640 1 -- 172.21.15.112:0/4198442778 learned_addr learned my addr 172.21.15.112:0/4198442778 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6d9d400640 1 --2- 172.21.15.112:0/4198442778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fc6a00 0x556260fc8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6d9e800640 1 -- 172.21.15.112:0/4198442778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 msgr2=0x556260fc64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6d9e800640 1 --2- 172.21.15.112:0/4198442778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fc64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6d9e800640 1 -- 172.21.15.112:0/4198442778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fc6a00 msgr2=0x556260fc8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6d9e800640 1 --2- 172.21.15.112:0/4198442778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fc6a00 0x556260fc8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.645+0000 7f6d9e800640 1 -- 172.21.15.112:0/4198442778 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556260c34210 con 0x556260fbb820 2024-04-25T18:51:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.646+0000 7f6d9e800640 1 --2- 172.21.15.112:0/4198442778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 0x556260fbbc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009f50 tx=0x7f6d980118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe510c29fcaad924 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.646+0000 7f6d8b400640 1 -- 172.21.15.112:0/4198442778 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98004d20 con 0x556260fbb820 2024-04-25T18:51:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.646+0000 7f6d8b400640 1 -- 172.21.15.112:0/4198442778 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d98004050 con 0x556260fbb820 2024-04-25T18:51:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.646+0000 7f6d8b400640 1 -- 172.21.15.112:0/4198442778 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98005280 con 0x556260fbb820 2024-04-25T18:51:07.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.646+0000 7f6d96a00640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d840a3e40 0x7f6d840a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.646+0000 7f6d96a00640 1 -- 172.21.15.112:0/1332214298 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f6d900d0ab0 con 0x7f6da006a9d0 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6d9de00640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d840a3e40 0x7f6d840a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6da7e2c740 1 -- 172.21.15.112:0/4198442778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 msgr2=0x556260fbbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/4198442778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 0x556260fbbc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009f50 tx=0x7f6d980118d0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6da7e2c740 1 -- 172.21.15.112:0/4198442778 shutdown_connections 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/4198442778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fc6a00 0x556260fc8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/4198442778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fc64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/4198442778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 0x556260fbbc00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6da7e2c740 1 -- 172.21.15.112:0/4198442778 >> 172.21.15.112:0/4198442778 conn(0x556260fb7d30 msgr2=0x556260fb8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6da7e2c740 1 -- 172.21.15.112:0/4198442778 shutdown_connections 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6da7e2c740 1 -- 172.21.15.112:0/4198442778 wait complete. 2024-04-25T18:51:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6d9de00640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d840a3e40 0x7f6d840a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x556260fbcdc0 tx=0x7f6d900d4560 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.647+0000 7f6da7e2c740 1 -- start start 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6da7e2c740 1 Processor -- start 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 0x556260fc3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fc3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fc6a00 0x556260fc43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556260fc9630 con 0x556260fc6a00 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556260edb310 con 0x556260fbc140 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556260fc97b0 con 0x556260fbb820 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6da7e2c740 10 client.0 _ll_register_callbacks cb 0x556260ea9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6d9d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fc3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6d9d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fc3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:36632/0 (socket says 172.21.15.112:36632) 2024-04-25T18:51:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6d9d400640 1 -- 172.21.15.112:0/594047060 learned_addr learned my addr 172.21.15.112:0/594047060 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.648+0000 7f6d9e800640 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 0x556260fc3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.649+0000 7f6d9e800640 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fc6a00 0x556260fc43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.649+0000 7f6d9d400640 1 -- 172.21.15.112:0/594047060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 msgr2=0x556260fc3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.649+0000 7f6d9d400640 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 0x556260fc3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.649+0000 7f6d9d400640 1 -- 172.21.15.112:0/594047060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fc6a00 msgr2=0x556260fc43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.649+0000 7f6d9d400640 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fc6a00 0x556260fc43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.649+0000 7f6d9d400640 1 -- 172.21.15.112:0/594047060 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556260d00620 con 0x556260fbc140 2024-04-25T18:51:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.649+0000 7f6d9d400640 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fc3e90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009900 tx=0x7f6d8c032de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a816bfc93aba04b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.649+0000 7f6d7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:51:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.649+0000 7f6d7fe00640 1 -- 172.21.15.112:0/594047060 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c035040 con 0x556260fbc140 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6d7fe00640 1 -- 172.21.15.112:0/594047060 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f6d8c031760 con 0x556260fbc140 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6da7e2c740 1 -- 172.21.15.112:0/594047060 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d98004050 con 0x556260fbc140 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6d7fe00640 1 -- 172.21.15.112:0/594047060 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d8c004040 con 0x556260fbc140 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6da7e2c740 10 client.25586 fetch_fsmap learned FSMap version 30 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6da7e2c740 1 -- 172.21.15.112:0/594047060 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556260fd5840 con 0x556260fbc140 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6d7fe00640 1 -- 172.21.15.112:0/594047060 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d98004050 con 0x556260fbc140 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6d7fe00640 1 -- 172.21.15.112:0/594047060 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6d8c004750 con 0x556260fbc140 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6da7e2c740 10 client.25586 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6da7e2c740 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6da7e2c740 20 client.25586 populate_metadata read hostname 'smithi112' 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556260fd6d50 0x556260fd9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6da7e2c740 4 client.25586 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.650+0000 7f6da7e2c740 1 -- 172.21.15.112:0/594047060 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556260f358b0 con 0x556260fd6d50 2024-04-25T18:51:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.651+0000 7f6d9e800640 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556260fd6d50 0x556260fd9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.651+0000 7f6d9e800640 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556260fd6d50 0x556260fd9160 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=667e513fc418ac3b server_cookie=eb8524b5b64940af in_seq=0 out_seq=0 2024-04-25T18:51:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.651+0000 7f6d7fe00640 10 client.25586 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.652+0000 7f6d7fe00640 1 -- 172.21.15.112:0/594047060 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556260f358b0 con 0x556260fd6d50 2024-04-25T18:51:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.652+0000 7f6d7fe00640 10 client.25586 handle_command_reply: tid=0 2024-04-25T18:51:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 1 client.25586 shutdown 2024-04-25T18:51:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 1 -- 172.21.15.112:0/594047060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 msgr2=0x556260fc3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fc3e90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009900 tx=0x7f6d8c032de0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 1 -- 172.21.15.112:0/594047060 shutdown_connections 2024-04-25T18:51:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556260fd6d50 0x556260fd9160 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fc6a00 0x556260fc43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fc3e90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6d7fe00640 0 client.25586 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/594047060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fbb820 0x556260fc3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 1 -- 172.21.15.112:0/594047060 >> 172.21.15.112:0/594047060 conn(0x556260fb7d30 msgr2=0x556260fbda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 1 -- 172.21.15.112:0/594047060 shutdown_connections 2024-04-25T18:51:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 1 -- 172.21.15.112:0/594047060 wait complete. 2024-04-25T18:51:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.653+0000 7f6da7e2c740 20 client.25586 trim_cache size 0 max 16384 2024-04-25T18:51:07.660 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0xf3194 2024-04-25T18:51:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6da7e2c740 1 Processor -- start 2024-04-25T18:51:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6da7e2c740 1 -- start start 2024-04-25T18:51:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fbb820 0x556260fd24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fbe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fc6a00 0x556260fc4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01b2720 con 0x556260fbb820 2024-04-25T18:51:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01aff90 con 0x556260fbc140 2024-04-25T18:51:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01adde0 con 0x556260fc6a00 2024-04-25T18:51:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6d9d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fbe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6d9e800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fbb820 0x556260fd24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6d9d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fbe1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:36640/0 (socket says 172.21.15.112:36640) 2024-04-25T18:51:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.670+0000 7f6d9d400640 1 -- 172.21.15.112:0/3418445493 learned_addr learned my addr 172.21.15.112:0/3418445493 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.671+0000 7f6d9e800640 1 --2- 172.21.15.112:0/3418445493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fc6a00 0x556260fc4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.671+0000 7f6d9d400640 1 -- 172.21.15.112:0/3418445493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fc6a00 msgr2=0x556260fc4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.671+0000 7f6d9d400640 1 --2- 172.21.15.112:0/3418445493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fc6a00 0x556260fc4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.671+0000 7f6d9d400640 1 -- 172.21.15.112:0/3418445493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fbb820 msgr2=0x556260fd24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.671+0000 7f6d9d400640 1 --2- 172.21.15.112:0/3418445493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fbb820 0x556260fd24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.671+0000 7f6d9d400640 1 -- 172.21.15.112:0/3418445493 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556260fbe720 con 0x556260fbc140 2024-04-25T18:51:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.671+0000 7f6d9e800640 1 --2- 172.21.15.112:0/3418445493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fbb820 0x556260fd24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:51:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.671+0000 7f6d9d400640 1 --2- 172.21.15.112:0/3418445493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fbe1e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c004ce0 tx=0x7f6d8c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b664770a243ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.671+0000 7f6d8aa00640 1 -- 172.21.15.112:0/3418445493 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c035070 con 0x556260fbc140 2024-04-25T18:51:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.671+0000 7f6d8aa00640 1 -- 172.21.15.112:0/3418445493 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d8c0048c0 con 0x556260fbc140 2024-04-25T18:51:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.671+0000 7f6d8aa00640 1 -- 172.21.15.112:0/3418445493 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c053070 con 0x556260fbc140 2024-04-25T18:51:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.672+0000 7f6da7e2c740 1 -- 172.21.15.112:0/3418445493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 msgr2=0x556260fbe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.672+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/3418445493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fbe1e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c004ce0 tx=0x7f6d8c02a3f0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.672+0000 7f6da7e2c740 1 -- 172.21.15.112:0/3418445493 shutdown_connections 2024-04-25T18:51:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.672+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/3418445493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fc6a00 0x556260fc4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.672+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/3418445493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fbe1e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.672+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/3418445493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fbb820 0x556260fd24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.672+0000 7f6da7e2c740 1 -- 172.21.15.112:0/3418445493 >> 172.21.15.112:0/3418445493 conn(0x556260fb7d30 msgr2=0x556260fc3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.672+0000 7f6da7e2c740 1 -- 172.21.15.112:0/3418445493 shutdown_connections 2024-04-25T18:51:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.672+0000 7f6da7e2c740 1 -- 172.21.15.112:0/3418445493 wait complete. 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.673+0000 7f6da7e2c740 1 -- start start 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.673+0000 7f6da7e2c740 1 Processor -- start 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.673+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fbb820 0x556260fcbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.673+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fcc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.673+0000 7f6da7e2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fc6a00 0x55626103b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.673+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01ae450 con 0x556260fbb820 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.673+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01af8c0 con 0x556260fbc140 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.673+0000 7f6da7e2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01aeb20 con 0x556260fc6a00 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.673+0000 7f6da7e2c740 10 client.0 _ll_register_callbacks cb 0x7f6d8c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.673+0000 7f6d9d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fcc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d9e800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fbb820 0x556260fcbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d9d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fcc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:36642/0 (socket says 172.21.15.112:36642) 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d9d400640 1 -- 172.21.15.112:0/1788350639 learned_addr learned my addr 172.21.15.112:0/1788350639 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d9e800640 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fc6a00 0x55626103b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d9d400640 1 -- 172.21.15.112:0/1788350639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fc6a00 msgr2=0x55626103b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d9d400640 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fc6a00 0x55626103b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d9d400640 1 -- 172.21.15.112:0/1788350639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fbb820 msgr2=0x556260fcbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d9d400640 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fbb820 0x556260fcbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d9d400640 1 -- 172.21.15.112:0/1788350639 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55626103b820 con 0x556260fbc140 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d9e800640 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fc6a00 0x55626103b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d9d400640 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fcc100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c004af0 tx=0x7f6d8c003fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be755be41bf54e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d7fe00640 1 -- 172.21.15.112:0/1788350639 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c035030 con 0x556260fbc140 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6da7e2c740 1 -- 172.21.15.112:0/1788350639 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d8c0048c0 con 0x556260fbc140 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.674+0000 7f6d7fe00640 1 -- 172.21.15.112:0/1788350639 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f6d8c003ac0 con 0x556260fbc140 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6d7fe00640 1 -- 172.21.15.112:0/1788350639 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d8c066050 con 0x556260fbc140 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6da7e2c740 10 client.25592 fetch_fsmap learned FSMap version 30 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6da7e2c740 1 -- 172.21.15.112:0/1788350639 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55626103ca30 con 0x556260fbc140 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6d7fe00640 1 -- 172.21.15.112:0/1788350639 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d8c0048c0 con 0x556260fbc140 2024-04-25T18:51:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6d7fe00640 1 -- 172.21.15.112:0/1788350639 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6d8c031760 con 0x556260fbc140 2024-04-25T18:51:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6da7e2c740 10 client.25592 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:51:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6da7e2c740 10 client.25592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:51:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6da7e2c740 20 client.25592 populate_metadata read hostname 'smithi112' 2024-04-25T18:51:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55626103d0c0 0x55626103f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6da7e2c740 4 client.25592 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:51:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6da7e2c740 1 -- 172.21.15.112:0/1788350639 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556260f2f320 con 0x55626103d0c0 2024-04-25T18:51:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.675+0000 7f6d9e800640 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55626103d0c0 0x55626103f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.676+0000 7f6d9e800640 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55626103d0c0 0x55626103f4e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ce360c1f471cce server_cookie=ac2c987016e81e2c in_seq=0 out_seq=0 2024-04-25T18:51:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.676+0000 7f6d7fe00640 10 client.25592 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:07.681 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xf3ceb 2024-04-25T18:51:07.721 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0xf47e4 2024-04-25T18:51:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6d7fe00640 1 -- 172.21.15.112:0/1788350639 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556260f2f320 con 0x55626103d0c0 2024-04-25T18:51:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6d7fe00640 10 client.25592 handle_command_reply: tid=0 2024-04-25T18:51:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6da7e2c740 1 client.25592 shutdown 2024-04-25T18:51:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6da7e2c740 1 -- 172.21.15.112:0/1788350639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 msgr2=0x556260fcc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fcc100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c004af0 tx=0x7f6d8c003fc0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6da7e2c740 1 -- 172.21.15.112:0/1788350639 shutdown_connections 2024-04-25T18:51:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556260fc6a00 0x55626103b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556260fbc140 0x556260fcc100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55626103d0c0 0x55626103f4e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6da7e2c740 1 --2- 172.21.15.112:0/1788350639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556260fbb820 0x556260fcbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6d7fe00640 0 client.25592 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.727+0000 7f6da7e2c740 1 -- 172.21.15.112:0/1788350639 >> 172.21.15.112:0/1788350639 conn(0x556260fb7d30 msgr2=0x556260fc38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.728+0000 7f6da7e2c740 1 -- 172.21.15.112:0/1788350639 shutdown_connections 2024-04-25T18:51:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.728+0000 7f6da7e2c740 1 -- 172.21.15.112:0/1788350639 wait complete. 2024-04-25T18:51:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.728+0000 7f6da7e2c740 20 client.25592 trim_cache size 0 max 16384 2024-04-25T18:51:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:51:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:51:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:51:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:51:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 -- 172.21.15.112:0/1332214298 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d840a3e40 msgr2=0x7f6d840a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d840a3e40 0x7f6d840a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x556260fbcdc0 tx=0x7f6d900d4560 comp rx=0 tx=0).stop 2024-04-25T18:51:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 -- 172.21.15.112:0/1332214298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6da006a9d0 msgr2=0x7f6da01a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6da006a9d0 0x7f6da01a05f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000ea60 tx=0x7f6d9000ef20 comp rx=0 tx=0).stop 2024-04-25T18:51:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 -- 172.21.15.112:0/1332214298 shutdown_connections 2024-04-25T18:51:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d840a3e40 0x7f6d840a6300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6da019a6b0 0x7f6da019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6da0107290 0x7f6da01a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 --2- 172.21.15.112:0/1332214298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6da006a9d0 0x7f6da01a05f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 -- 172.21.15.112:0/1332214298 >> 172.21.15.112:0/1332214298 conn(0x7f6da0077dc0 msgr2=0x7f6da010d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 -- 172.21.15.112:0/1332214298 shutdown_connections 2024-04-25T18:51:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:07.729+0000 7f6da5000640 1 -- 172.21.15.112:0/1332214298 wait complete. 2024-04-25T18:51:07.821 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xfaccc 2024-04-25T18:51:08.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:07 smithi171 ceph-mon[33266]: pgmap v115: 97 pgs: 97 active+clean; 34 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 516 KiB/s wr, 95 op/s 2024-04-25T18:51:08.459 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xfe423 2024-04-25T18:51:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:07 smithi161 ceph-mon[32609]: pgmap v115: 97 pgs: 97 active+clean; 34 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 516 KiB/s wr, 95 op/s 2024-04-25T18:51:08.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:07 smithi112 ceph-mon[27864]: pgmap v115: 97 pgs: 97 active+clean; 34 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 516 KiB/s wr, 95 op/s 2024-04-25T18:51:10.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:10 smithi171 ceph-mon[33266]: pgmap v116: 97 pgs: 97 active+clean; 34 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 455 KiB/s wr, 35 op/s 2024-04-25T18:51:10.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:10 smithi161 ceph-mon[32609]: pgmap v116: 97 pgs: 97 active+clean; 34 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 455 KiB/s wr, 35 op/s 2024-04-25T18:51:10.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:10 smithi112 ceph-mon[27864]: pgmap v116: 97 pgs: 97 active+clean; 34 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 455 KiB/s wr, 35 op/s 2024-04-25T18:51:12.257 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xfe492 2024-04-25T18:51:12.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:12 smithi171 ceph-mon[33266]: pgmap v117: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-04-25T18:51:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:12 smithi161 ceph-mon[32609]: pgmap v117: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-04-25T18:51:12.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:12 smithi112 ceph-mon[27864]: pgmap v117: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-04-25T18:51:14.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:14 smithi171 ceph-mon[33266]: pgmap v118: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-04-25T18:51:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:14 smithi161 ceph-mon[32609]: pgmap v118: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-04-25T18:51:14.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:14 smithi112 ceph-mon[27864]: pgmap v118: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-04-25T18:51:15.494 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xff038 2024-04-25T18:51:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:16 smithi171 ceph-mon[33266]: pgmap v119: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-04-25T18:51:16.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:16 smithi161 ceph-mon[32609]: pgmap v119: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-04-25T18:51:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:16 smithi112 ceph-mon[27864]: pgmap v119: 97 pgs: 97 active+clean; 38 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-04-25T18:51:16.918 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0xffcac 2024-04-25T18:51:17.114 INFO:tasks.workunit.client.0.smithi112.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-04-25T18:51:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:18 smithi171 ceph-mon[33266]: pgmap v120: 97 pgs: 97 active+clean; 40 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 393 op/s 2024-04-25T18:51:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:18 smithi161 ceph-mon[32609]: pgmap v120: 97 pgs: 97 active+clean; 40 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 393 op/s 2024-04-25T18:51:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:18 smithi112 ceph-mon[27864]: pgmap v120: 97 pgs: 97 active+clean; 40 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 393 op/s 2024-04-25T18:51:20.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:20 smithi171 ceph-mon[33266]: pgmap v121: 97 pgs: 97 active+clean; 40 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 370 op/s 2024-04-25T18:51:20.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:20 smithi161 ceph-mon[32609]: pgmap v121: 97 pgs: 97 active+clean; 40 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 370 op/s 2024-04-25T18:51:20.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:20 smithi112 ceph-mon[27864]: pgmap v121: 97 pgs: 97 active+clean; 40 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 370 op/s 2024-04-25T18:51:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.873+0000 7fac79000640 1 -- 172.21.15.112:0/4241308296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac7410c330 msgr2=0x7fac7410e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.873+0000 7fac79000640 1 --2- 172.21.15.112:0/4241308296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac7410c330 0x7fac7410e7d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009ca0 tx=0x7fac6c02a940 comp rx=0 tx=0).stop 2024-04-25T18:51:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.873+0000 7fac79000640 1 -- 172.21.15.112:0/4241308296 shutdown_connections 2024-04-25T18:51:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.873+0000 7fac79000640 1 --2- 172.21.15.112:0/4241308296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac7410c330 0x7fac7410e7d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.873+0000 7fac79000640 1 --2- 172.21.15.112:0/4241308296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac74103830 0x7fac7410bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.873+0000 7fac79000640 1 --2- 172.21.15.112:0/4241308296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac74102f10 0x7fac741032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.873+0000 7fac79000640 1 -- 172.21.15.112:0/4241308296 >> 172.21.15.112:0/4241308296 conn(0x7fac740fc770 msgr2=0x7fac740feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.873+0000 7fac79000640 1 -- 172.21.15.112:0/4241308296 shutdown_connections 2024-04-25T18:51:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.874+0000 7fac79000640 1 -- 172.21.15.112:0/4241308296 wait complete. 2024-04-25T18:51:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.874+0000 7fac79000640 1 Processor -- start 2024-04-25T18:51:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac79000640 1 -- start start 2024-04-25T18:51:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac79000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac74102f10 0x7fac7419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac79000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac74103830 0x7fac7419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac79000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac7410c330 0x7fac7419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac79000640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac74112010 con 0x7fac74103830 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac79000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fac74111e90 con 0x7fac7410c330 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac79000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fac74112190 con 0x7fac74102f10 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac73400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac74102f10 0x7fac7419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac72a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac74103830 0x7fac7419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac73e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac7410c330 0x7fac7419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac73400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac74102f10 0x7fac7419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:40004/0 (socket says 172.21.15.112:40004) 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.875+0000 7fac73400640 1 -- 172.21.15.112:0/3087785579 learned_addr learned my addr 172.21.15.112:0/3087785579 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac73e00640 1 -- 172.21.15.112:0/3087785579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac74102f10 msgr2=0x7fac7419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac73e00640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac74102f10 0x7fac7419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac73e00640 1 -- 172.21.15.112:0/3087785579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac74103830 msgr2=0x7fac7419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac73e00640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac74103830 0x7fac7419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac73e00640 1 -- 172.21.15.112:0/3087785579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7419f750 con 0x7fac7410c330 2024-04-25T18:51:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac73400640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac74102f10 0x7fac7419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:51:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac73e00640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac7410c330 0x7fac7419f020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fac6c005e30 tx=0x7fac6c0308e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac6be00640 1 -- 172.21.15.112:0/3087785579 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c034040 con 0x7fac7410c330 2024-04-25T18:51:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac6be00640 1 -- 172.21.15.112:0/3087785579 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac6c02fd10 con 0x7fac7410c330 2024-04-25T18:51:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac79000640 1 -- 172.21.15.112:0/3087785579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7419f9e0 con 0x7fac7410c330 2024-04-25T18:51:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac6be00640 1 -- 172.21.15.112:0/3087785579 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c004540 con 0x7fac7410c330 2024-04-25T18:51:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.876+0000 7fac79000640 1 -- 172.21.15.112:0/3087785579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac741a72d0 con 0x7fac7410c330 2024-04-25T18:51:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.880+0000 7fac7d482740 1 Processor -- start 2024-04-25T18:51:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.880+0000 7fac6be00640 1 -- 172.21.15.112:0/3087785579 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fac6c037020 con 0x7fac7410c330 2024-04-25T18:51:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.880+0000 7fac7d482740 1 -- start start 2024-04-25T18:51:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.880+0000 7fac7d482740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.880+0000 7fac7d482740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b512d0 0x564e61b5b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.880+0000 7fac7d482740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 0x564e61b5df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.880+0000 7fac7d482740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e61ab6900 con 0x564e61b512d0 2024-04-25T18:51:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.880+0000 7fac7d482740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564e61a5cef0 con 0x564e61b509b0 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.880+0000 7fac7d482740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564e61ab28c0 con 0x564e61b5bb90 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.881+0000 7fac72a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.881+0000 7fac73400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b512d0 0x564e61b5b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.881+0000 7fac72a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b50d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:52632/0 (socket says 172.21.15.112:52632) 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.881+0000 7fac72a00640 1 -- 172.21.15.112:0/410542657 learned_addr learned my addr 172.21.15.112:0/410542657 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.881+0000 7fac72a00640 1 --2- 172.21.15.112:0/410542657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 0x564e61b5df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.881+0000 7fac73400640 1 -- 172.21.15.112:0/410542657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 msgr2=0x564e61b5df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.881+0000 7fac73400640 1 --2- 172.21.15.112:0/410542657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 0x564e61b5df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.881+0000 7fac73400640 1 -- 172.21.15.112:0/410542657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 msgr2=0x564e61b50d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.881+0000 7fac73400640 1 --2- 172.21.15.112:0/410542657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b50d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.881+0000 7fac73400640 1 -- 172.21.15.112:0/410542657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e617c9210 con 0x564e61b512d0 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.881+0000 7fac72a00640 1 --2- 172.21.15.112:0/410542657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b50d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:51:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.882+0000 7fac73400640 1 --2- 172.21.15.112:0/410542657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b512d0 0x564e61b5b650 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fac64011510 tx=0x7fac640119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f95c1792356308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.882+0000 7fac68c00640 1 -- 172.21.15.112:0/410542657 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64005020 con 0x564e61b512d0 2024-04-25T18:51:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.882+0000 7fac6be00640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fac580a3d10 0x7fac580a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.882+0000 7fac68c00640 1 -- 172.21.15.112:0/410542657 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac640047a0 con 0x564e61b512d0 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.882+0000 7fac68c00640 1 -- 172.21.15.112:0/410542657 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6400f880 con 0x564e61b512d0 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.882+0000 7fac6be00640 1 -- 172.21.15.112:0/3087785579 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fac6c0f14d0 con 0x7fac7410c330 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.882+0000 7fac72a00640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fac580a3d10 0x7fac580a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.883+0000 7fac7d482740 1 -- 172.21.15.112:0/410542657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b512d0 msgr2=0x564e61b5b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.883+0000 7fac7d482740 1 --2- 172.21.15.112:0/410542657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b512d0 0x564e61b5b650 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fac64011510 tx=0x7fac640119d0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.883+0000 7fac7d482740 1 -- 172.21.15.112:0/410542657 shutdown_connections 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.883+0000 7fac7d482740 1 --2- 172.21.15.112:0/410542657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 0x564e61b5df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.883+0000 7fac7d482740 1 --2- 172.21.15.112:0/410542657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b512d0 0x564e61b5b650 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.883+0000 7fac7d482740 1 --2- 172.21.15.112:0/410542657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b50d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.883+0000 7fac7d482740 1 -- 172.21.15.112:0/410542657 >> 172.21.15.112:0/410542657 conn(0x564e61b4cec0 msgr2=0x564e61b4d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.883+0000 7fac72a00640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fac580a3d10 0x7fac580a61d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac60005f60 tx=0x7fac60005850 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.883+0000 7fac7d482740 1 -- 172.21.15.112:0/410542657 shutdown_connections 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.883+0000 7fac7d482740 1 -- 172.21.15.112:0/410542657 wait complete. 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.884+0000 7fac7d482740 1 -- start start 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.884+0000 7fac7d482740 1 Processor -- start 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.884+0000 7fac7d482740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b58ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.884+0000 7fac7d482740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b512d0 0x564e61b59010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.884+0000 7fac7d482740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 0x564e61b59550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.884+0000 7fac7d482740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e61b5e7c0 con 0x564e61b512d0 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.884+0000 7fac7d482740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564e61a70310 con 0x564e61b509b0 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.884+0000 7fac7d482740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564e61b5e940 con 0x564e61b5bb90 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.884+0000 7fac7d482740 10 client.0 _ll_register_callbacks cb 0x564e61a3ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac73e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 0x564e61b59550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac73400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b58ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac73e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 0x564e61b59550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:40012/0 (socket says 172.21.15.112:40012) 2024-04-25T18:51:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac73e00640 1 -- 172.21.15.112:0/83355663 learned_addr learned my addr 172.21.15.112:0/83355663 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac72a00640 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b512d0 0x564e61b59010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac73e00640 1 -- 172.21.15.112:0/83355663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 msgr2=0x564e61b58ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac73e00640 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b58ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac73e00640 1 -- 172.21.15.112:0/83355663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b512d0 msgr2=0x564e61b59010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac73e00640 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b512d0 0x564e61b59010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac73e00640 1 -- 172.21.15.112:0/83355663 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e61895620 con 0x564e61b5bb90 2024-04-25T18:51:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac73400640 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b58ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:51:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac73e00640 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 0x564e61b59550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fac6c004af0 tx=0x7fac6c0bd370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fe36dcb58eda428 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac5d600640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:51:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.885+0000 7fac5d600640 1 -- 172.21.15.112:0/83355663 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0bdc20 con 0x564e61b5bb90 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac7d482740 1 -- 172.21.15.112:0/83355663 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac640047a0 con 0x564e61b5bb90 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac5d600640 1 -- 172.21.15.112:0/83355663 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fac6c0b9b90 con 0x564e61b5bb90 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac5d600640 1 -- 172.21.15.112:0/83355663 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fac6c0db050 con 0x564e61b5bb90 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac7d482740 10 client.25621 fetch_fsmap learned FSMap version 30 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac7d482740 1 -- 172.21.15.112:0/83355663 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564e61b5e870 con 0x564e61b5bb90 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac5d600640 1 -- 172.21.15.112:0/83355663 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac640047a0 con 0x564e61b5bb90 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac5d600640 1 -- 172.21.15.112:0/83355663 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fac6c02f070 con 0x564e61b5bb90 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac7d482740 10 client.25621 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac7d482740 10 client.25621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac7d482740 20 client.25621 populate_metadata read hostname 'smithi112' 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac7d482740 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564e61b6bf40 0x564e61b6e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac7d482740 4 client.25621 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:51:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.886+0000 7fac7d482740 1 -- 172.21.15.112:0/83355663 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e61aca8b0 con 0x564e61b6bf40 2024-04-25T18:51:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.887+0000 7fac73400640 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564e61b6bf40 0x564e61b6e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.887+0000 7fac73400640 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564e61b6bf40 0x564e61b6e3c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe49695392224541 server_cookie=bc8e8ae1f24899ee in_seq=0 out_seq=0 2024-04-25T18:51:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.887+0000 7fac5d600640 10 client.25621 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac5d600640 1 -- 172.21.15.112:0/83355663 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e61aca8b0 con 0x564e61b6bf40 2024-04-25T18:51:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac5d600640 10 client.25621 handle_command_reply: tid=0 2024-04-25T18:51:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 1 client.25621 shutdown 2024-04-25T18:51:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 1 -- 172.21.15.112:0/83355663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 msgr2=0x564e61b59550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 0x564e61b59550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fac6c004af0 tx=0x7fac6c0bd370 comp rx=0 tx=0).stop 2024-04-25T18:51:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 1 -- 172.21.15.112:0/83355663 shutdown_connections 2024-04-25T18:51:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b5bb90 0x564e61b59550 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564e61b6bf40 0x564e61b6e3c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b512d0 0x564e61b59010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 1 --2- 172.21.15.112:0/83355663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b58ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 1 -- 172.21.15.112:0/83355663 >> 172.21.15.112:0/83355663 conn(0x564e61b4cec0 msgr2=0x564e61b52c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac5d600640 0 client.25621 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 1 -- 172.21.15.112:0/83355663 shutdown_connections 2024-04-25T18:51:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 1 -- 172.21.15.112:0/83355663 wait complete. 2024-04-25T18:51:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.889+0000 7fac7d482740 20 client.25621 trim_cache size 0 max 16384 2024-04-25T18:51:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.905+0000 7fac7d482740 1 Processor -- start 2024-04-25T18:51:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac7d482740 1 -- start start 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac7d482740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b509b0 0x564e61b67680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac7d482740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b512d0 0x564e61b57af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac7d482740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b5bb90 0x564e61b73c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac7d482740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac741b2530 con 0x564e61b509b0 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac7d482740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fac741afda0 con 0x564e61b5bb90 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac7d482740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fac741adbf0 con 0x564e61b512d0 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac73400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b509b0 0x564e61b67680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac72a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b512d0 0x564e61b57af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac73e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b5bb90 0x564e61b73c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac73e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b5bb90 0x564e61b73c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:52638/0 (socket says 172.21.15.112:52638) 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac73400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b509b0 0x564e61b67680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47356/0 (socket says 172.21.15.112:47356) 2024-04-25T18:51:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac73e00640 1 -- 172.21.15.112:0/2285488844 learned_addr learned my addr 172.21.15.112:0/2285488844 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.906+0000 7fac72a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b512d0 0x564e61b57af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:40028/0 (socket says 172.21.15.112:40028) 2024-04-25T18:51:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.907+0000 7fac73400640 1 -- 172.21.15.112:0/2285488844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b512d0 msgr2=0x564e61b57af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.907+0000 7fac73400640 1 --2- 172.21.15.112:0/2285488844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b512d0 0x564e61b57af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.907+0000 7fac73400640 1 -- 172.21.15.112:0/2285488844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b5bb90 msgr2=0x564e61b73c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.907+0000 7fac73400640 1 --2- 172.21.15.112:0/2285488844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b5bb90 0x564e61b73c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.907+0000 7fac73400640 1 -- 172.21.15.112:0/2285488844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e61b5e6a0 con 0x564e61b509b0 2024-04-25T18:51:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.907+0000 7fac73400640 1 --2- 172.21.15.112:0/2285488844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b509b0 0x564e61b67680 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac6400a010 tx=0x7fac64009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d7320858d0400b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.907+0000 7fac5fe00640 1 -- 172.21.15.112:0/2285488844 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64033070 con 0x564e61b509b0 2024-04-25T18:51:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.908+0000 7fac5fe00640 1 -- 172.21.15.112:0/2285488844 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac64004570 con 0x564e61b509b0 2024-04-25T18:51:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.908+0000 7fac5fe00640 1 -- 172.21.15.112:0/2285488844 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64005320 con 0x564e61b509b0 2024-04-25T18:51:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.908+0000 7fac7d482740 1 -- 172.21.15.112:0/2285488844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b509b0 msgr2=0x564e61b67680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.908+0000 7fac7d482740 1 --2- 172.21.15.112:0/2285488844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b509b0 0x564e61b67680 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac6400a010 tx=0x7fac64009470 comp rx=0 tx=0).stop 2024-04-25T18:51:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.908+0000 7fac7d482740 1 -- 172.21.15.112:0/2285488844 shutdown_connections 2024-04-25T18:51:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.908+0000 7fac7d482740 1 --2- 172.21.15.112:0/2285488844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b5bb90 0x564e61b73c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.908+0000 7fac7d482740 1 --2- 172.21.15.112:0/2285488844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b512d0 0x564e61b57af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.908+0000 7fac7d482740 1 --2- 172.21.15.112:0/2285488844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b509b0 0x564e61b67680 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.908+0000 7fac7d482740 1 -- 172.21.15.112:0/2285488844 >> 172.21.15.112:0/2285488844 conn(0x564e61b4cec0 msgr2=0x564e61b573e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.909+0000 7fac7d482740 1 -- 172.21.15.112:0/2285488844 shutdown_connections 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.909+0000 7fac7d482740 1 -- 172.21.15.112:0/2285488844 wait complete. 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.909+0000 7fac7d482740 1 -- start start 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.909+0000 7fac7d482740 1 Processor -- start 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac7d482740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b64090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac7d482740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b512d0 0x564e61b60b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac7d482740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b5bb90 0x564e61b612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac7d482740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac741ae260 con 0x564e61b5bb90 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac7d482740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fac741af6d0 con 0x564e61b509b0 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac7d482740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fac741ae930 con 0x564e61b512d0 2024-04-25T18:51:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac7d482740 10 client.0 _ll_register_callbacks cb 0x7fac6c0b97c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac73400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b64090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac73400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b64090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:52642/0 (socket says 172.21.15.112:52642) 2024-04-25T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac73400640 1 -- 172.21.15.112:0/1787916269 learned_addr learned my addr 172.21.15.112:0/1787916269 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac73e00640 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b5bb90 0x564e61b612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac72a00640 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b512d0 0x564e61b60b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac73400640 1 -- 172.21.15.112:0/1787916269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b512d0 msgr2=0x564e61b60b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac73400640 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b512d0 0x564e61b60b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac73400640 1 -- 172.21.15.112:0/1787916269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b5bb90 msgr2=0x564e61b612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac73400640 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b5bb90 0x564e61b612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.910+0000 7fac73400640 1 -- 172.21.15.112:0/1787916269 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e61bd0500 con 0x564e61b509b0 2024-04-25T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.911+0000 7fac73400640 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b64090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fac64031040 tx=0x7fac64009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=817f9dffbcbfbabb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.911+0000 7fac5d600640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:51:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.911+0000 7fac5d600640 1 -- 172.21.15.112:0/1787916269 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6403b060 con 0x564e61b509b0 2024-04-25T18:51:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.911+0000 7fac7d482740 1 -- 172.21.15.112:0/1787916269 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac64004570 con 0x564e61b509b0 2024-04-25T18:51:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.911+0000 7fac5d600640 1 -- 172.21.15.112:0/1787916269 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fac6402b040 con 0x564e61b509b0 2024-04-25T18:51:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.911+0000 7fac5d600640 1 -- 172.21.15.112:0/1787916269 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fac64033070 con 0x564e61b509b0 2024-04-25T18:51:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.911+0000 7fac7d482740 10 client.25610 fetch_fsmap learned FSMap version 30 2024-04-25T18:51:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.911+0000 7fac7d482740 1 -- 172.21.15.112:0/1787916269 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564e61bd1680 con 0x564e61b509b0 2024-04-25T18:51:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.912+0000 7fac5d600640 1 -- 172.21.15.112:0/1787916269 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac64004570 con 0x564e61b509b0 2024-04-25T18:51:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.912+0000 7fac5d600640 1 -- 172.21.15.112:0/1787916269 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fac6400f710 con 0x564e61b509b0 2024-04-25T18:51:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.912+0000 7fac7d482740 10 client.25610 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:51:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.912+0000 7fac7d482740 10 client.25610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:51:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.912+0000 7fac7d482740 20 client.25610 populate_metadata read hostname 'smithi112' 2024-04-25T18:51:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.912+0000 7fac7d482740 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564e61bd1c30 0x564e61bd4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.912+0000 7fac7d482740 4 client.25610 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:51:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.912+0000 7fac7d482740 1 -- 172.21.15.112:0/1787916269 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e61ac4320 con 0x564e61bd1c30 2024-04-25T18:51:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.912+0000 7fac72a00640 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564e61bd1c30 0x564e61bd4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.913+0000 7fac72a00640 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564e61bd1c30 0x564e61bd4050 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c801623e85c89f2 server_cookie=b309bf87bdd6ba27 in_seq=0 out_seq=0 2024-04-25T18:51:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.913+0000 7fac5d600640 10 client.25610 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.928+0000 7fac5d600640 1 -- 172.21.15.112:0/1787916269 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e61ac4320 con 0x564e61bd1c30 2024-04-25T18:51:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.928+0000 7fac5d600640 10 client.25610 handle_command_reply: tid=0 2024-04-25T18:51:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.928+0000 7fac7d482740 1 client.25610 shutdown 2024-04-25T18:51:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.928+0000 7fac7d482740 1 -- 172.21.15.112:0/1787916269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 msgr2=0x564e61b64090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.928+0000 7fac7d482740 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b64090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fac64031040 tx=0x7fac64009570 comp rx=0 tx=0).stop 2024-04-25T18:51:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.929+0000 7fac7d482740 1 -- 172.21.15.112:0/1787916269 shutdown_connections 2024-04-25T18:51:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.929+0000 7fac7d482740 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564e61bd1c30 0x564e61bd4050 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.929+0000 7fac7d482740 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e61b5bb90 0x564e61b612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.929+0000 7fac7d482740 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564e61b512d0 0x564e61b60b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.929+0000 7fac7d482740 1 --2- 172.21.15.112:0/1787916269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564e61b509b0 0x564e61b64090 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.929+0000 7fac5d600640 0 client.25610 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.929+0000 7fac7d482740 1 -- 172.21.15.112:0/1787916269 >> 172.21.15.112:0/1787916269 conn(0x564e61b4cec0 msgr2=0x564e61b58a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.929+0000 7fac7d482740 1 -- 172.21.15.112:0/1787916269 shutdown_connections 2024-04-25T18:51:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.929+0000 7fac7d482740 1 -- 172.21.15.112:0/1787916269 wait complete. 2024-04-25T18:51:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.929+0000 7fac7d482740 20 client.25610 trim_cache size 0 max 16384 2024-04-25T18:51:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:51:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:51:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:51:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:51:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 -- 172.21.15.112:0/3087785579 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fac580a3d10 msgr2=0x7fac580a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fac580a3d10 0x7fac580a61d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac60005f60 tx=0x7fac60005850 comp rx=0 tx=0).stop 2024-04-25T18:51:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 -- 172.21.15.112:0/3087785579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac7410c330 msgr2=0x7fac7419f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac7410c330 0x7fac7419f020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fac6c005e30 tx=0x7fac6c0308e0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 -- 172.21.15.112:0/3087785579 shutdown_connections 2024-04-25T18:51:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fac580a3d10 0x7fac580a61d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac7410c330 0x7fac7419f020 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac74103830 0x7fac7419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 --2- 172.21.15.112:0/3087785579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac74102f10 0x7fac7419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 -- 172.21.15.112:0/3087785579 >> 172.21.15.112:0/3087785579 conn(0x7fac740fc770 msgr2=0x7fac74101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 -- 172.21.15.112:0/3087785579 shutdown_connections 2024-04-25T18:51:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:20.930+0000 7fac79000640 1 -- 172.21.15.112:0/3087785579 wait complete. 2024-04-25T18:51:22.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:22 smithi171 ceph-mon[33266]: pgmap v122: 97 pgs: 97 active+clean; 43 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 556 op/s 2024-04-25T18:51:22.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:22 smithi161 ceph-mon[32609]: pgmap v122: 97 pgs: 97 active+clean; 43 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 556 op/s 2024-04-25T18:51:22.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:22 smithi112 ceph-mon[27864]: pgmap v122: 97 pgs: 97 active+clean; 43 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 556 op/s 2024-04-25T18:51:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:24 smithi171 ceph-mon[33266]: pgmap v123: 97 pgs: 97 active+clean; 43 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-04-25T18:51:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:24 smithi161 ceph-mon[32609]: pgmap v123: 97 pgs: 97 active+clean; 43 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-04-25T18:51:24.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:24 smithi112 ceph-mon[27864]: pgmap v123: 97 pgs: 97 active+clean; 43 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-04-25T18:51:26.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:26 smithi171 ceph-mon[33266]: pgmap v124: 97 pgs: 97 active+clean; 43 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-04-25T18:51:26.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:26 smithi112 ceph-mon[27864]: pgmap v124: 97 pgs: 97 active+clean; 43 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-04-25T18:51:26.486 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:26 smithi161 ceph-mon[32609]: pgmap v124: 97 pgs: 97 active+clean; 43 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 386 op/s 2024-04-25T18:51:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:28 smithi171 ceph-mon[33266]: pgmap v125: 97 pgs: 97 active+clean; 37 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 558 op/s 2024-04-25T18:51:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:28 smithi161 ceph-mon[32609]: pgmap v125: 97 pgs: 97 active+clean; 37 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 558 op/s 2024-04-25T18:51:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:28 smithi112 ceph-mon[27864]: pgmap v125: 97 pgs: 97 active+clean; 37 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 558 op/s 2024-04-25T18:51:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:30 smithi171 ceph-mon[33266]: pgmap v126: 97 pgs: 97 active+clean; 37 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-04-25T18:51:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:30 smithi161 ceph-mon[32609]: pgmap v126: 97 pgs: 97 active+clean; 37 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-04-25T18:51:30.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:30 smithi112 ceph-mon[27864]: pgmap v126: 97 pgs: 97 active+clean; 37 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-04-25T18:51:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:32 smithi171 ceph-mon[33266]: pgmap v127: 97 pgs: 97 active+clean; 39 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 498 op/s 2024-04-25T18:51:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:32 smithi161 ceph-mon[32609]: pgmap v127: 97 pgs: 97 active+clean; 39 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 498 op/s 2024-04-25T18:51:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:32 smithi112 ceph-mon[27864]: pgmap v127: 97 pgs: 97 active+clean; 39 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 498 op/s 2024-04-25T18:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.079+0000 7f778ca00640 1 -- 172.21.15.112:0/1092202118 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788107090 msgr2=0x7f778810ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.079+0000 7f778ca00640 1 --2- 172.21.15.112:0/1092202118 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788107090 0x7f778810ec60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7780009ca0 tx=0x7f778002a8b0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.080+0000 7f778ca00640 1 -- 172.21.15.112:0/1092202118 shutdown_connections 2024-04-25T18:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.080+0000 7f778ca00640 1 --2- 172.21.15.112:0/1092202118 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788107090 0x7f778810ec60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.080+0000 7f778ca00640 1 --2- 172.21.15.112:0/1092202118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7788102970 0x7f77881069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.080+0000 7f778ca00640 1 --2- 172.21.15.112:0/1092202118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7788101fc0 0x7f77881023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.080+0000 7f778ca00640 1 -- 172.21.15.112:0/1092202118 >> 172.21.15.112:0/1092202118 conn(0x7f77880fd8a0 msgr2=0x7f77880ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.080+0000 7f778ca00640 1 -- 172.21.15.112:0/1092202118 shutdown_connections 2024-04-25T18:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.080+0000 7f778ca00640 1 -- 172.21.15.112:0/1092202118 wait complete. 2024-04-25T18:51:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f778ca00640 1 Processor -- start 2024-04-25T18:51:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f778ca00640 1 -- start start 2024-04-25T18:51:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f778ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788101fc0 0x7f778819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f778ca00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7788107090 0x7f778819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f778ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778819f010 0x7f778819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f778ca00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7788112030 con 0x7f7788107090 2024-04-25T18:51:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f778ca00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7788111eb0 con 0x7f778819f010 2024-04-25T18:51:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f778ca00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f77881121b0 con 0x7f7788101fc0 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f7787400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788101fc0 0x7f778819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f7787400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788101fc0 0x7f778819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:57368/0 (socket says 172.21.15.112:57368) 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f7787400640 1 -- 172.21.15.112:0/3736311809 learned_addr learned my addr 172.21.15.112:0/3736311809 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f7787e00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778819f010 0x7f778819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.081+0000 7f7786a00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7788107090 0x7f778819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f7787400640 1 -- 172.21.15.112:0/3736311809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778819f010 msgr2=0x7f778819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f7787400640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778819f010 0x7f778819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f7787400640 1 -- 172.21.15.112:0/3736311809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7788107090 msgr2=0x7f778819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f7787400640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7788107090 0x7f778819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f7787400640 1 -- 172.21.15.112:0/3736311809 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778819fb00 con 0x7f7788101fc0 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f7786a00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7788107090 0x7f778819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f7787400640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788101fc0 0x7f778819a750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f777800e9f0 tx=0x7f777800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f777fe00640 1 -- 172.21.15.112:0/3736311809 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777800cd80 con 0x7f7788101fc0 2024-04-25T18:51:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f777fe00640 1 -- 172.21.15.112:0/3736311809 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77780108e0 con 0x7f7788101fc0 2024-04-25T18:51:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f778ca00640 1 -- 172.21.15.112:0/3736311809 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778819fdf0 con 0x7f7788101fc0 2024-04-25T18:51:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f777fe00640 1 -- 172.21.15.112:0/3736311809 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778018980 con 0x7f7788101fc0 2024-04-25T18:51:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.082+0000 7f778ca00640 1 -- 172.21.15.112:0/3736311809 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77881a7630 con 0x7f7788101fc0 2024-04-25T18:51:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.086+0000 7f777fe00640 1 -- 172.21.15.112:0/3736311809 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f7778010430 con 0x7f7788101fc0 2024-04-25T18:51:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.086+0000 7f7790e10740 1 Processor -- start 2024-04-25T18:51:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.086+0000 7f7790e10740 1 -- start start 2024-04-25T18:51:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.086+0000 7f7790e10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d82cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.086+0000 7f7790e10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82d2d0 0x56312d837650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.086+0000 7f7790e10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d837b90 0x56312d839f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7790e10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56312d792900 con 0x56312d837b90 2024-04-25T18:51:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7790e10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56312d738ef0 con 0x56312d82c9b0 2024-04-25T18:51:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7790e10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56312d78e8c0 con 0x56312d82d2d0 2024-04-25T18:51:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7787400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82d2d0 0x56312d837650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7787400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82d2d0 0x56312d837650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:57372/0 (socket says 172.21.15.112:57372) 2024-04-25T18:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7787400640 1 -- 172.21.15.112:0/539135105 learned_addr learned my addr 172.21.15.112:0/539135105 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7786a00640 1 --2- 172.21.15.112:0/539135105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d837b90 0x56312d839f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7787e00640 1 --2- 172.21.15.112:0/539135105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d82cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7787400640 1 -- 172.21.15.112:0/539135105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 msgr2=0x56312d82cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7787400640 1 --2- 172.21.15.112:0/539135105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d82cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7787400640 1 -- 172.21.15.112:0/539135105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d837b90 msgr2=0x56312d839f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7787400640 1 --2- 172.21.15.112:0/539135105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d837b90 0x56312d839f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7787400640 1 -- 172.21.15.112:0/539135105 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312d4a5210 con 0x56312d82d2d0 2024-04-25T18:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7787e00640 1 --2- 172.21.15.112:0/539135105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d82cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.087+0000 7f7786a00640 1 --2- 172.21.15.112:0/539135105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d837b90 0x56312d839f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.088+0000 7f7787400640 1 --2- 172.21.15.112:0/539135105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82d2d0 0x56312d837650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77780ce210 tx=0x7f77780d96a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad318aa96a499057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.088+0000 7f777fe00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f776c0a3d90 0x7f776c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.088+0000 7f7773e00640 1 -- 172.21.15.112:0/539135105 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77780e0020 con 0x56312d82d2d0 2024-04-25T18:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.088+0000 7f7773e00640 1 -- 172.21.15.112:0/539135105 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77780a5790 con 0x56312d82d2d0 2024-04-25T18:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.088+0000 7f7773e00640 1 -- 172.21.15.112:0/539135105 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777809b820 con 0x56312d82d2d0 2024-04-25T18:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.088+0000 7f777fe00640 1 -- 172.21.15.112:0/3736311809 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f77780d0be0 con 0x7f7788101fc0 2024-04-25T18:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.088+0000 7f7787e00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f776c0a3d90 0x7f776c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.089+0000 7f7790e10740 1 -- 172.21.15.112:0/539135105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82d2d0 msgr2=0x56312d837650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.089+0000 7f7790e10740 1 --2- 172.21.15.112:0/539135105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82d2d0 0x56312d837650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77780ce210 tx=0x7f77780d96a0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.089+0000 7f7790e10740 1 -- 172.21.15.112:0/539135105 shutdown_connections 2024-04-25T18:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.089+0000 7f7790e10740 1 --2- 172.21.15.112:0/539135105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d837b90 0x56312d839f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.089+0000 7f7790e10740 1 --2- 172.21.15.112:0/539135105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82d2d0 0x56312d837650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.089+0000 7f7790e10740 1 --2- 172.21.15.112:0/539135105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d82cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.089+0000 7f7790e10740 1 -- 172.21.15.112:0/539135105 >> 172.21.15.112:0/539135105 conn(0x56312d828ec0 msgr2=0x56312d8292a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.089+0000 7f7787e00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f776c0a3d90 0x7f776c0a6250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7780006010 tx=0x7f7780005d80 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.089+0000 7f7790e10740 1 -- 172.21.15.112:0/539135105 shutdown_connections 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.089+0000 7f7790e10740 1 -- 172.21.15.112:0/539135105 wait complete. 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.090+0000 7f7790e10740 1 -- start start 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.090+0000 7f7790e10740 1 Processor -- start 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.090+0000 7f7790e10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d834b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.090+0000 7f7790e10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d835060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.090+0000 7f7790e10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d837b90 0x56312d8355a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.090+0000 7f7790e10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56312d83a7c0 con 0x56312d82d2d0 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.090+0000 7f7790e10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56312d74c310 con 0x56312d82c9b0 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.090+0000 7f7790e10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56312d83a940 con 0x56312d837b90 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.090+0000 7f7790e10740 10 client.0 _ll_register_callbacks cb 0x56312d71acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7787400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d834b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7787e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d837b90 0x56312d8355a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7787400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d834b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:46966/0 (socket says 172.21.15.112:46966) 2024-04-25T18:51:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7787400640 1 -- 172.21.15.112:0/1054975445 learned_addr learned my addr 172.21.15.112:0/1054975445 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7786a00640 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d835060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7787400640 1 -- 172.21.15.112:0/1054975445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d837b90 msgr2=0x56312d8355a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7787400640 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d837b90 0x56312d8355a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7787400640 1 -- 172.21.15.112:0/1054975445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 msgr2=0x56312d835060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7787400640 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d835060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7787400640 1 -- 172.21.15.112:0/1054975445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312d571620 con 0x56312d82c9b0 2024-04-25T18:51:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7786a00640 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d835060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:51:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7787400640 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d834b20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f777809d670 tx=0x7f777801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ec93336db606151 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7770c00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:51:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7770c00640 1 -- 172.21.15.112:0/1054975445 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77780a1a00 con 0x56312d82c9b0 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7790e10740 1 -- 172.21.15.112:0/1054975445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77780a5790 con 0x56312d82c9b0 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.091+0000 7f7770c00640 1 -- 172.21.15.112:0/1054975445 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f7778010a90 con 0x56312d82c9b0 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.092+0000 7f7770c00640 1 -- 172.21.15.112:0/1054975445 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f77780e0020 con 0x56312d82c9b0 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.092+0000 7f7790e10740 10 client.25622 fetch_fsmap learned FSMap version 30 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.092+0000 7f7790e10740 1 -- 172.21.15.112:0/1054975445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56312d846a60 con 0x56312d82c9b0 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.092+0000 7f7770c00640 1 -- 172.21.15.112:0/1054975445 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77780a5790 con 0x56312d82c9b0 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.092+0000 7f7770c00640 1 -- 172.21.15.112:0/1054975445 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f77780bbce0 con 0x56312d82c9b0 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.093+0000 7f7790e10740 10 client.25622 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.093+0000 7f7790e10740 10 client.25622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.093+0000 7f7790e10740 20 client.25622 populate_metadata read hostname 'smithi112' 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.093+0000 7f7790e10740 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56312d847db0 0x56312d84a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.093+0000 7f7790e10740 4 client.25622 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.093+0000 7f7790e10740 1 -- 172.21.15.112:0/1054975445 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56312d7a68b0 con 0x56312d847db0 2024-04-25T18:51:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.093+0000 7f7786a00640 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56312d847db0 0x56312d84a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.093+0000 7f7786a00640 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56312d847db0 0x56312d84a1a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733cec69fc985317 server_cookie=1c0f11435230e672 in_seq=0 out_seq=0 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.093+0000 7f7770c00640 10 client.25622 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.094+0000 7f7770c00640 1 -- 172.21.15.112:0/1054975445 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56312d7a68b0 con 0x56312d847db0 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.094+0000 7f7770c00640 10 client.25622 handle_command_reply: tid=0 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.094+0000 7f7790e10740 1 client.25622 shutdown 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7790e10740 1 -- 172.21.15.112:0/1054975445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 msgr2=0x56312d834b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7790e10740 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d834b20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f777809d670 tx=0x7f777801f000 comp rx=0 tx=0).stop 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7790e10740 1 -- 172.21.15.112:0/1054975445 shutdown_connections 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7790e10740 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d837b90 0x56312d8355a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7790e10740 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56312d847db0 0x56312d84a1a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7790e10740 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d835060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7790e10740 1 --2- 172.21.15.112:0/1054975445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d834b20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7790e10740 1 -- 172.21.15.112:0/1054975445 >> 172.21.15.112:0/1054975445 conn(0x56312d828ec0 msgr2=0x56312d82ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7770c00640 0 client.25622 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7790e10740 1 -- 172.21.15.112:0/1054975445 shutdown_connections 2024-04-25T18:51:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7790e10740 1 -- 172.21.15.112:0/1054975445 wait complete. 2024-04-25T18:51:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.095+0000 7f7790e10740 20 client.25622 trim_cache size 0 max 16384 2024-04-25T18:51:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.113+0000 7f7790e10740 1 Processor -- start 2024-04-25T18:51:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.113+0000 7f7790e10740 1 -- start start 2024-04-25T18:51:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.113+0000 7f7790e10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d843680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.113+0000 7f7790e10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d84f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.113+0000 7f7790e10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d837b90 0x56312d84fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.113+0000 7f7790e10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77881b2620 con 0x56312d82d2d0 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.113+0000 7f7790e10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f77881afe90 con 0x56312d82c9b0 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.113+0000 7f7790e10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f77881adc80 con 0x56312d837b90 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7787400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d843680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7787e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d837b90 0x56312d84fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7787400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d843680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:46980/0 (socket says 172.21.15.112:46980) 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7787e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d837b90 0x56312d84fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:57386/0 (socket says 172.21.15.112:57386) 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7787400640 1 -- 172.21.15.112:0/486674900 learned_addr learned my addr 172.21.15.112:0/486674900 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7786a00640 1 --2- 172.21.15.112:0/486674900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d84f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7787400640 1 -- 172.21.15.112:0/486674900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d837b90 msgr2=0x56312d84fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7787400640 1 --2- 172.21.15.112:0/486674900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d837b90 0x56312d84fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7787400640 1 -- 172.21.15.112:0/486674900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 msgr2=0x56312d84f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7787400640 1 --2- 172.21.15.112:0/486674900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d84f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7787400640 1 -- 172.21.15.112:0/486674900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312d82dc40 con 0x56312d82c9b0 2024-04-25T18:51:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7786a00640 1 --2- 172.21.15.112:0/486674900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d84f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:51:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7787400640 1 --2- 172.21.15.112:0/486674900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d843680 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f777800c6a0 tx=0x7f7778018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef35ab979c794b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.114+0000 7f7773400640 1 -- 172.21.15.112:0/486674900 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77780d0e60 con 0x56312d82c9b0 2024-04-25T18:51:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.115+0000 7f7773400640 1 -- 172.21.15.112:0/486674900 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77780a2200 con 0x56312d82c9b0 2024-04-25T18:51:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.115+0000 7f7773400640 1 -- 172.21.15.112:0/486674900 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77780a1c20 con 0x56312d82c9b0 2024-04-25T18:51:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.115+0000 7f7790e10740 1 -- 172.21.15.112:0/486674900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 msgr2=0x56312d843680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.115+0000 7f7790e10740 1 --2- 172.21.15.112:0/486674900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d843680 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f777800c6a0 tx=0x7f7778018420 comp rx=0 tx=0).stop 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.115+0000 7f7790e10740 1 -- 172.21.15.112:0/486674900 shutdown_connections 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.115+0000 7f7790e10740 1 --2- 172.21.15.112:0/486674900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d837b90 0x56312d84fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.115+0000 7f7790e10740 1 --2- 172.21.15.112:0/486674900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d84f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.115+0000 7f7790e10740 1 --2- 172.21.15.112:0/486674900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d82c9b0 0x56312d843680 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.115+0000 7f7790e10740 1 -- 172.21.15.112:0/486674900 >> 172.21.15.112:0/486674900 conn(0x56312d828ec0 msgr2=0x56312d82ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.116+0000 7f7790e10740 1 -- 172.21.15.112:0/486674900 shutdown_connections 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.116+0000 7f7790e10740 1 -- 172.21.15.112:0/486674900 wait complete. 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.116+0000 7f7790e10740 1 -- start start 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.116+0000 7f7790e10740 1 Processor -- start 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7790e10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82c9b0 0x56312d83d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7790e10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d83de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7790e10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d837b90 0x56312d8acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7790e10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77881ae350 con 0x56312d82d2d0 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7790e10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f77881af7c0 con 0x56312d837b90 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7790e10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f77881aea20 con 0x56312d82c9b0 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7790e10740 10 client.0 _ll_register_callbacks cb 0x7f77780d4460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7787400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82c9b0 0x56312d83d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7786a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d83de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7787e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d837b90 0x56312d8acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7787400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82c9b0 0x56312d83d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:57392/0 (socket says 172.21.15.112:57392) 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.117+0000 7f7787400640 1 -- 172.21.15.112:0/134609333 learned_addr learned my addr 172.21.15.112:0/134609333 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7787e00640 1 -- 172.21.15.112:0/134609333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82c9b0 msgr2=0x56312d83d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7787e00640 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82c9b0 0x56312d83d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7787e00640 1 -- 172.21.15.112:0/134609333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 msgr2=0x56312d83de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7787e00640 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d83de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7787e00640 1 -- 172.21.15.112:0/134609333 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312d83e700 con 0x56312d837b90 2024-04-25T18:51:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7787400640 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82c9b0 0x56312d83d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7787e00640 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d837b90 0x56312d8acff0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7780005e30 tx=0x7f778002a8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc6720b6db907c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7770c00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7770c00640 1 -- 172.21.15.112:0/134609333 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780059060 con 0x56312d837b90 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7770c00640 1 -- 172.21.15.112:0/134609333 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f7780049420 con 0x56312d837b90 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7790e10740 1 -- 172.21.15.112:0/134609333 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77780a2200 con 0x56312d837b90 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.118+0000 7f7770c00640 1 -- 172.21.15.112:0/134609333 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7780049620 con 0x56312d837b90 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.119+0000 7f7790e10740 10 client.25628 fetch_fsmap learned FSMap version 30 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.119+0000 7f7790e10740 1 -- 172.21.15.112:0/134609333 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56312d8ae610 con 0x56312d837b90 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.119+0000 7f7770c00640 1 -- 172.21.15.112:0/134609333 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77780a2200 con 0x56312d837b90 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.119+0000 7f7770c00640 1 -- 172.21.15.112:0/134609333 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7780053030 con 0x56312d837b90 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.119+0000 7f7790e10740 10 client.25628 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.119+0000 7f7790e10740 10 client.25628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:51:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.119+0000 7f7790e10740 20 client.25628 populate_metadata read hostname 'smithi112' 2024-04-25T18:51:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.119+0000 7f7790e10740 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56312d8aedc0 0x56312d8b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.119+0000 7f7790e10740 4 client.25628 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:51:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.119+0000 7f7790e10740 1 -- 172.21.15.112:0/134609333 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56312d7a0320 con 0x56312d8aedc0 2024-04-25T18:51:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.119+0000 7f7787400640 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56312d8aedc0 0x56312d8b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.120+0000 7f7787400640 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56312d8aedc0 0x56312d8b11e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589f9495c72b16dd server_cookie=152ee3051db985d3 in_seq=0 out_seq=0 2024-04-25T18:51:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.120+0000 7f7770c00640 10 client.25628 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7770c00640 1 -- 172.21.15.112:0/134609333 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56312d7a0320 con 0x56312d8aedc0 2024-04-25T18:51:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7770c00640 10 client.25628 handle_command_reply: tid=0 2024-04-25T18:51:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 1 client.25628 shutdown 2024-04-25T18:51:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 1 -- 172.21.15.112:0/134609333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d837b90 msgr2=0x56312d8acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d837b90 0x56312d8acff0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7780005e30 tx=0x7f778002a8b0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 1 -- 172.21.15.112:0/134609333 shutdown_connections 2024-04-25T18:51:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56312d837b90 0x56312d8acff0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56312d8aedc0 0x56312d8b11e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56312d82d2d0 0x56312d83de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 1 --2- 172.21.15.112:0/134609333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56312d82c9b0 0x56312d83d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7770c00640 0 client.25628 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 1 -- 172.21.15.112:0/134609333 >> 172.21.15.112:0/134609333 conn(0x56312d828ec0 msgr2=0x56312d836350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 1 -- 172.21.15.112:0/134609333 shutdown_connections 2024-04-25T18:51:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 1 -- 172.21.15.112:0/134609333 wait complete. 2024-04-25T18:51:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.136+0000 7f7790e10740 20 client.25628 trim_cache size 0 max 16384 2024-04-25T18:51:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:51:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:51:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:51:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:51:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.137+0000 7f778ca00640 1 -- 172.21.15.112:0/3736311809 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f776c0a3d90 msgr2=0x7f776c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.137+0000 7f778ca00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f776c0a3d90 0x7f776c0a6250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7780006010 tx=0x7f7780005d80 comp rx=0 tx=0).stop 2024-04-25T18:51:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.137+0000 7f778ca00640 1 -- 172.21.15.112:0/3736311809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788101fc0 msgr2=0x7f778819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.137+0000 7f778ca00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788101fc0 0x7f778819a750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f777800e9f0 tx=0x7f777800eeb0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.137+0000 7f778ca00640 1 -- 172.21.15.112:0/3736311809 shutdown_connections 2024-04-25T18:51:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.137+0000 7f778ca00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f776c0a3d90 0x7f776c0a6250 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.137+0000 7f778ca00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778819f010 0x7f778819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.137+0000 7f778ca00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7788107090 0x7f778819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.137+0000 7f778ca00640 1 --2- 172.21.15.112:0/3736311809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788101fc0 0x7f778819a750 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.137+0000 7f778ca00640 1 -- 172.21.15.112:0/3736311809 >> 172.21.15.112:0/3736311809 conn(0x7f77880fd8a0 msgr2=0x7f77880ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.138+0000 7f778ca00640 1 -- 172.21.15.112:0/3736311809 shutdown_connections 2024-04-25T18:51:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:34.138+0000 7f778ca00640 1 -- 172.21.15.112:0/3736311809 wait complete. 2024-04-25T18:51:34.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:34 smithi171 ceph-mon[33266]: pgmap v128: 97 pgs: 97 active+clean; 39 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-04-25T18:51:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:34 smithi161 ceph-mon[32609]: pgmap v128: 97 pgs: 97 active+clean; 39 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-04-25T18:51:34.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:34 smithi112 ceph-mon[27864]: pgmap v128: 97 pgs: 97 active+clean; 39 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-04-25T18:51:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:36 smithi171 ceph-mon[33266]: pgmap v129: 97 pgs: 97 active+clean; 39 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-04-25T18:51:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:36 smithi161 ceph-mon[32609]: pgmap v129: 97 pgs: 97 active+clean; 39 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-04-25T18:51:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:36 smithi112 ceph-mon[27864]: pgmap v129: 97 pgs: 97 active+clean; 39 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-04-25T18:51:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:38 smithi171 ceph-mon[33266]: pgmap v130: 97 pgs: 97 active+clean; 37 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 448 op/s 2024-04-25T18:51:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:38 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:38 smithi161 ceph-mon[32609]: pgmap v130: 97 pgs: 97 active+clean; 37 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 448 op/s 2024-04-25T18:51:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:38 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:38 smithi112 ceph-mon[27864]: pgmap v130: 97 pgs: 97 active+clean; 37 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 448 op/s 2024-04-25T18:51:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:38 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:39.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:39 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:39.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:39 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:39.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:39 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:39.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:39 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:39.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:39 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:39.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:39 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:40.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:40 smithi171 ceph-mon[33266]: pgmap v131: 97 pgs: 97 active+clean; 37 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-04-25T18:51:40.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:40 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:51:40.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:40 smithi161 ceph-mon[32609]: pgmap v131: 97 pgs: 97 active+clean; 37 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-04-25T18:51:40.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:40 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:51:40.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:40 smithi112 ceph-mon[27864]: pgmap v131: 97 pgs: 97 active+clean; 37 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 276 op/s 2024-04-25T18:51:40.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:40 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:51:42.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:42 smithi171 ceph-mon[33266]: pgmap v132: 97 pgs: 97 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 415 op/s 2024-04-25T18:51:42.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:42 smithi161 ceph-mon[32609]: pgmap v132: 97 pgs: 97 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 415 op/s 2024-04-25T18:51:42.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:42 smithi112 ceph-mon[27864]: pgmap v132: 97 pgs: 97 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 415 op/s 2024-04-25T18:51:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:44 smithi171 ceph-mon[33266]: pgmap v133: 97 pgs: 97 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2024-04-25T18:51:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:44 smithi161 ceph-mon[32609]: pgmap v133: 97 pgs: 97 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2024-04-25T18:51:44.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:44 smithi112 ceph-mon[27864]: pgmap v133: 97 pgs: 97 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2024-04-25T18:51:44.803 INFO:tasks.workunit.client.0.smithi112.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-04-25T18:51:45.554 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xffcdd 2024-04-25T18:51:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:46 smithi171 ceph-mon[33266]: pgmap v134: 97 pgs: 97 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2024-04-25T18:51:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:46 smithi161 ceph-mon[32609]: pgmap v134: 97 pgs: 97 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2024-04-25T18:51:46.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:46 smithi112 ceph-mon[27864]: pgmap v134: 97 pgs: 97 active+clean; 39 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2024-04-25T18:51:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.297+0000 7fb8f5200640 1 -- 172.21.15.112:0/1016517340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f0104bb0 msgr2=0x7fb8f0108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.297+0000 7fb8f5200640 1 --2- 172.21.15.112:0/1016517340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f0104bb0 0x7fb8f0108bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009860 tx=0x7fb8d802a340 comp rx=0 tx=0).stop 2024-04-25T18:51:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.297+0000 7fb8f5200640 1 -- 172.21.15.112:0/1016517340 shutdown_connections 2024-04-25T18:51:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.297+0000 7fb8f5200640 1 --2- 172.21.15.112:0/1016517340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f01092d0 0x7fb8f0110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.297+0000 7fb8f5200640 1 --2- 172.21.15.112:0/1016517340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f0104bb0 0x7fb8f0108bf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.297+0000 7fb8f5200640 1 --2- 172.21.15.112:0/1016517340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb8f0104200 0x7fb8f01045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.297+0000 7fb8f5200640 1 -- 172.21.15.112:0/1016517340 >> 172.21.15.112:0/1016517340 conn(0x7fb8f00ffa60 msgr2=0x7fb8f0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.297+0000 7fb8f5200640 1 -- 172.21.15.112:0/1016517340 shutdown_connections 2024-04-25T18:51:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.297+0000 7fb8f5200640 1 -- 172.21.15.112:0/1016517340 wait complete. 2024-04-25T18:51:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.298+0000 7fb8f5200640 1 Processor -- start 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.298+0000 7fb8f5200640 1 -- start start 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.299+0000 7fb8f5200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f0104200 0x7fb8f01a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.299+0000 7fb8f5200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f01092d0 0x7fb8f01a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.299+0000 7fb8f5200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb8f019a5d0 0x7fb8f019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.299+0000 7fb8f5200640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01141f0 con 0x7fb8f01092d0 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.299+0000 7fb8f5200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f0114070 con 0x7fb8f0104200 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.299+0000 7fb8f5200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f0114370 con 0x7fb8f019a5d0 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.299+0000 7fb8ef400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f0104200 0x7fb8f01a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.299+0000 7fb8ef400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f0104200 0x7fb8f01a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:55590/0 (socket says 172.21.15.112:55590) 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.299+0000 7fb8ef400640 1 -- 172.21.15.112:0/1972036138 learned_addr learned my addr 172.21.15.112:0/1972036138 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.299+0000 7fb8e7e00640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f01092d0 0x7fb8f01a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.299+0000 7fb8efe00640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb8f019a5d0 0x7fb8f019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.300+0000 7fb8ef400640 1 -- 172.21.15.112:0/1972036138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb8f019a5d0 msgr2=0x7fb8f019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.300+0000 7fb8ef400640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb8f019a5d0 0x7fb8f019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.300+0000 7fb8ef400640 1 -- 172.21.15.112:0/1972036138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f01092d0 msgr2=0x7fb8f01a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.300+0000 7fb8ef400640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f01092d0 0x7fb8f01a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.300+0000 7fb8ef400640 1 -- 172.21.15.112:0/1972036138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f019b0f0 con 0x7fb8f0104200 2024-04-25T18:51:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.300+0000 7fb8ef400640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f0104200 0x7fb8f01a0420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000e9c0 tx=0x7fb8e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.300+0000 7fb8ecc00640 1 -- 172.21.15.112:0/1972036138 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e000cde0 con 0x7fb8f0104200 2024-04-25T18:51:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.300+0000 7fb8ecc00640 1 -- 172.21.15.112:0/1972036138 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8e00035f0 con 0x7fb8f0104200 2024-04-25T18:51:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.300+0000 7fb8f5200640 1 -- 172.21.15.112:0/1972036138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f019b3e0 con 0x7fb8f0104200 2024-04-25T18:51:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.300+0000 7fb8ecc00640 1 -- 172.21.15.112:0/1972036138 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0010690 con 0x7fb8f0104200 2024-04-25T18:51:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.300+0000 7fb8f5200640 1 -- 172.21.15.112:0/1972036138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f01a7630 con 0x7fb8f0104200 2024-04-25T18:51:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.303+0000 7fb8ecc00640 1 -- 172.21.15.112:0/1972036138 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fb8e0004000 con 0x7fb8f0104200 2024-04-25T18:51:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.304+0000 7fb8f9792740 1 Processor -- start 2024-04-25T18:51:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.304+0000 7fb8f9792740 1 -- start start 2024-04-25T18:51:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.304+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee8e9b0 0x557b3ee8ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.304+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee99650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.304+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee99b90 0x557b3ee9bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.304+0000 7fb8f9792740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557b3edf4900 con 0x557b3ee99b90 2024-04-25T18:51:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.304+0000 7fb8f9792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557b3ed9aef0 con 0x557b3ee8e9b0 2024-04-25T18:51:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.304+0000 7fb8f9792740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557b3edf08c0 con 0x557b3ee8f2d0 2024-04-25T18:51:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8ef400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee99650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8ef400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee99650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:38520/0 (socket says 172.21.15.112:38520) 2024-04-25T18:51:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8ef400640 1 -- 172.21.15.112:0/2503423290 learned_addr learned my addr 172.21.15.112:0/2503423290 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8efe00640 1 --2- 172.21.15.112:0/2503423290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee8e9b0 0x557b3ee8ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8e7e00640 1 --2- 172.21.15.112:0/2503423290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee99b90 0x557b3ee9bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8ef400640 1 -- 172.21.15.112:0/2503423290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee8e9b0 msgr2=0x557b3ee8ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8ef400640 1 --2- 172.21.15.112:0/2503423290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee8e9b0 0x557b3ee8ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8ef400640 1 -- 172.21.15.112:0/2503423290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee99b90 msgr2=0x557b3ee9bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8ef400640 1 --2- 172.21.15.112:0/2503423290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee99b90 0x557b3ee9bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8ef400640 1 -- 172.21.15.112:0/2503423290 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3eb07210 con 0x557b3ee8f2d0 2024-04-25T18:51:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8e7e00640 1 --2- 172.21.15.112:0/2503423290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee99b90 0x557b3ee9bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:51:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.305+0000 7fb8ef400640 1 --2- 172.21.15.112:0/2503423290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee99650 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00ce190 tx=0x7fb8e00d9250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e3c33402ba14339 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.306+0000 7fb8ecc00640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb8d40a3d10 0x7fb8d40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.306+0000 7fb8ecc00640 1 -- 172.21.15.112:0/1972036138 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fb8e00d0b60 con 0x7fb8f0104200 2024-04-25T18:51:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.306+0000 7fb8e4c00640 1 -- 172.21.15.112:0/2503423290 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00042f0 con 0x557b3ee8f2d0 2024-04-25T18:51:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.306+0000 7fb8efe00640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb8d40a3d10 0x7fb8d40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.306+0000 7fb8e4c00640 1 -- 172.21.15.112:0/2503423290 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8e00e2070 con 0x557b3ee8f2d0 2024-04-25T18:51:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.306+0000 7fb8e4c00640 1 -- 172.21.15.112:0/2503423290 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e009c400 con 0x557b3ee8f2d0 2024-04-25T18:51:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.307+0000 7fb8f9792740 1 -- 172.21.15.112:0/2503423290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 msgr2=0x557b3ee99650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.307+0000 7fb8efe00640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb8d40a3d10 0x7fb8d40a61d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8005fd0 tx=0x7fb8e8009500 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.307+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2503423290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee99650 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00ce190 tx=0x7fb8e00d9250 comp rx=0 tx=0).stop 2024-04-25T18:51:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.307+0000 7fb8f9792740 1 -- 172.21.15.112:0/2503423290 shutdown_connections 2024-04-25T18:51:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.307+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2503423290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee99b90 0x557b3ee9bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.307+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2503423290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee99650 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.307+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2503423290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee8e9b0 0x557b3ee8ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.307+0000 7fb8f9792740 1 -- 172.21.15.112:0/2503423290 >> 172.21.15.112:0/2503423290 conn(0x557b3ee8aec0 msgr2=0x557b3ee8b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.307+0000 7fb8f9792740 1 -- 172.21.15.112:0/2503423290 shutdown_connections 2024-04-25T18:51:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.307+0000 7fb8f9792740 1 -- 172.21.15.112:0/2503423290 wait complete. 2024-04-25T18:51:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.308+0000 7fb8f9792740 1 -- start start 2024-04-25T18:51:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.308+0000 7fb8f9792740 1 Processor -- start 2024-04-25T18:51:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee8e9b0 0x557b3ee96b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee970a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 0x557b3ee975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8f9792740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557b3ee9c760 con 0x557b3ee8e9b0 2024-04-25T18:51:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8f9792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557b3edae310 con 0x557b3ee99b90 2024-04-25T18:51:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8f9792740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557b3ee9c8e0 con 0x557b3ee8f2d0 2024-04-25T18:51:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8f9792740 10 client.0 _ll_register_callbacks cb 0x557b3ed7ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:51:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8e7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8ef400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee8e9b0 0x557b3ee96b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8e7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:38524/0 (socket says 172.21.15.112:38524) 2024-04-25T18:51:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8e7e00640 1 -- 172.21.15.112:0/2929563132 learned_addr learned my addr 172.21.15.112:0/2929563132 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8efe00640 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 0x557b3ee975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8e7e00640 1 -- 172.21.15.112:0/2929563132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 msgr2=0x557b3ee975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8e7e00640 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 0x557b3ee975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8e7e00640 1 -- 172.21.15.112:0/2929563132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee8e9b0 msgr2=0x557b3ee96b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8e7e00640 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee8e9b0 0x557b3ee96b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8e7e00640 1 -- 172.21.15.112:0/2929563132 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3ebd3620 con 0x557b3ee8f2d0 2024-04-25T18:51:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8efe00640 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 0x557b3ee975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:51:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.309+0000 7fb8ef400640 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee8e9b0 0x557b3ee96b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.310+0000 7fb8e7e00640 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee970a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb8d802a930 tx=0x7fb8d8030fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6faca0d09e32e890 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.310+0000 7fb8dd600640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.310+0000 7fb8dd600640 1 -- 172.21.15.112:0/2929563132 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8041070 con 0x557b3ee8f2d0 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.310+0000 7fb8f9792740 1 -- 172.21.15.112:0/2929563132 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e00e2070 con 0x557b3ee8f2d0 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.310+0000 7fb8dd600640 1 -- 172.21.15.112:0/2929563132 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fb8d8030450 con 0x557b3ee8f2d0 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.310+0000 7fb8dd600640 1 -- 172.21.15.112:0/2929563132 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb8d804d050 con 0x557b3ee8f2d0 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.310+0000 7fb8f9792740 10 client.25657 fetch_fsmap learned FSMap version 30 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.310+0000 7fb8f9792740 1 -- 172.21.15.112:0/2929563132 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557b3eea8a10 con 0x557b3ee8f2d0 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.311+0000 7fb8dd600640 1 -- 172.21.15.112:0/2929563132 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8e00e2070 con 0x557b3ee8f2d0 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.311+0000 7fb8dd600640 1 -- 172.21.15.112:0/2929563132 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb8d8032440 con 0x557b3ee8f2d0 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.311+0000 7fb8f9792740 10 client.25657 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:51:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.311+0000 7fb8f9792740 10 client.25657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.311+0000 7fb8f9792740 20 client.25657 populate_metadata read hostname 'smithi112' 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.311+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557b3eea9ed0 0x557b3eeac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.311+0000 7fb8f9792740 4 client.25657 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.311+0000 7fb8f9792740 1 -- 172.21.15.112:0/2929563132 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b3ee088b0 con 0x557b3eea9ed0 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.311+0000 7fb8ef400640 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557b3eea9ed0 0x557b3eeac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.312+0000 7fb8ef400640 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557b3eea9ed0 0x557b3eeac340 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc5309cfee0a1b15 server_cookie=de36318a7707a5f5 in_seq=0 out_seq=0 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.312+0000 7fb8dd600640 10 client.25657 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.312+0000 7fb8dd600640 1 -- 172.21.15.112:0/2929563132 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b3ee088b0 con 0x557b3eea9ed0 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.312+0000 7fb8dd600640 10 client.25657 handle_command_reply: tid=0 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.312+0000 7fb8f9792740 1 client.25657 shutdown 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8f9792740 1 -- 172.21.15.112:0/2929563132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 msgr2=0x557b3ee970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:47.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee970a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb8d802a930 tx=0x7fb8d8030fc0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8f9792740 1 -- 172.21.15.112:0/2929563132 shutdown_connections 2024-04-25T18:51:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 0x557b3ee975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee970a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557b3eea9ed0 0x557b3eeac340 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2929563132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee8e9b0 0x557b3ee96b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8f9792740 1 -- 172.21.15.112:0/2929563132 >> 172.21.15.112:0/2929563132 conn(0x557b3ee8aec0 msgr2=0x557b3ee90bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8dd600640 0 client.25657 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8f9792740 1 -- 172.21.15.112:0/2929563132 shutdown_connections 2024-04-25T18:51:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8f9792740 1 -- 172.21.15.112:0/2929563132 wait complete. 2024-04-25T18:51:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.313+0000 7fb8f9792740 20 client.25657 trim_cache size 0 max 16384 2024-04-25T18:51:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.330+0000 7fb8f9792740 1 Processor -- start 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8f9792740 1 -- start start 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee8e9b0 0x557b3eea5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee91370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 0x557b3eeb1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8f9792740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01b6a50 con 0x557b3ee8e9b0 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8f9792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01b42c0 con 0x557b3ee99b90 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8f9792740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01b2110 con 0x557b3ee8f2d0 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8efe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 0x557b3eeb1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8efe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 0x557b3eeb1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:55614/0 (socket says 172.21.15.112:55614) 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8efe00640 1 -- 172.21.15.112:0/4201151040 learned_addr learned my addr 172.21.15.112:0/4201151040 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8ef400640 1 --2- 172.21.15.112:0/4201151040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee8e9b0 0x557b3eea5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.331+0000 7fb8e7e00640 1 --2- 172.21.15.112:0/4201151040 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee91370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.332+0000 7fb8efe00640 1 -- 172.21.15.112:0/4201151040 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 msgr2=0x557b3ee91370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.332+0000 7fb8efe00640 1 --2- 172.21.15.112:0/4201151040 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee91370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.332+0000 7fb8efe00640 1 -- 172.21.15.112:0/4201151040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee8e9b0 msgr2=0x557b3eea5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.332+0000 7fb8efe00640 1 --2- 172.21.15.112:0/4201151040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee8e9b0 0x557b3eea5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.332+0000 7fb8efe00640 1 -- 172.21.15.112:0/4201151040 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3ee86820 con 0x557b3ee99b90 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.332+0000 7fb8e7e00640 1 --2- 172.21.15.112:0/4201151040 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee91370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.332+0000 7fb8efe00640 1 --2- 172.21.15.112:0/4201151040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 0x557b3eeb1c10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8003e50 tx=0x7fb8e8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f14375bb42e3141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.332+0000 7fb8dfe00640 1 -- 172.21.15.112:0/4201151040 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e801e460 con 0x557b3ee99b90 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.333+0000 7fb8dfe00640 1 -- 172.21.15.112:0/4201151040 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8e801eab0 con 0x557b3ee99b90 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.333+0000 7fb8dfe00640 1 -- 172.21.15.112:0/4201151040 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e801f990 con 0x557b3ee99b90 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.333+0000 7fb8f9792740 1 -- 172.21.15.112:0/4201151040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 msgr2=0x557b3eeb1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.333+0000 7fb8f9792740 1 --2- 172.21.15.112:0/4201151040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 0x557b3eeb1c10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8003e50 tx=0x7fb8e8017000 comp rx=0 tx=0).stop 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.333+0000 7fb8f9792740 1 -- 172.21.15.112:0/4201151040 shutdown_connections 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.333+0000 7fb8f9792740 1 --2- 172.21.15.112:0/4201151040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee99b90 0x557b3eeb1c10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.333+0000 7fb8f9792740 1 --2- 172.21.15.112:0/4201151040 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8f2d0 0x557b3ee91370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.333+0000 7fb8f9792740 1 --2- 172.21.15.112:0/4201151040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee8e9b0 0x557b3eea5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.333+0000 7fb8f9792740 1 -- 172.21.15.112:0/4201151040 >> 172.21.15.112:0/4201151040 conn(0x557b3ee8aec0 msgr2=0x557b3ee91eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.334+0000 7fb8f9792740 1 -- 172.21.15.112:0/4201151040 shutdown_connections 2024-04-25T18:51:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.334+0000 7fb8f9792740 1 -- 172.21.15.112:0/4201151040 wait complete. 2024-04-25T18:51:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.334+0000 7fb8f9792740 1 -- start start 2024-04-25T18:51:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.334+0000 7fb8f9792740 1 Processor -- start 2024-04-25T18:51:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8e9b0 0x557b3eea1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee8f2d0 0x557b3ee9f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8f9792740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee99b90 0x557b3ee9f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8f9792740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01b2780 con 0x557b3ee99b90 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8f9792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01b3bf0 con 0x557b3ee8f2d0 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8f9792740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01b2e50 con 0x557b3ee8e9b0 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8f9792740 10 client.0 _ll_register_callbacks cb 0x7fb8d80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8ef400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8e9b0 0x557b3eea1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8ef400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8e9b0 0x557b3eea1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:38554/0 (socket says 172.21.15.112:38554) 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8e7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee8f2d0 0x557b3ee9f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8ef400640 1 -- 172.21.15.112:0/2462686235 learned_addr learned my addr 172.21.15.112:0/2462686235 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8efe00640 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee99b90 0x557b3ee9f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8e7e00640 1 -- 172.21.15.112:0/2462686235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8e9b0 msgr2=0x557b3eea1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8e7e00640 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8e9b0 0x557b3eea1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8e7e00640 1 -- 172.21.15.112:0/2462686235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee99b90 msgr2=0x557b3ee9f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8e7e00640 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee99b90 0x557b3ee9f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8e7e00640 1 -- 172.21.15.112:0/2462686235 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3ee9ffe0 con 0x557b3ee8f2d0 2024-04-25T18:51:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.335+0000 7fb8ef400640 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8e9b0 0x557b3eea1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.336+0000 7fb8e7e00640 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee8f2d0 0x557b3ee9f2a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80057c0 tx=0x7fb8d8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1d9fb36149296f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.336+0000 7fb8dd600640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.336+0000 7fb8dd600640 1 -- 172.21.15.112:0/2462686235 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8056060 con 0x557b3ee8f2d0 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.336+0000 7fb8f9792740 1 -- 172.21.15.112:0/2462686235 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e801eab0 con 0x557b3ee8f2d0 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.336+0000 7fb8dd600640 1 -- 172.21.15.112:0/2462686235 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fb8d8004450 con 0x557b3ee8f2d0 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.336+0000 7fb8dd600640 1 -- 172.21.15.112:0/2462686235 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb8d8041040 con 0x557b3ee8f2d0 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.336+0000 7fb8f9792740 10 client.25646 fetch_fsmap learned FSMap version 30 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.336+0000 7fb8f9792740 1 -- 172.21.15.112:0/2462686235 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557b3ef104d0 con 0x557b3ee8f2d0 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.337+0000 7fb8dd600640 1 -- 172.21.15.112:0/2462686235 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8e801eab0 con 0x557b3ee8f2d0 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.337+0000 7fb8dd600640 1 -- 172.21.15.112:0/2462686235 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb8d80052a0 con 0x557b3ee8f2d0 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.337+0000 7fb8f9792740 10 client.25646 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.337+0000 7fb8f9792740 10 client.25646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.337+0000 7fb8f9792740 20 client.25646 populate_metadata read hostname 'smithi112' 2024-04-25T18:51:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.337+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557b3ef10ce0 0x557b3ef13100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.337+0000 7fb8f9792740 4 client.25646 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:51:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.337+0000 7fb8f9792740 1 -- 172.21.15.112:0/2462686235 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b3ee02320 con 0x557b3ef10ce0 2024-04-25T18:51:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.337+0000 7fb8ef400640 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557b3ef10ce0 0x557b3ef13100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.338+0000 7fb8ef400640 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557b3ef10ce0 0x557b3ef13100 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5c1ef94c7ac4e4b server_cookie=e5cd58ff01e9f491 in_seq=0 out_seq=0 2024-04-25T18:51:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.338+0000 7fb8dd600640 10 client.25646 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8dd600640 1 -- 172.21.15.112:0/2462686235 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b3ee02320 con 0x557b3ef10ce0 2024-04-25T18:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8dd600640 10 client.25646 handle_command_reply: tid=0 2024-04-25T18:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 1 client.25646 shutdown 2024-04-25T18:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 1 -- 172.21.15.112:0/2462686235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee8f2d0 msgr2=0x557b3ee9f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee8f2d0 0x557b3ee9f2a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80057c0 tx=0x7fb8d8000bc0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 1 -- 172.21.15.112:0/2462686235 shutdown_connections 2024-04-25T18:51:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557b3ef10ce0 0x557b3ef13100 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b3ee99b90 0x557b3ee9f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557b3ee8f2d0 0x557b3ee9f2a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 1 --2- 172.21.15.112:0/2462686235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b3ee8e9b0 0x557b3eea1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8dd600640 0 client.25646 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:51:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 1 -- 172.21.15.112:0/2462686235 >> 172.21.15.112:0/2462686235 conn(0x557b3ee8aec0 msgr2=0x557b3ee96b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 1 -- 172.21.15.112:0/2462686235 shutdown_connections 2024-04-25T18:51:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 1 -- 172.21.15.112:0/2462686235 wait complete. 2024-04-25T18:51:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.354+0000 7fb8f9792740 20 client.25646 trim_cache size 0 max 16384 2024-04-25T18:51:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:51:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:51:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:51:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:51:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.355+0000 7fb8f5200640 1 -- 172.21.15.112:0/1972036138 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb8d40a3d10 msgr2=0x7fb8d40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.355+0000 7fb8f5200640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb8d40a3d10 0x7fb8d40a61d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8005fd0 tx=0x7fb8e8009500 comp rx=0 tx=0).stop 2024-04-25T18:51:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.355+0000 7fb8f5200640 1 -- 172.21.15.112:0/1972036138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f0104200 msgr2=0x7fb8f01a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.355+0000 7fb8f5200640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f0104200 0x7fb8f01a0420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000e9c0 tx=0x7fb8e000ee80 comp rx=0 tx=0).stop 2024-04-25T18:51:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.355+0000 7fb8f5200640 1 -- 172.21.15.112:0/1972036138 shutdown_connections 2024-04-25T18:51:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.355+0000 7fb8f5200640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb8d40a3d10 0x7fb8d40a61d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.355+0000 7fb8f5200640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb8f019a5d0 0x7fb8f019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.355+0000 7fb8f5200640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f01092d0 0x7fb8f01a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.355+0000 7fb8f5200640 1 --2- 172.21.15.112:0/1972036138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f0104200 0x7fb8f01a0420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.355+0000 7fb8f5200640 1 -- 172.21.15.112:0/1972036138 >> 172.21.15.112:0/1972036138 conn(0x7fb8f00ffa60 msgr2=0x7fb8f0100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.355+0000 7fb8f5200640 1 -- 172.21.15.112:0/1972036138 shutdown_connections 2024-04-25T18:51:47.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:51:47.356+0000 7fb8f5200640 1 -- 172.21.15.112:0/1972036138 wait complete. 2024-04-25T18:51:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:48 smithi171 ceph-mon[33266]: pgmap v135: 97 pgs: 97 active+clean; 41 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-04-25T18:51:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:48 smithi161 ceph-mon[32609]: pgmap v135: 97 pgs: 97 active+clean; 41 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-04-25T18:51:48.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:48 smithi112 ceph-mon[27864]: pgmap v135: 97 pgs: 97 active+clean; 41 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-04-25T18:51:48.840 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xffd97 2024-04-25T18:51:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:50 smithi171 ceph-mon[33266]: pgmap v136: 97 pgs: 97 active+clean; 41 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-04-25T18:51:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:50 smithi161 ceph-mon[32609]: pgmap v136: 97 pgs: 97 active+clean; 41 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-04-25T18:51:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:50 smithi112 ceph-mon[27864]: pgmap v136: 97 pgs: 97 active+clean; 41 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-04-25T18:51:52.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:52 smithi171 ceph-mon[33266]: pgmap v137: 97 pgs: 97 active+clean; 38 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-04-25T18:51:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:52 smithi161 ceph-mon[32609]: pgmap v137: 97 pgs: 97 active+clean; 38 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-04-25T18:51:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:52 smithi112 ceph-mon[27864]: pgmap v137: 97 pgs: 97 active+clean; 38 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-04-25T18:51:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:54 smithi171 ceph-mon[33266]: pgmap v138: 97 pgs: 97 active+clean; 38 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2024-04-25T18:51:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:54 smithi161 ceph-mon[32609]: pgmap v138: 97 pgs: 97 active+clean; 38 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2024-04-25T18:51:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:54 smithi112 ceph-mon[27864]: pgmap v138: 97 pgs: 97 active+clean; 38 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2024-04-25T18:51:56.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:56 smithi171 ceph-mon[33266]: pgmap v139: 97 pgs: 97 active+clean; 38 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2024-04-25T18:51:56.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:56 smithi161 ceph-mon[32609]: pgmap v139: 97 pgs: 97 active+clean; 38 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2024-04-25T18:51:56.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:56 smithi112 ceph-mon[27864]: pgmap v139: 97 pgs: 97 active+clean; 38 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2024-04-25T18:51:58.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:51:58 smithi171 ceph-mon[33266]: pgmap v140: 97 pgs: 97 active+clean; 41 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 384 op/s 2024-04-25T18:51:58.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:51:58 smithi161 ceph-mon[32609]: pgmap v140: 97 pgs: 97 active+clean; 41 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 384 op/s 2024-04-25T18:51:58.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:51:58 smithi112 ceph-mon[27864]: pgmap v140: 97 pgs: 97 active+clean; 41 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 384 op/s 2024-04-25T18:52:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:00 smithi171 ceph-mon[33266]: pgmap v141: 97 pgs: 97 active+clean; 41 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2024-04-25T18:52:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:00 smithi161 ceph-mon[32609]: pgmap v141: 97 pgs: 97 active+clean; 41 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2024-04-25T18:52:00.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:00 smithi112 ceph-mon[27864]: pgmap v141: 97 pgs: 97 active+clean; 41 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2024-04-25T18:52:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.503+0000 7f6d54c00640 1 -- 172.21.15.112:0/883798894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d500742f0 msgr2=0x7f6d500746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.503+0000 7f6d54c00640 1 --2- 172.21.15.112:0/883798894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d500742f0 0x7f6d500746d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009900 tx=0x7f6d4002a480 comp rx=0 tx=0).stop 2024-04-25T18:52:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.503+0000 7f6d54c00640 1 -- 172.21.15.112:0/883798894 shutdown_connections 2024-04-25T18:52:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.503+0000 7f6d54c00640 1 --2- 172.21.15.112:0/883798894 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d50112d60 0x7f6d50115150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.503+0000 7f6d54c00640 1 --2- 172.21.15.112:0/883798894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d50074c10 0x7f6d50112820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.503+0000 7f6d54c00640 1 --2- 172.21.15.112:0/883798894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d500742f0 0x7f6d500746d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.503+0000 7f6d54c00640 1 -- 172.21.15.112:0/883798894 >> 172.21.15.112:0/883798894 conn(0x7f6d500fd8a0 msgr2=0x7f6d500ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.504+0000 7f6d54c00640 1 -- 172.21.15.112:0/883798894 shutdown_connections 2024-04-25T18:52:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.504+0000 7f6d54c00640 1 -- 172.21.15.112:0/883798894 wait complete. 2024-04-25T18:52:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.505+0000 7f6d54c00640 1 Processor -- start 2024-04-25T18:52:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.505+0000 7f6d54c00640 1 -- start start 2024-04-25T18:52:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d54c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d500742f0 0x7f6d501a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d54c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d50074c10 0x7f6d501a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d54c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d50112d60 0x7f6d5019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d54c00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d50118600 con 0x7f6d500742f0 2024-04-25T18:52:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d54c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d50118480 con 0x7f6d50112d60 2024-04-25T18:52:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d54c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6d50118780 con 0x7f6d50074c10 2024-04-25T18:52:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d4ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d50074c10 0x7f6d501a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d500742f0 0x7f6d501a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d4ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d50074c10 0x7f6d501a4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:54264/0 (socket says 172.21.15.112:54264) 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d500742f0 0x7f6d501a4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60510/0 (socket says 172.21.15.112:60510) 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d4ea00640 1 -- 172.21.15.112:0/3124459682 learned_addr learned my addr 172.21.15.112:0/3124459682 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.506+0000 7f6d4fe00640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d50112d60 0x7f6d5019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.507+0000 7f6d4ea00640 1 -- 172.21.15.112:0/3124459682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d50112d60 msgr2=0x7f6d5019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.507+0000 7f6d4ea00640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d50112d60 0x7f6d5019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.507+0000 7f6d4ea00640 1 -- 172.21.15.112:0/3124459682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d500742f0 msgr2=0x7f6d501a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.507+0000 7f6d4ea00640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d500742f0 0x7f6d501a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.507+0000 7f6d4ea00640 1 -- 172.21.15.112:0/3124459682 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5019f180 con 0x7f6d50074c10 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.507+0000 7f6d4ea00640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d50074c10 0x7f6d501a4d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00d950 tx=0x7f6d3c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.507+0000 7f6d47e00640 1 -- 172.21.15.112:0/3124459682 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c017030 con 0x7f6d50074c10 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.507+0000 7f6d47e00640 1 -- 172.21.15.112:0/3124459682 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d3c004020 con 0x7f6d50074c10 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.507+0000 7f6d54c00640 1 -- 172.21.15.112:0/3124459682 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5019f470 con 0x7f6d50074c10 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.507+0000 7f6d47e00640 1 -- 172.21.15.112:0/3124459682 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0047a0 con 0x7f6d50074c10 2024-04-25T18:52:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.507+0000 7f6d54c00640 1 -- 172.21.15.112:0/3124459682 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d501ab710 con 0x7f6d50074c10 2024-04-25T18:52:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.511+0000 7f6d47e00640 1 -- 172.21.15.112:0/3124459682 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f6d3c009a30 con 0x7f6d50074c10 2024-04-25T18:52:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.511+0000 7f6d591ed740 1 Processor -- start 2024-04-25T18:52:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.511+0000 7f6d591ed740 1 -- start start 2024-04-25T18:52:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.511+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d049e9b0 0x5591d049ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.511+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049f2d0 0x5591d04a9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.511+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d04a9b90 0x5591d04abf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.511+0000 7f6d591ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591d0404900 con 0x5591d049f2d0 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.511+0000 7f6d591ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5591d03aaef0 con 0x5591d049e9b0 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.511+0000 7f6d591ed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5591d04008c0 con 0x5591d04a9b90 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.511+0000 7f6d4fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d049e9b0 0x5591d049ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.512+0000 7f6d4fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d049e9b0 0x5591d049ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:45838/0 (socket says 172.21.15.112:45838) 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.512+0000 7f6d4fe00640 1 -- 172.21.15.112:0/363345970 learned_addr learned my addr 172.21.15.112:0/363345970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.512+0000 7f6d4f400640 1 --2- 172.21.15.112:0/363345970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049f2d0 0x5591d04a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.512+0000 7f6d4ea00640 1 --2- 172.21.15.112:0/363345970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d04a9b90 0x5591d04abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.512+0000 7f6d4fe00640 1 -- 172.21.15.112:0/363345970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d04a9b90 msgr2=0x5591d04abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.512+0000 7f6d4fe00640 1 --2- 172.21.15.112:0/363345970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d04a9b90 0x5591d04abf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.512+0000 7f6d4fe00640 1 -- 172.21.15.112:0/363345970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049f2d0 msgr2=0x5591d04a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.512+0000 7f6d4fe00640 1 --2- 172.21.15.112:0/363345970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049f2d0 0x5591d04a9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.512+0000 7f6d4fe00640 1 -- 172.21.15.112:0/363345970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591d0117210 con 0x5591d049e9b0 2024-04-25T18:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.512+0000 7f6d4fe00640 1 --2- 172.21.15.112:0/363345970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d049e9b0 0x5591d049ed90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009f30 tx=0x7f6d480118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9740ccb5e41cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.512+0000 7f6d3be00640 1 -- 172.21.15.112:0/363345970 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48004d20 con 0x5591d049e9b0 2024-04-25T18:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.513+0000 7f6d3be00640 1 -- 172.21.15.112:0/363345970 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d48004050 con 0x5591d049e9b0 2024-04-25T18:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.513+0000 7f6d3be00640 1 -- 172.21.15.112:0/363345970 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48005260 con 0x5591d049e9b0 2024-04-25T18:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.513+0000 7f6d47e00640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d340a3d10 0x7f6d340a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.513+0000 7f6d47e00640 1 -- 172.21.15.112:0/3124459682 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f6d3c0d0230 con 0x7f6d50074c10 2024-04-25T18:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.513+0000 7f6d4f400640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d340a3d10 0x7f6d340a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.513+0000 7f6d591ed740 1 -- 172.21.15.112:0/363345970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d049e9b0 msgr2=0x5591d049ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.513+0000 7f6d591ed740 1 --2- 172.21.15.112:0/363345970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d049e9b0 0x5591d049ed90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009f30 tx=0x7f6d480118d0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.514+0000 7f6d591ed740 1 -- 172.21.15.112:0/363345970 shutdown_connections 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.514+0000 7f6d591ed740 1 --2- 172.21.15.112:0/363345970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d04a9b90 0x5591d04abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.514+0000 7f6d591ed740 1 --2- 172.21.15.112:0/363345970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049f2d0 0x5591d04a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.514+0000 7f6d591ed740 1 --2- 172.21.15.112:0/363345970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d049e9b0 0x5591d049ed90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.514+0000 7f6d591ed740 1 -- 172.21.15.112:0/363345970 >> 172.21.15.112:0/363345970 conn(0x5591d049aec0 msgr2=0x5591d049b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.514+0000 7f6d4f400640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d340a3d10 0x7f6d340a61d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d40031400 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.514+0000 7f6d591ed740 1 -- 172.21.15.112:0/363345970 shutdown_connections 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.514+0000 7f6d591ed740 1 -- 172.21.15.112:0/363345970 wait complete. 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.514+0000 7f6d591ed740 1 -- start start 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d591ed740 1 Processor -- start 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049e9b0 0x5591d04a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04a7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d591ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5591d04ac7c0 con 0x5591d049e9b0 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d591ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5591d03be310 con 0x5591d04a9b90 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d591ed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5591d04ac940 con 0x5591d049f2d0 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d591ed740 10 client.0 _ll_register_callbacks cb 0x5591d038ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d4ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049e9b0 0x5591d04a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d4fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d4ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:54280/0 (socket says 172.21.15.112:54280) 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.515+0000 7f6d4ea00640 1 -- 172.21.15.112:0/2766664696 learned_addr learned my addr 172.21.15.112:0/2766664696 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.516+0000 7f6d4ea00640 1 -- 172.21.15.112:0/2766664696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 msgr2=0x5591d04a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.516+0000 7f6d4ea00640 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.516+0000 7f6d4ea00640 1 -- 172.21.15.112:0/2766664696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049e9b0 msgr2=0x5591d04a6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.516+0000 7f6d4ea00640 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049e9b0 0x5591d04a6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.516+0000 7f6d4ea00640 1 -- 172.21.15.112:0/2766664696 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591d01e3620 con 0x5591d049f2d0 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.516+0000 7f6d4fe00640 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.516+0000 7f6d4ea00640 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04a7080 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0cd980 tx=0x7f6d3c09bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cefa836705ab1592 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.516+0000 7f6d38c00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.516+0000 7f6d38c00640 1 -- 172.21.15.112:0/2766664696 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c09c170 con 0x5591d049f2d0 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.516+0000 7f6d38c00640 1 -- 172.21.15.112:0/2766664696 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f6d3c0987e0 con 0x5591d049f2d0 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.516+0000 7f6d591ed740 1 -- 172.21.15.112:0/2766664696 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d48004050 con 0x5591d049f2d0 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d38c00640 1 -- 172.21.15.112:0/2766664696 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d3c0a0520 con 0x5591d049f2d0 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d591ed740 10 client.25675 fetch_fsmap learned FSMap version 30 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d591ed740 1 -- 172.21.15.112:0/2766664696 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591d04b8aa0 con 0x5591d049f2d0 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d38c00640 1 -- 172.21.15.112:0/2766664696 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d48004050 con 0x5591d049f2d0 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d38c00640 1 -- 172.21.15.112:0/2766664696 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6d3c096ce0 con 0x5591d049f2d0 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d591ed740 10 client.25675 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d591ed740 10 client.25675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d591ed740 20 client.25675 populate_metadata read hostname 'smithi112' 2024-04-25T18:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5591d04ba290 0x5591d04bc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d591ed740 4 client.25675 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d591ed740 1 -- 172.21.15.112:0/2766664696 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591d04188b0 con 0x5591d04ba290 2024-04-25T18:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.517+0000 7f6d4f400640 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5591d04ba290 0x5591d04bc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.518+0000 7f6d4f400640 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5591d04ba290 0x5591d04bc6f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c3820fde11be205 server_cookie=c2babf75d9f7d0b2 in_seq=0 out_seq=0 2024-04-25T18:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.518+0000 7f6d38c00640 10 client.25675 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d38c00640 1 -- 172.21.15.112:0/2766664696 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591d04188b0 con 0x5591d04ba290 2024-04-25T18:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d38c00640 10 client.25675 handle_command_reply: tid=0 2024-04-25T18:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d591ed740 1 client.25675 shutdown 2024-04-25T18:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d591ed740 1 -- 172.21.15.112:0/2766664696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 msgr2=0x5591d04a7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04a7080 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0cd980 tx=0x7f6d3c09bd70 comp rx=0 tx=0).stop 2024-04-25T18:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d591ed740 1 -- 172.21.15.112:0/2766664696 shutdown_connections 2024-04-25T18:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04a7080 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5591d04ba290 0x5591d04bc6f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2766664696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049e9b0 0x5591d04a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d38c00640 0 client.25675 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.519+0000 7f6d591ed740 1 -- 172.21.15.112:0/2766664696 >> 172.21.15.112:0/2766664696 conn(0x5591d049aec0 msgr2=0x5591d04a0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.520+0000 7f6d591ed740 1 -- 172.21.15.112:0/2766664696 shutdown_connections 2024-04-25T18:52:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.520+0000 7f6d591ed740 1 -- 172.21.15.112:0/2766664696 wait complete. 2024-04-25T18:52:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.520+0000 7f6d591ed740 20 client.25675 trim_cache size 0 max 16384 2024-04-25T18:52:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.538+0000 7f6d591ed740 1 Processor -- start 2024-04-25T18:52:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.538+0000 7f6d591ed740 1 -- start start 2024-04-25T18:52:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049e9b0 0x5591d04b5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049f2d0 0x5591d04c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d591ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501b6950 con 0x5591d049f2d0 2024-04-25T18:52:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d591ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501b41c0 con 0x5591d04a9b90 2024-04-25T18:52:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d591ed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501b2010 con 0x5591d049e9b0 2024-04-25T18:52:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d4fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049e9b0 0x5591d04b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d4fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04c2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:45862/0 (socket says 172.21.15.112:45862) 2024-04-25T18:52:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d4fe00640 1 -- 172.21.15.112:0/4011852735 learned_addr learned my addr 172.21.15.112:0/4011852735 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d4ea00640 1 --2- 172.21.15.112:0/4011852735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049f2d0 0x5591d04c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d4fe00640 1 -- 172.21.15.112:0/4011852735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049e9b0 msgr2=0x5591d04b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d4fe00640 1 --2- 172.21.15.112:0/4011852735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049e9b0 0x5591d04b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d4fe00640 1 -- 172.21.15.112:0/4011852735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049f2d0 msgr2=0x5591d04c1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d4fe00640 1 --2- 172.21.15.112:0/4011852735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049f2d0 0x5591d04c1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.539+0000 7f6d4fe00640 1 -- 172.21.15.112:0/4011852735 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591d04abf80 con 0x5591d04a9b90 2024-04-25T18:52:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.540+0000 7f6d4f400640 1 --2- 172.21.15.112:0/4011852735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049e9b0 0x5591d04b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:52:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.540+0000 7f6d4fe00640 1 --2- 172.21.15.112:0/4011852735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04c2500 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6d48011dd0 tx=0x7f6d48009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f4777bf3cf00e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.540+0000 7f6d3b400640 1 -- 172.21.15.112:0/4011852735 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48003c30 con 0x5591d04a9b90 2024-04-25T18:52:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.540+0000 7f6d3b400640 1 -- 172.21.15.112:0/4011852735 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d48003dd0 con 0x5591d04a9b90 2024-04-25T18:52:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.540+0000 7f6d3b400640 1 -- 172.21.15.112:0/4011852735 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d480054e0 con 0x5591d04a9b90 2024-04-25T18:52:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.541+0000 7f6d591ed740 1 -- 172.21.15.112:0/4011852735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 msgr2=0x5591d04c2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.541+0000 7f6d591ed740 1 --2- 172.21.15.112:0/4011852735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04c2500 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6d48011dd0 tx=0x7f6d48009b00 comp rx=0 tx=0).stop 2024-04-25T18:52:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.541+0000 7f6d591ed740 1 -- 172.21.15.112:0/4011852735 shutdown_connections 2024-04-25T18:52:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.541+0000 7f6d591ed740 1 --2- 172.21.15.112:0/4011852735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04c2500 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.541+0000 7f6d591ed740 1 --2- 172.21.15.112:0/4011852735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049f2d0 0x5591d04c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.541+0000 7f6d591ed740 1 --2- 172.21.15.112:0/4011852735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049e9b0 0x5591d04b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.541+0000 7f6d591ed740 1 -- 172.21.15.112:0/4011852735 >> 172.21.15.112:0/4011852735 conn(0x5591d049aec0 msgr2=0x5591d04a1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.541+0000 7f6d591ed740 1 -- 172.21.15.112:0/4011852735 shutdown_connections 2024-04-25T18:52:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.541+0000 7f6d591ed740 1 -- 172.21.15.112:0/4011852735 wait complete. 2024-04-25T18:52:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.542+0000 7f6d591ed740 1 -- start start 2024-04-25T18:52:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.542+0000 7f6d591ed740 1 Processor -- start 2024-04-25T18:52:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.542+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049e9b0 0x5591d04b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.542+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d591ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04af310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d591ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501b2680 con 0x5591d049e9b0 2024-04-25T18:52:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d591ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501b3af0 con 0x5591d04a9b90 2024-04-25T18:52:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d591ed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6d501b2d50 con 0x5591d049f2d0 2024-04-25T18:52:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d591ed740 10 client.0 _ll_register_callbacks cb 0x7f6d3c09bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:52:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d4ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d4fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049e9b0 0x5591d04b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d4ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:54302/0 (socket says 172.21.15.112:54302) 2024-04-25T18:52:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d4ea00640 1 -- 172.21.15.112:0/2495127525 learned_addr learned my addr 172.21.15.112:0/2495127525 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d4ea00640 1 -- 172.21.15.112:0/2495127525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 msgr2=0x5591d04af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d4ea00640 1 --2- 172.21.15.112:0/2495127525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d4ea00640 1 -- 172.21.15.112:0/2495127525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049e9b0 msgr2=0x5591d04b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d4ea00640 1 --2- 172.21.15.112:0/2495127525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049e9b0 0x5591d04b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d4ea00640 1 -- 172.21.15.112:0/2495127525 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591d04afac0 con 0x5591d049f2d0 2024-04-25T18:52:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d4ea00640 1 --2- 172.21.15.112:0/2495127525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04b2090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0a48b0 tx=0x7f6d3c09bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c4135f4e9aef26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.543+0000 7f6d38c00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:52:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d38c00640 1 -- 172.21.15.112:0/2495127525 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0cdda0 con 0x5591d049f2d0 2024-04-25T18:52:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d591ed740 1 -- 172.21.15.112:0/2495127525 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d48003dd0 con 0x5591d049f2d0 2024-04-25T18:52:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d38c00640 1 -- 172.21.15.112:0/2495127525 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f6d3c0a0020 con 0x5591d049f2d0 2024-04-25T18:52:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d38c00640 1 -- 172.21.15.112:0/2495127525 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d3c09c170 con 0x5591d049f2d0 2024-04-25T18:52:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d591ed740 10 client.25681 fetch_fsmap learned FSMap version 30 2024-04-25T18:52:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d591ed740 1 -- 172.21.15.112:0/2495127525 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591d051f840 con 0x5591d049f2d0 2024-04-25T18:52:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d38c00640 1 -- 172.21.15.112:0/2495127525 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d48003dd0 con 0x5591d049f2d0 2024-04-25T18:52:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d38c00640 1 -- 172.21.15.112:0/2495127525 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6d3c09b030 con 0x5591d049f2d0 2024-04-25T18:52:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d591ed740 10 client.25681 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T18:52:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d591ed740 10 client.25681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:52:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d591ed740 20 client.25681 populate_metadata read hostname 'smithi112' 2024-04-25T18:52:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2495127525 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5591d051ffb0 0x5591d05223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d591ed740 4 client.25681 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:52:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.544+0000 7f6d591ed740 1 -- 172.21.15.112:0/2495127525 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591d0412320 con 0x5591d051ffb0 2024-04-25T18:52:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.545+0000 7f6d4f400640 1 --2- 172.21.15.112:0/2495127525 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5591d051ffb0 0x5591d05223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.545+0000 7f6d4f400640 1 --2- 172.21.15.112:0/2495127525 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5591d051ffb0 0x5591d05223d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547e2f3dfea7bf30 server_cookie=f9e72611854fc808 in_seq=0 out_seq=0 2024-04-25T18:52:00.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.545+0000 7f6d38c00640 10 client.25681 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d38c00640 1 -- 172.21.15.112:0/2495127525 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591d0412320 con 0x5591d051ffb0 2024-04-25T18:52:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d38c00640 10 client.25681 handle_command_reply: tid=0 2024-04-25T18:52:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d591ed740 1 client.25681 shutdown 2024-04-25T18:52:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d591ed740 1 -- 172.21.15.112:0/2495127525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 msgr2=0x5591d04b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2495127525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04b2090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0a48b0 tx=0x7f6d3c09bc00 comp rx=0 tx=0).stop 2024-04-25T18:52:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d591ed740 1 -- 172.21.15.112:0/2495127525 shutdown_connections 2024-04-25T18:52:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2495127525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591d04a9b90 0x5591d04af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2495127525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5591d049f2d0 0x5591d04b2090 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2495127525 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5591d051ffb0 0x5591d05223d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d591ed740 1 --2- 172.21.15.112:0/2495127525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591d049e9b0 0x5591d04b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d38c00640 0 client.25681 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.558+0000 7f6d591ed740 1 -- 172.21.15.112:0/2495127525 >> 172.21.15.112:0/2495127525 conn(0x5591d049aec0 msgr2=0x5591d04a6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.559+0000 7f6d591ed740 1 -- 172.21.15.112:0/2495127525 shutdown_connections 2024-04-25T18:52:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.559+0000 7f6d591ed740 1 -- 172.21.15.112:0/2495127525 wait complete. 2024-04-25T18:52:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.559+0000 7f6d591ed740 20 client.25681 trim_cache size 0 max 16384 2024-04-25T18:52:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:52:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:52:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:52:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:52:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 -- 172.21.15.112:0/3124459682 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d340a3d10 msgr2=0x7f6d340a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d340a3d10 0x7f6d340a61d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d40031400 comp rx=0 tx=0).stop 2024-04-25T18:52:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 -- 172.21.15.112:0/3124459682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d50074c10 msgr2=0x7f6d501a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d50074c10 0x7f6d501a4d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00d950 tx=0x7f6d3c00de10 comp rx=0 tx=0).stop 2024-04-25T18:52:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 -- 172.21.15.112:0/3124459682 shutdown_connections 2024-04-25T18:52:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6d340a3d10 0x7f6d340a61d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d50112d60 0x7f6d5019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d50074c10 0x7f6d501a4d90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 --2- 172.21.15.112:0/3124459682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6d500742f0 0x7f6d501a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 -- 172.21.15.112:0/3124459682 >> 172.21.15.112:0/3124459682 conn(0x7f6d500fd8a0 msgr2=0x7f6d500ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 -- 172.21.15.112:0/3124459682 shutdown_connections 2024-04-25T18:52:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:00.560+0000 7f6d54c00640 1 -- 172.21.15.112:0/3124459682 wait complete. 2024-04-25T18:52:01.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:01 smithi171 ceph-mon[33266]: daemon mds.g restarted 2024-04-25T18:52:01.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:01 smithi161 ceph-mon[32609]: daemon mds.g restarted 2024-04-25T18:52:01.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:01 smithi112 ceph-mon[27864]: daemon mds.g restarted 2024-04-25T18:52:01.474 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:52:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: ignoring --setuser ceph since I am not root 2024-04-25T18:52:01.474 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:52:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: ignoring --setgroup ceph since I am not root 2024-04-25T18:52:01.474 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:52:01 smithi112 ceph-mds[86791]: -- 172.21.15.112:0/3145588640 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a913fdba40 con 0x55a913faec00 2024-04-25T18:52:01.474 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:52:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: starting mds.g at 2024-04-25T18:52:02.435 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:02 smithi171 ceph-mon[33266]: pgmap v142: 97 pgs: 97 active+clean; 34 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 388 op/s 2024-04-25T18:52:02.435 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:02 smithi171 ceph-mon[33266]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T18:52:02.435 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:02 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:52:02.435 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:02 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.112:6838/519179678,v1:172.21.15.112:6839/519179678] up:boot 2024-04-25T18:52:02.435 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:02 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T18:52:02.435 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:02 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:52:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:02 smithi161 ceph-mon[32609]: pgmap v142: 97 pgs: 97 active+clean; 34 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 388 op/s 2024-04-25T18:52:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:02 smithi161 ceph-mon[32609]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T18:52:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:02 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:52:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:02 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.112:6838/519179678,v1:172.21.15.112:6839/519179678] up:boot 2024-04-25T18:52:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:02 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T18:52:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:02 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:52:02.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:02 smithi112 ceph-mon[27864]: pgmap v142: 97 pgs: 97 active+clean; 34 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 388 op/s 2024-04-25T18:52:02.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:02 smithi112 ceph-mon[27864]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T18:52:02.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:02 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:52:02.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:02 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.112:6838/519179678,v1:172.21.15.112:6839/519179678] up:boot 2024-04-25T18:52:02.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:02 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T18:52:02.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:02 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:52:03.805 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0xffdb8 2024-04-25T18:52:04.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:04 smithi171 ceph-mon[33266]: pgmap v144: 97 pgs: 97 active+clean; 34 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-04-25T18:52:04.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:04 smithi161 ceph-mon[32609]: pgmap v144: 97 pgs: 97 active+clean; 34 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-04-25T18:52:04.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:04 smithi112 ceph-mon[27864]: pgmap v144: 97 pgs: 97 active+clean; 34 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-04-25T18:52:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:06 smithi171 ceph-mon[33266]: pgmap v145: 97 pgs: 97 active+clean; 34 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-04-25T18:52:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:06 smithi161 ceph-mon[32609]: pgmap v145: 97 pgs: 97 active+clean; 34 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-04-25T18:52:06.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:06 smithi112 ceph-mon[27864]: pgmap v145: 97 pgs: 97 active+clean; 34 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-04-25T18:52:08.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:08 smithi171 ceph-mon[33266]: pgmap v146: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 293 op/s 2024-04-25T18:52:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:08 smithi161 ceph-mon[32609]: pgmap v146: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 293 op/s 2024-04-25T18:52:08.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:08 smithi112 ceph-mon[27864]: pgmap v146: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 293 op/s 2024-04-25T18:52:10.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:10 smithi171 ceph-mon[33266]: pgmap v147: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 293 op/s 2024-04-25T18:52:10.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:10 smithi161 ceph-mon[32609]: pgmap v147: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 293 op/s 2024-04-25T18:52:10.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:10 smithi112 ceph-mon[27864]: pgmap v147: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 293 op/s 2024-04-25T18:52:12.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:12 smithi171 ceph-mon[33266]: pgmap v148: 97 pgs: 97 active+clean; 40 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-04-25T18:52:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:12 smithi161 ceph-mon[32609]: pgmap v148: 97 pgs: 97 active+clean; 40 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-04-25T18:52:12.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:12 smithi112 ceph-mon[27864]: pgmap v148: 97 pgs: 97 active+clean; 40 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-04-25T18:52:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.714+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2854576621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6dc105f90 msgr2=0x7fd6dc10db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.714+0000 7fd6e0c00640 1 --2- 172.21.15.112:0/2854576621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6dc105f90 0x7fd6dc10db60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009ca0 tx=0x7fd6d402a8c0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.714+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2854576621 shutdown_connections 2024-04-25T18:52:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.714+0000 7fd6e0c00640 1 --2- 172.21.15.112:0/2854576621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6dc105f90 0x7fd6dc10db60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.714+0000 7fd6e0c00640 1 --2- 172.21.15.112:0/2854576621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6dc101870 0x7fd6dc1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.714+0000 7fd6e0c00640 1 --2- 172.21.15.112:0/2854576621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6dc100ec0 0x7fd6dc1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.714+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2854576621 >> 172.21.15.112:0/2854576621 conn(0x7fd6dc0fc760 msgr2=0x7fd6dc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.714+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2854576621 shutdown_connections 2024-04-25T18:52:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.714+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2854576621 wait complete. 2024-04-25T18:52:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.715+0000 7fd6e0c00640 1 Processor -- start 2024-04-25T18:52:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.715+0000 7fd6e0c00640 1 -- start start 2024-04-25T18:52:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.715+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6dc100ec0 0x7fd6dc102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.715+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6dc105f90 0x7fd6dc102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.715+0000 7fd6e0c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6dc1033d0 0x7fd6dc1a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.715+0000 7fd6e0c00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc110f40 con 0x7fd6dc100ec0 2024-04-25T18:52:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.715+0000 7fd6e0c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc110dc0 con 0x7fd6dc105f90 2024-04-25T18:52:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.715+0000 7fd6e0c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc1110c0 con 0x7fd6dc1033d0 2024-04-25T18:52:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6db400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6dc100ec0 0x7fd6dc102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6dc105f90 0x7fd6dc102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6dc105f90 0x7fd6dc102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:38740/0 (socket says 172.21.15.112:38740) 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6daa00640 1 -- 172.21.15.112:0/2920634110 learned_addr learned my addr 172.21.15.112:0/2920634110 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6db400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6dc100ec0 0x7fd6dc102950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41198/0 (socket says 172.21.15.112:41198) 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6dbe00640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6dc1033d0 0x7fd6dc1a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6daa00640 1 -- 172.21.15.112:0/2920634110 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6dc1033d0 msgr2=0x7fd6dc1a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6daa00640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6dc1033d0 0x7fd6dc1a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6daa00640 1 -- 172.21.15.112:0/2920634110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6dc100ec0 msgr2=0x7fd6dc102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6daa00640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6dc100ec0 0x7fd6dc102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6daa00640 1 -- 172.21.15.112:0/2920634110 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc1a6df0 con 0x7fd6dc105f90 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.716+0000 7fd6daa00640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6dc105f90 0x7fd6dc102e90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800e9f0 tx=0x7fd6c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.717+0000 7fd6d3e00640 1 -- 172.21.15.112:0/2920634110 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c800ccc0 con 0x7fd6dc105f90 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.717+0000 7fd6d3e00640 1 -- 172.21.15.112:0/2920634110 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c80054d0 con 0x7fd6dc105f90 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.717+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2920634110 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc1a70e0 con 0x7fd6dc105f90 2024-04-25T18:52:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.717+0000 7fd6d3e00640 1 -- 172.21.15.112:0/2920634110 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8010430 con 0x7fd6dc105f90 2024-04-25T18:52:13.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.717+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2920634110 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc1a7630 con 0x7fd6dc105f90 2024-04-25T18:52:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.720+0000 7fd6e51c5740 1 Processor -- start 2024-04-25T18:52:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6e51c5740 1 -- start start 2024-04-25T18:52:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ab5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ac0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ac0b90 0x556cb1ac2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556cb1a1b900 con 0x556cb1ab62d0 2024-04-25T18:52:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556cb19c1ef0 con 0x556cb1ac0b90 2024-04-25T18:52:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556cb1a178c0 con 0x556cb1ab59b0 2024-04-25T18:52:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6d3e00640 1 -- 172.21.15.112:0/2920634110 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fd6c80046c0 con 0x7fd6dc105f90 2024-04-25T18:52:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ab5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ab5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:46622/0 (socket says 172.21.15.112:46622) 2024-04-25T18:52:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6dbe00640 1 -- 172.21.15.112:0/2904942401 learned_addr learned my addr 172.21.15.112:0/2904942401 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.721+0000 7fd6db400640 1 --2- 172.21.15.112:0/2904942401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ac0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.722+0000 7fd6daa00640 1 --2- 172.21.15.112:0/2904942401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ac0b90 0x556cb1ac2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.722+0000 7fd6dbe00640 1 -- 172.21.15.112:0/2904942401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ac0b90 msgr2=0x556cb1ac2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.722+0000 7fd6dbe00640 1 --2- 172.21.15.112:0/2904942401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ac0b90 0x556cb1ac2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.722+0000 7fd6dbe00640 1 -- 172.21.15.112:0/2904942401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 msgr2=0x556cb1ac0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.722+0000 7fd6dbe00640 1 --2- 172.21.15.112:0/2904942401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ac0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.722+0000 7fd6dbe00640 1 -- 172.21.15.112:0/2904942401 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb172e210 con 0x556cb1ab59b0 2024-04-25T18:52:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.722+0000 7fd6daa00640 1 --2- 172.21.15.112:0/2904942401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ac0b90 0x556cb1ac2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:52:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.722+0000 7fd6dbe00640 1 --2- 172.21.15.112:0/2904942401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ab5d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009ca0 tx=0x7fd6d40093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf44e52d8b6f192 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.723+0000 7fd6c7e00640 1 -- 172.21.15.112:0/2904942401 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4031cf0 con 0x556cb1ab59b0 2024-04-25T18:52:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.723+0000 7fd6c7e00640 1 -- 172.21.15.112:0/2904942401 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d4036030 con 0x556cb1ab59b0 2024-04-25T18:52:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.723+0000 7fd6c7e00640 1 -- 172.21.15.112:0/2904942401 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40042a0 con 0x556cb1ab59b0 2024-04-25T18:52:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.723+0000 7fd6d3e00640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd6c00a3dc0 0x7fd6c00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6d3e00640 1 -- 172.21.15.112:0/2920634110 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7fd6c80d0a30 con 0x7fd6dc105f90 2024-04-25T18:52:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6db400640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd6c00a3dc0 0x7fd6c00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2904942401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 msgr2=0x556cb1ab5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/2904942401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ab5d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009ca0 tx=0x7fd6d40093e0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2904942401 shutdown_connections 2024-04-25T18:52:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/2904942401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ac0b90 0x556cb1ac2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/2904942401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ac0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/2904942401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ab5d90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2904942401 >> 172.21.15.112:0/2904942401 conn(0x556cb1ab1ec0 msgr2=0x556cb1ab22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2904942401 shutdown_connections 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2904942401 wait complete. 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6db400640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd6c00a3dc0 0x7fd6c00a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x556cb1ab6f50 tx=0x7fd6cc005f90 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.724+0000 7fd6e51c5740 1 -- start start 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6e51c5740 1 Processor -- start 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 0x556cb1abdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab62d0 0x556cb1abe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ac0b90 0x556cb1abe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556cb1ac35e0 con 0x556cb1ac0b90 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556cb19d5310 con 0x556cb1ab59b0 2024-04-25T18:52:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556cb1ac3760 con 0x556cb1ab62d0 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6db400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 0x556cb1abdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6e51c5740 10 client.0 _ll_register_callbacks cb 0x556cb19a3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab62d0 0x556cb1abe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6db400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 0x556cb1abdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:38754/0 (socket says 172.21.15.112:38754) 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6db400640 1 -- 172.21.15.112:0/3385252768 learned_addr learned my addr 172.21.15.112:0/3385252768 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.725+0000 7fd6dbe00640 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ac0b90 0x556cb1abe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.726+0000 7fd6db400640 1 -- 172.21.15.112:0/3385252768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab62d0 msgr2=0x556cb1abe060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.726+0000 7fd6db400640 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab62d0 0x556cb1abe060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.726+0000 7fd6db400640 1 -- 172.21.15.112:0/3385252768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ac0b90 msgr2=0x556cb1abe5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.726+0000 7fd6db400640 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ac0b90 0x556cb1abe5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.726+0000 7fd6db400640 1 -- 172.21.15.112:0/3385252768 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb17fa620 con 0x556cb1ab59b0 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.726+0000 7fd6daa00640 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab62d0 0x556cb1abe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.726+0000 7fd6db400640 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 0x556cb1abdb20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc004630 tx=0x7fd6cc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6361965d2a28bc11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.726+0000 7fd6c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:52:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.726+0000 7fd6c4c00640 1 -- 172.21.15.112:0/3385252768 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc009e50 con 0x556cb1ab59b0 2024-04-25T18:52:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.727+0000 7fd6c4c00640 1 -- 172.21.15.112:0/3385252768 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7fd6cc01f630 con 0x556cb1ab59b0 2024-04-25T18:52:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.727+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3385252768 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d4036030 con 0x556cb1ab59b0 2024-04-25T18:52:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.727+0000 7fd6c4c00640 1 -- 172.21.15.112:0/3385252768 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd6cc01f830 con 0x556cb1ab59b0 2024-04-25T18:52:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.727+0000 7fd6e51c5740 10 client.25682 fetch_fsmap learned FSMap version 32 2024-04-25T18:52:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.727+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3385252768 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556cb1acf8a0 con 0x556cb1ab59b0 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.727+0000 7fd6c4c00640 1 -- 172.21.15.112:0/3385252768 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d4036030 con 0x556cb1ab59b0 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.727+0000 7fd6c4c00640 1 -- 172.21.15.112:0/3385252768 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd6cc027070 con 0x556cb1ab59b0 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.728+0000 7fd6e51c5740 10 client.25682 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.728+0000 7fd6e51c5740 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.728+0000 7fd6e51c5740 20 client.25682 populate_metadata read hostname 'smithi112' 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.728+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556cb1ad0e00 0x556cb1ad3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.728+0000 7fd6e51c5740 4 client.25682 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.728+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3385252768 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cb1a2f8b0 con 0x556cb1ad0e00 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.728+0000 7fd6daa00640 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556cb1ad0e00 0x556cb1ad3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.728+0000 7fd6daa00640 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556cb1ad0e00 0x556cb1ad3270 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe43c3682c9ef44 server_cookie=39cbb3fef1fe434b in_seq=0 out_seq=0 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.728+0000 7fd6c4c00640 10 client.25682 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6c4c00640 1 -- 172.21.15.112:0/3385252768 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cb1a2f8b0 con 0x556cb1ad0e00 2024-04-25T18:52:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6c4c00640 10 client.25682 handle_command_reply: tid=0 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6e51c5740 1 client.25682 shutdown 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3385252768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 msgr2=0x556cb1abdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 0x556cb1abdb20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc004630 tx=0x7fd6cc004730 comp rx=0 tx=0).stop 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3385252768 shutdown_connections 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556cb1ad0e00 0x556cb1ad3270 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ac0b90 0x556cb1abe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab62d0 0x556cb1abe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/3385252768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 0x556cb1abdb20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6c4c00640 0 client.25682 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.729+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3385252768 >> 172.21.15.112:0/3385252768 conn(0x556cb1ab1ec0 msgr2=0x556cb1ab7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.730+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3385252768 shutdown_connections 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.730+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3385252768 wait complete. 2024-04-25T18:52:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.730+0000 7fd6e51c5740 20 client.25682 trim_cache size 0 max 16384 2024-04-25T18:52:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.750+0000 7fd6e51c5740 1 Processor -- start 2024-04-25T18:52:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.750+0000 7fd6e51c5740 1 -- start start 2024-04-25T18:52:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.750+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 0x556cb1acc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.750+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ab8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.750+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ac0b90 0x556cb1abfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.750+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc1b24f0 con 0x556cb1ab62d0 2024-04-25T18:52:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.750+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc1afd60 con 0x556cb1ab59b0 2024-04-25T18:52:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.750+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc1adbb0 con 0x556cb1ac0b90 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.751+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ac0b90 0x556cb1abfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.751+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ac0b90 0x556cb1abfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:46634/0 (socket says 172.21.15.112:46634) 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.751+0000 7fd6dbe00640 1 -- 172.21.15.112:0/3911827526 learned_addr learned my addr 172.21.15.112:0/3911827526 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.751+0000 7fd6daa00640 1 --2- 172.21.15.112:0/3911827526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ab8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.751+0000 7fd6db400640 1 --2- 172.21.15.112:0/3911827526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 0x556cb1acc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.751+0000 7fd6dbe00640 1 -- 172.21.15.112:0/3911827526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 msgr2=0x556cb1acc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.751+0000 7fd6dbe00640 1 --2- 172.21.15.112:0/3911827526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 0x556cb1acc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.751+0000 7fd6dbe00640 1 -- 172.21.15.112:0/3911827526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 msgr2=0x556cb1ab8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.751+0000 7fd6dbe00640 1 --2- 172.21.15.112:0/3911827526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ab8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.751+0000 7fd6dbe00640 1 -- 172.21.15.112:0/3911827526 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb1aad820 con 0x556cb1ac0b90 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.752+0000 7fd6dbe00640 1 --2- 172.21.15.112:0/3911827526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ac0b90 0x556cb1abfa60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4003d70 tx=0x7fd6d4003d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7b141ff4e18960 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.752+0000 7fd6c7400640 1 -- 172.21.15.112:0/3911827526 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d404a040 con 0x556cb1ac0b90 2024-04-25T18:52:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.752+0000 7fd6c7400640 1 -- 172.21.15.112:0/3911827526 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d4036040 con 0x556cb1ac0b90 2024-04-25T18:52:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.752+0000 7fd6c7400640 1 -- 172.21.15.112:0/3911827526 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d402f870 con 0x556cb1ac0b90 2024-04-25T18:52:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.753+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3911827526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ac0b90 msgr2=0x556cb1abfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.753+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/3911827526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ac0b90 0x556cb1abfa60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4003d70 tx=0x7fd6d4003d00 comp rx=0 tx=0).stop 2024-04-25T18:52:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.753+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3911827526 shutdown_connections 2024-04-25T18:52:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.753+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/3911827526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ac0b90 0x556cb1abfa60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.753+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/3911827526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ab8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.753+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/3911827526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ab59b0 0x556cb1acc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.753+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3911827526 >> 172.21.15.112:0/3911827526 conn(0x556cb1ab1ec0 msgr2=0x556cb1abf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.753+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3911827526 shutdown_connections 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.753+0000 7fd6e51c5740 1 -- 172.21.15.112:0/3911827526 wait complete. 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.753+0000 7fd6e51c5740 1 -- start start 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6e51c5740 1 Processor -- start 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ac8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ac60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6e51c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ac0b90 0x556cb1ac65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc1ae220 con 0x556cb1ab62d0 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc1af690 con 0x556cb1ac0b90 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6e51c5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc1ae8f0 con 0x556cb1ab59b0 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6e51c5740 10 client.0 _ll_register_callbacks cb 0x7fd6cc0044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6db400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ac8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6db400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ac8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:46642/0 (socket says 172.21.15.112:46642) 2024-04-25T18:52:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6db400640 1 -- 172.21.15.112:0/2335100400 learned_addr learned my addr 172.21.15.112:0/2335100400 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6dbe00640 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ac0b90 0x556cb1ac65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.754+0000 7fd6daa00640 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ac60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6db400640 1 -- 172.21.15.112:0/2335100400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ac0b90 msgr2=0x556cb1ac65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6db400640 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ac0b90 0x556cb1ac65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6db400640 1 -- 172.21.15.112:0/2335100400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 msgr2=0x556cb1ac60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6db400640 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ac60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6db400640 1 -- 172.21.15.112:0/2335100400 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb1ac6db0 con 0x556cb1ab59b0 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6daa00640 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ac60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6db400640 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ac8d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009730 tx=0x7fd6cc029670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=824ec477f95083e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6c4c00640 1 -- 172.21.15.112:0/2335100400 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc009e50 con 0x556cb1ab59b0 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2335100400 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d4036040 con 0x556cb1ab59b0 2024-04-25T18:52:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6c4c00640 1 -- 172.21.15.112:0/2335100400 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7fd6cc031400 con 0x556cb1ab59b0 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6c4c00640 1 -- 172.21.15.112:0/2335100400 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd6cc031600 con 0x556cb1ab59b0 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6e51c5740 10 client.25705 fetch_fsmap learned FSMap version 32 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.755+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2335100400 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556cb1b373b0 con 0x556cb1ab59b0 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.756+0000 7fd6c4c00640 1 -- 172.21.15.112:0/2335100400 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d4036040 con 0x556cb1ab59b0 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.756+0000 7fd6c4c00640 1 -- 172.21.15.112:0/2335100400 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd6cc038070 con 0x556cb1ab59b0 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.756+0000 7fd6e51c5740 10 client.25705 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.756+0000 7fd6e51c5740 10 client.25705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.756+0000 7fd6e51c5740 20 client.25705 populate_metadata read hostname 'smithi112' 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.756+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556cb1b37b50 0x556cb1b39f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.756+0000 7fd6e51c5740 4 client.25705 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.756+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2335100400 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cb1a29320 con 0x556cb1b37b50 2024-04-25T18:52:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.756+0000 7fd6daa00640 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556cb1b37b50 0x556cb1b39f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.757+0000 7fd6daa00640 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556cb1b37b50 0x556cb1b39f70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7def0b75ba29767 server_cookie=72474a44039e011f in_seq=0 out_seq=0 2024-04-25T18:52:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.757+0000 7fd6c4c00640 10 client.25705 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6c4c00640 1 -- 172.21.15.112:0/2335100400 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556cb1a29320 con 0x556cb1b37b50 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6c4c00640 10 client.25705 handle_command_reply: tid=0 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6e51c5740 1 client.25705 shutdown 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2335100400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 msgr2=0x556cb1ac8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ac8d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009730 tx=0x7fd6cc029670 comp rx=0 tx=0).stop 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2335100400 shutdown_connections 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556cb1ac0b90 0x556cb1ac65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x556cb1b37b50 0x556cb1b39f70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1ab62d0 0x556cb1ac60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6e51c5740 1 --2- 172.21.15.112:0/2335100400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556cb1ab59b0 0x556cb1ac8d10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6c4c00640 0 client.25705 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.827+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2335100400 >> 172.21.15.112:0/2335100400 conn(0x556cb1ab1ec0 msgr2=0x556cb1ab1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.828+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2335100400 shutdown_connections 2024-04-25T18:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.828+0000 7fd6e51c5740 1 -- 172.21.15.112:0/2335100400 wait complete. 2024-04-25T18:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.828+0000 7fd6e51c5740 20 client.25705 trim_cache size 0 max 16384 2024-04-25T18:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:52:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2920634110 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd6c00a3dc0 msgr2=0x7fd6c00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd6c00a3dc0 0x7fd6c00a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x556cb1ab6f50 tx=0x7fd6cc005f90 comp rx=0 tx=0).stop 2024-04-25T18:52:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2920634110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6dc105f90 msgr2=0x7fd6dc102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6dc105f90 0x7fd6dc102e90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800e9f0 tx=0x7fd6c800eeb0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2920634110 shutdown_connections 2024-04-25T18:52:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd6c00a3dc0 0x7fd6c00a6280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6dc1033d0 0x7fd6dc1a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6dc105f90 0x7fd6dc102e90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 --2- 172.21.15.112:0/2920634110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6dc100ec0 0x7fd6dc102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2920634110 >> 172.21.15.112:0/2920634110 conn(0x7fd6dc0fc760 msgr2=0x7fd6dc06a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2920634110 shutdown_connections 2024-04-25T18:52:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:13.829+0000 7fd6e0c00640 1 -- 172.21.15.112:0/2920634110 wait complete. 2024-04-25T18:52:14.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:14 smithi171 ceph-mon[33266]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-04-25T18:52:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:14 smithi161 ceph-mon[32609]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-04-25T18:52:14.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:14 smithi112 ceph-mon[27864]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-04-25T18:52:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:16 smithi171 ceph-mon[33266]: pgmap v150: 97 pgs: 97 active+clean; 40 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-04-25T18:52:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:16 smithi161 ceph-mon[32609]: pgmap v150: 97 pgs: 97 active+clean; 40 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-04-25T18:52:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:16 smithi112 ceph-mon[27864]: pgmap v150: 97 pgs: 97 active+clean; 40 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-04-25T18:52:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:18 smithi171 ceph-mon[33266]: pgmap v151: 97 pgs: 97 active+clean; 37 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-04-25T18:52:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:18 smithi161 ceph-mon[32609]: pgmap v151: 97 pgs: 97 active+clean; 37 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-04-25T18:52:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:18 smithi112 ceph-mon[27864]: pgmap v151: 97 pgs: 97 active+clean; 37 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-04-25T18:52:20.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:19 smithi171 ceph-mon[33266]: pgmap v152: 97 pgs: 97 active+clean; 37 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 246 op/s 2024-04-25T18:52:20.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:19 smithi161 ceph-mon[32609]: pgmap v152: 97 pgs: 97 active+clean; 37 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 246 op/s 2024-04-25T18:52:20.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:19 smithi112 ceph-mon[27864]: pgmap v152: 97 pgs: 97 active+clean; 37 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 246 op/s 2024-04-25T18:52:20.654 INFO:tasks.workunit.client.0.smithi112.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-04-25T18:52:20.655 INFO:tasks.workunit.client.0.smithi112.stdout:All 50000 operations completed A-OK! 2024-04-25T18:52:20.660 INFO:tasks.workunit.client.0.smithi112.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-04-25T18:52:20.665 INFO:tasks.workunit.client.0.smithi112.stdout:Seed set to 1 2024-04-25T18:52:20.822 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T18:52:20.822 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T18:52:20.824 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T18:52:20.825 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T18:52:20.825 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T18:52:20.829 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T18:52:20.832 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero length punch hole 2024-04-25T18:52:20.832 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero length punch hole 2024-04-25T18:52:20.832 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T18:52:20.832 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero length punch hole 2024-04-25T18:52:20.832 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-04-25T18:52:20.833 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero length punch hole 2024-04-25T18:52:20.833 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-04-25T18:52:20.833 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero length punch hole 2024-04-25T18:52:20.833 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x4310e9 2024-04-25T18:52:20.842 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x954a45 2024-04-25T18:52:20.894 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x9bc582 2024-04-25T18:52:20.943 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x9eecbc 2024-04-25T18:52:22.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:21 smithi171 ceph-mon[33266]: pgmap v153: 97 pgs: 97 active+clean; 39 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-04-25T18:52:22.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:21 smithi161 ceph-mon[32609]: pgmap v153: 97 pgs: 97 active+clean; 39 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-04-25T18:52:22.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:21 smithi112 ceph-mon[27864]: pgmap v153: 97 pgs: 97 active+clean; 39 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-04-25T18:52:24.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:23 smithi171 ceph-mon[33266]: pgmap v154: 97 pgs: 97 active+clean; 39 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-04-25T18:52:24.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:23 smithi161 ceph-mon[32609]: pgmap v154: 97 pgs: 97 active+clean; 39 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-04-25T18:52:24.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:23 smithi112 ceph-mon[27864]: pgmap v154: 97 pgs: 97 active+clean; 39 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-04-25T18:52:25.978 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x9fdd84 2024-04-25T18:52:26.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:25 smithi171 ceph-mon[33266]: pgmap v155: 97 pgs: 97 active+clean; 39 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-04-25T18:52:26.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:25 smithi161 ceph-mon[32609]: pgmap v155: 97 pgs: 97 active+clean; 39 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-04-25T18:52:26.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:25 smithi112 ceph-mon[27864]: pgmap v155: 97 pgs: 97 active+clean; 39 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-04-25T18:52:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.982+0000 7f204fe00640 1 -- 172.21.15.112:0/4131560211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204806a150 msgr2=0x7f204806a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.982+0000 7f204fe00640 1 --2- 172.21.15.112:0/4131560211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204806a150 0x7f204806a5b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2034009900 tx=0x7f203402a410 comp rx=0 tx=0).stop 2024-04-25T18:52:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.982+0000 7f204fe00640 1 -- 172.21.15.112:0/4131560211 shutdown_connections 2024-04-25T18:52:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.982+0000 7f204fe00640 1 --2- 172.21.15.112:0/4131560211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f204806aaf0 0x7f204810ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.982+0000 7f204fe00640 1 --2- 172.21.15.112:0/4131560211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204806a150 0x7f204806a5b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.982+0000 7f204fe00640 1 --2- 172.21.15.112:0/4131560211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2048108060 0x7f2048108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.982+0000 7f204fe00640 1 -- 172.21.15.112:0/4131560211 >> 172.21.15.112:0/4131560211 conn(0x7f2048077d30 msgr2=0x7f2048078140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.982+0000 7f204fe00640 1 -- 172.21.15.112:0/4131560211 shutdown_connections 2024-04-25T18:52:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.982+0000 7f204fe00640 1 -- 172.21.15.112:0/4131560211 wait complete. 2024-04-25T18:52:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.983+0000 7f204fe00640 1 Processor -- start 2024-04-25T18:52:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.983+0000 7f204fe00640 1 -- start start 2024-04-25T18:52:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.984+0000 7f204fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204806a150 0x7f20481a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.984+0000 7f204fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f204806aaf0 0x7f20481a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.984+0000 7f204fe00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2048108060 0x7f204819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.984+0000 7f204fe00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2048112200 con 0x7f2048108060 2024-04-25T18:52:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.984+0000 7f204fe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2048112080 con 0x7f204806a150 2024-04-25T18:52:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.984+0000 7f204fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2048112380 con 0x7f204806aaf0 2024-04-25T18:52:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.984+0000 7f204ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f204806aaf0 0x7f20481a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.984+0000 7f204d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204806a150 0x7f20481a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.984+0000 7f204ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f204806aaf0 0x7f20481a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:34716/0 (socket says 172.21.15.112:34716) 2024-04-25T18:52:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.984+0000 7f204ca00640 1 -- 172.21.15.112:0/2387136837 learned_addr learned my addr 172.21.15.112:0/2387136837 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.985+0000 7f204de00640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2048108060 0x7f204819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.985+0000 7f204d400640 1 -- 172.21.15.112:0/2387136837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f204806aaf0 msgr2=0x7f20481a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.985+0000 7f204d400640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f204806aaf0 0x7f20481a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.985+0000 7f204d400640 1 -- 172.21.15.112:0/2387136837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2048108060 msgr2=0x7f204819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.985+0000 7f204d400640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2048108060 0x7f204819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.985+0000 7f204d400640 1 -- 172.21.15.112:0/2387136837 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204819aef0 con 0x7f204806a150 2024-04-25T18:52:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.986+0000 7f204d400640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204806a150 0x7f20481a0570 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f203c00e9f0 tx=0x7f203c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.986+0000 7f2046000640 1 -- 172.21.15.112:0/2387136837 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c00ccc0 con 0x7f204806a150 2024-04-25T18:52:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.986+0000 7f2046000640 1 -- 172.21.15.112:0/2387136837 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f203c005020 con 0x7f204806a150 2024-04-25T18:52:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.986+0000 7f204fe00640 1 -- 172.21.15.112:0/2387136837 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204819b1e0 con 0x7f204806a150 2024-04-25T18:52:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.986+0000 7f2046000640 1 -- 172.21.15.112:0/2387136837 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c004a00 con 0x7f204806a150 2024-04-25T18:52:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.986+0000 7f204fe00640 1 -- 172.21.15.112:0/2387136837 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20481a74f0 con 0x7f204806a150 2024-04-25T18:52:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.990+0000 7f2052c30740 1 Processor -- start 2024-04-25T18:52:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.990+0000 7f2052c30740 1 -- start start 2024-04-25T18:52:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.990+0000 7f2046000640 1 -- 172.21.15.112:0/2387136837 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f203c004ba0 con 0x7f204806a150 2024-04-25T18:52:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.990+0000 7f2052c30740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad395fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.990+0000 7f2052c30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 0x55aad396a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.990+0000 7f2052c30740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad396cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.990+0000 7f2052c30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aad38c5900 con 0x55aad3960140 2024-04-25T18:52:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.990+0000 7f2052c30740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aad386bef0 con 0x55aad395f820 2024-04-25T18:52:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.990+0000 7f2052c30740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55aad38c18c0 con 0x55aad396aa00 2024-04-25T18:52:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.991+0000 7f204de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad395fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.991+0000 7f204ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad396cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.991+0000 7f204de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad395fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:51784/0 (socket says 172.21.15.112:51784) 2024-04-25T18:52:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.991+0000 7f204de00640 1 -- 172.21.15.112:0/1444484682 learned_addr learned my addr 172.21.15.112:0/1444484682 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.991+0000 7f204de00640 1 -- 172.21.15.112:0/1444484682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 msgr2=0x55aad396cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.991+0000 7f204de00640 1 --2- 172.21.15.112:0/1444484682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad396cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.991+0000 7f204de00640 1 -- 172.21.15.112:0/1444484682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 msgr2=0x55aad396a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.991+0000 7f204de00640 1 --2- 172.21.15.112:0/1444484682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 0x55aad396a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.991+0000 7f204de00640 1 -- 172.21.15.112:0/1444484682 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad35d8210 con 0x55aad395f820 2024-04-25T18:52:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.991+0000 7f204de00640 1 --2- 172.21.15.112:0/1444484682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad395fc00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2040009fc0 tx=0x7f2040010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16e82c3b6dbf94d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.992+0000 7f203a600640 1 -- 172.21.15.112:0/1444484682 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040003b70 con 0x55aad395f820 2024-04-25T18:52:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.992+0000 7f203a600640 1 -- 172.21.15.112:0/1444484682 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2040003d10 con 0x55aad395f820 2024-04-25T18:52:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.992+0000 7f203a600640 1 -- 172.21.15.112:0/1444484682 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040011a40 con 0x55aad395f820 2024-04-25T18:52:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.992+0000 7f2046000640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f20300a3e40 0x7f20300a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.992+0000 7f2046000640 1 -- 172.21.15.112:0/2387136837 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7f203c0d07a0 con 0x7f204806a150 2024-04-25T18:52:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.992+0000 7f204d400640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f20300a3e40 0x7f20300a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.993+0000 7f2052c30740 1 -- 172.21.15.112:0/1444484682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 msgr2=0x55aad395fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.993+0000 7f2052c30740 1 --2- 172.21.15.112:0/1444484682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad395fc00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2040009fc0 tx=0x7f2040010860 comp rx=0 tx=0).stop 2024-04-25T18:52:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.993+0000 7f2052c30740 1 -- 172.21.15.112:0/1444484682 shutdown_connections 2024-04-25T18:52:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.993+0000 7f2052c30740 1 --2- 172.21.15.112:0/1444484682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad396cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.993+0000 7f2052c30740 1 --2- 172.21.15.112:0/1444484682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 0x55aad396a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.993+0000 7f2052c30740 1 --2- 172.21.15.112:0/1444484682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad395fc00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.993+0000 7f2052c30740 1 -- 172.21.15.112:0/1444484682 >> 172.21.15.112:0/1444484682 conn(0x55aad395bd30 msgr2=0x55aad395c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.994+0000 7f204d400640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f20300a3e40 0x7f20300a6300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55aad3960dc0 tx=0x7f203c0096e0 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.994+0000 7f2052c30740 1 -- 172.21.15.112:0/1444484682 shutdown_connections 2024-04-25T18:52:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.994+0000 7f2052c30740 1 -- 172.21.15.112:0/1444484682 wait complete. 2024-04-25T18:52:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.994+0000 7f2052c30740 1 -- start start 2024-04-25T18:52:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.995+0000 7f2052c30740 1 Processor -- start 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.995+0000 7f2052c30740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3967a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.995+0000 7f2052c30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 0x55aad3967f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.995+0000 7f2052c30740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad3968490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.995+0000 7f2052c30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aad396d630 con 0x55aad3960140 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.995+0000 7f2052c30740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aad387f310 con 0x55aad395f820 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.995+0000 7f2052c30740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55aad396d7b0 con 0x55aad396aa00 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.995+0000 7f2052c30740 10 client.0 _ll_register_callbacks cb 0x55aad384dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.995+0000 7f204de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3967a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.996+0000 7f204de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3967a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:51786/0 (socket says 172.21.15.112:51786) 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.996+0000 7f204de00640 1 -- 172.21.15.112:0/2752181008 learned_addr learned my addr 172.21.15.112:0/2752181008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.996+0000 7f204de00640 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad3968490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.996+0000 7f204ca00640 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 0x55aad3967f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.996+0000 7f204de00640 1 -- 172.21.15.112:0/2752181008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 msgr2=0x55aad3968490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.996+0000 7f204de00640 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad3968490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.996+0000 7f204de00640 1 -- 172.21.15.112:0/2752181008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 msgr2=0x55aad3967f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.996+0000 7f204de00640 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 0x55aad3967f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.996+0000 7f204de00640 1 -- 172.21.15.112:0/2752181008 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad36a4620 con 0x55aad395f820 2024-04-25T18:52:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.996+0000 7f204de00640 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3967a10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2040004980 tx=0x7f20400115a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3661a9edb26784b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.996+0000 7f202ea00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:52:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.997+0000 7f202ea00640 1 -- 172.21.15.112:0/2752181008 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040024040 con 0x55aad395f820 2024-04-25T18:52:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.997+0000 7f202ea00640 1 -- 172.21.15.112:0/2752181008 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7f2040004a40 con 0x55aad395f820 2024-04-25T18:52:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.997+0000 7f2052c30740 1 -- 172.21.15.112:0/2752181008 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2040003d10 con 0x55aad395f820 2024-04-25T18:52:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.997+0000 7f202ea00640 1 -- 172.21.15.112:0/2752181008 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2040003ec0 con 0x55aad395f820 2024-04-25T18:52:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.997+0000 7f2052c30740 10 client.25700 fetch_fsmap learned FSMap version 33 2024-04-25T18:52:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.997+0000 7f2052c30740 1 -- 172.21.15.112:0/2752181008 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aad3979920 con 0x55aad395f820 2024-04-25T18:52:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.998+0000 7f202ea00640 1 -- 172.21.15.112:0/2752181008 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2040003d10 con 0x55aad395f820 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.998+0000 7f202ea00640 1 -- 172.21.15.112:0/2752181008 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7f204002b670 con 0x55aad395f820 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.998+0000 7f2052c30740 10 client.25700 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.998+0000 7f2052c30740 10 client.25700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.998+0000 7f2052c30740 20 client.25700 populate_metadata read hostname 'smithi112' 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.998+0000 7f2052c30740 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aad397ad20 0x55aad397d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.998+0000 7f2052c30740 4 client.25700 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.998+0000 7f2052c30740 1 -- 172.21.15.112:0/2752181008 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aad38d98b0 con 0x55aad397ad20 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.998+0000 7f204ca00640 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aad397ad20 0x55aad397d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.999+0000 7f204ca00640 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aad397ad20 0x55aad397d110 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa5e6ec491fd4e3f server_cookie=231447b4848ebef5 in_seq=0 out_seq=0 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:26.999+0000 7f202ea00640 10 client.25700 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f202ea00640 1 -- 172.21.15.112:0/2752181008 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aad38d98b0 con 0x55aad397ad20 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f202ea00640 10 client.25700 handle_command_reply: tid=0 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f2052c30740 1 client.25700 shutdown 2024-04-25T18:52:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f2052c30740 1 -- 172.21.15.112:0/2752181008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 msgr2=0x55aad3967a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f2052c30740 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3967a10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2040004980 tx=0x7f20400115a0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f2052c30740 1 -- 172.21.15.112:0/2752181008 shutdown_connections 2024-04-25T18:52:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f2052c30740 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad3968490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f2052c30740 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aad397ad20 0x55aad397d110 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f2052c30740 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 0x55aad3967f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f202ea00640 0 client.25700 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f2052c30740 1 --2- 172.21.15.112:0/2752181008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3967a10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f2052c30740 1 -- 172.21.15.112:0/2752181008 >> 172.21.15.112:0/2752181008 conn(0x55aad395bd30 msgr2=0x55aad3961ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.000+0000 7f2052c30740 1 -- 172.21.15.112:0/2752181008 shutdown_connections 2024-04-25T18:52:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.001+0000 7f2052c30740 1 -- 172.21.15.112:0/2752181008 wait complete. 2024-04-25T18:52:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.001+0000 7f2052c30740 20 client.25700 trim_cache size 0 max 16384 2024-04-25T18:52:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f2052c30740 1 Processor -- start 2024-04-25T18:52:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f2052c30740 1 -- start start 2024-04-25T18:52:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f2052c30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad395f820 0x55aad39764f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f2052c30740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad3960140 0x55aad39621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f2052c30740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad396aa00 0x55aad39680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f2052c30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20481b2690 con 0x55aad395f820 2024-04-25T18:52:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f2052c30740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f20481aff30 con 0x55aad396aa00 2024-04-25T18:52:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f2052c30740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f20481add80 con 0x55aad3960140 2024-04-25T18:52:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f204ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad3960140 0x55aad39621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f204ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad3960140 0x55aad39621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:34742/0 (socket says 172.21.15.112:34742) 2024-04-25T18:52:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f204ca00640 1 -- 172.21.15.112:0/2098591411 learned_addr learned my addr 172.21.15.112:0/2098591411 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.021+0000 7f204de00640 1 --2- 172.21.15.112:0/2098591411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad395f820 0x55aad39764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.022+0000 7f204de00640 1 --2- 172.21.15.112:0/2098591411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad396aa00 0x55aad39680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.022+0000 7f204ca00640 1 -- 172.21.15.112:0/2098591411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad396aa00 msgr2=0x55aad39680e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.022+0000 7f204ca00640 1 --2- 172.21.15.112:0/2098591411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad396aa00 0x55aad39680e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.022+0000 7f204ca00640 1 -- 172.21.15.112:0/2098591411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad395f820 msgr2=0x55aad39764f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.022+0000 7f204ca00640 1 --2- 172.21.15.112:0/2098591411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad395f820 0x55aad39764f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.022+0000 7f204ca00640 1 -- 172.21.15.112:0/2098591411 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad39626f0 con 0x55aad3960140 2024-04-25T18:52:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.022+0000 7f204de00640 1 --2- 172.21.15.112:0/2098591411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad396aa00 0x55aad39680e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:52:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.022+0000 7f204ca00640 1 --2- 172.21.15.112:0/2098591411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad3960140 0x55aad39621b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2034005c40 tx=0x7f20340043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701e117ccba5c062 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.023+0000 7f2039c00640 1 -- 172.21.15.112:0/2098591411 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203404a070 con 0x55aad3960140 2024-04-25T18:52:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.023+0000 7f2039c00640 1 -- 172.21.15.112:0/2098591411 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2034005020 con 0x55aad3960140 2024-04-25T18:52:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.023+0000 7f2039c00640 1 -- 172.21.15.112:0/2098591411 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034035040 con 0x55aad3960140 2024-04-25T18:52:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.023+0000 7f2052c30740 1 -- 172.21.15.112:0/2098591411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad3960140 msgr2=0x55aad39621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.023+0000 7f2052c30740 1 --2- 172.21.15.112:0/2098591411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad3960140 0x55aad39621b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2034005c40 tx=0x7f20340043d0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.024+0000 7f2052c30740 1 -- 172.21.15.112:0/2098591411 shutdown_connections 2024-04-25T18:52:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.024+0000 7f2052c30740 1 --2- 172.21.15.112:0/2098591411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad396aa00 0x55aad39680e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.024+0000 7f2052c30740 1 --2- 172.21.15.112:0/2098591411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad3960140 0x55aad39621b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.024+0000 7f2052c30740 1 --2- 172.21.15.112:0/2098591411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad395f820 0x55aad39764f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.024+0000 7f2052c30740 1 -- 172.21.15.112:0/2098591411 >> 172.21.15.112:0/2098591411 conn(0x55aad395bd30 msgr2=0x55aad39679d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.024+0000 7f2052c30740 1 -- 172.21.15.112:0/2098591411 shutdown_connections 2024-04-25T18:52:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.024+0000 7f2052c30740 1 -- 172.21.15.112:0/2098591411 wait complete. 2024-04-25T18:52:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.024+0000 7f2052c30740 1 -- start start 2024-04-25T18:52:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.024+0000 7f2052c30740 1 Processor -- start 2024-04-25T18:52:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.025+0000 7f2052c30740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3972cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.025+0000 7f2052c30740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 0x55aad3970030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.025+0000 7f2052c30740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad3970570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.025+0000 7f2052c30740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20481ae3f0 con 0x55aad3960140 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.025+0000 7f2052c30740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f20481af860 con 0x55aad395f820 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.025+0000 7f2052c30740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f20481aeac0 con 0x55aad396aa00 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.025+0000 7f2052c30740 10 client.0 _ll_register_callbacks cb 0x7f2040011bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.025+0000 7f204de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3972cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.025+0000 7f204de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3972cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:51806/0 (socket says 172.21.15.112:51806) 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.025+0000 7f204de00640 1 -- 172.21.15.112:0/513773056 learned_addr learned my addr 172.21.15.112:0/513773056 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.025+0000 7f204de00640 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad3970570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.026+0000 7f204ca00640 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 0x55aad3970030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.026+0000 7f204de00640 1 -- 172.21.15.112:0/513773056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 msgr2=0x55aad3970570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.026+0000 7f204de00640 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad3970570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.026+0000 7f204de00640 1 -- 172.21.15.112:0/513773056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 msgr2=0x55aad3970030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.026+0000 7f204de00640 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 0x55aad3970030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.026+0000 7f204de00640 1 -- 172.21.15.112:0/513773056 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad3970c60 con 0x55aad395f820 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.026+0000 7f204de00640 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3972cf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2040011b90 tx=0x7f204002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e350b572038133 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.026+0000 7f202ea00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.026+0000 7f202ea00640 1 -- 172.21.15.112:0/513773056 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040031030 con 0x55aad395f820 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.026+0000 7f2052c30740 1 -- 172.21.15.112:0/513773056 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2034005020 con 0x55aad395f820 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.026+0000 7f202ea00640 1 -- 172.21.15.112:0/513773056 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7f2040003970 con 0x55aad395f820 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f202ea00640 1 -- 172.21.15.112:0/513773056 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2040005bd0 con 0x55aad395f820 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f2052c30740 10 client.25703 fetch_fsmap learned FSMap version 33 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f2052c30740 1 -- 172.21.15.112:0/513773056 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aad39e0ea0 con 0x55aad395f820 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f202ea00640 1 -- 172.21.15.112:0/513773056 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2034005020 con 0x55aad395f820 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f202ea00640 1 -- 172.21.15.112:0/513773056 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7f204002b5f0 con 0x55aad395f820 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f2052c30740 10 client.25703 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T18:52:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f2052c30740 10 client.25703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:52:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f2052c30740 20 client.25703 populate_metadata read hostname 'smithi112' 2024-04-25T18:52:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f2052c30740 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aad39e1500 0x55aad39e3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f2052c30740 4 client.25703 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:52:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f2052c30740 1 -- 172.21.15.112:0/513773056 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aad38d3320 con 0x55aad39e1500 2024-04-25T18:52:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.027+0000 7f204ca00640 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aad39e1500 0x55aad39e3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.028+0000 7f204ca00640 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aad39e1500 0x55aad39e3920 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241a6e95f63b61b1 server_cookie=c99ec6f887ba0ca2 in_seq=0 out_seq=0 2024-04-25T18:52:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.028+0000 7f202ea00640 10 client.25703 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f202ea00640 1 -- 172.21.15.112:0/513773056 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aad38d3320 con 0x55aad39e1500 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f202ea00640 10 client.25703 handle_command_reply: tid=0 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f2052c30740 1 client.25703 shutdown 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f2052c30740 1 -- 172.21.15.112:0/513773056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 msgr2=0x55aad3972cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f2052c30740 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3972cf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2040011b90 tx=0x7f204002e040 comp rx=0 tx=0).stop 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f2052c30740 1 -- 172.21.15.112:0/513773056 shutdown_connections 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f2052c30740 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aad396aa00 0x55aad3970570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f2052c30740 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aad39e1500 0x55aad39e3920 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f2052c30740 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aad3960140 0x55aad3970030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f2052c30740 1 --2- 172.21.15.112:0/513773056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aad395f820 0x55aad3972cf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f2052c30740 1 -- 172.21.15.112:0/513773056 >> 172.21.15.112:0/513773056 conn(0x55aad395bd30 msgr2=0x55aad3968260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.046+0000 7f202ea00640 0 client.25703 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.047+0000 7f2052c30740 1 -- 172.21.15.112:0/513773056 shutdown_connections 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.047+0000 7f2052c30740 1 -- 172.21.15.112:0/513773056 wait complete. 2024-04-25T18:52:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.047+0000 7f2052c30740 20 client.25703 trim_cache size 0 max 16384 2024-04-25T18:52:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:52:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:52:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:52:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:52:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.047+0000 7f204fe00640 1 -- 172.21.15.112:0/2387136837 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f20300a3e40 msgr2=0x7f20300a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.047+0000 7f204fe00640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f20300a3e40 0x7f20300a6300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55aad3960dc0 tx=0x7f203c0096e0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.048+0000 7f204fe00640 1 -- 172.21.15.112:0/2387136837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204806a150 msgr2=0x7f20481a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.048+0000 7f204fe00640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204806a150 0x7f20481a0570 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f203c00e9f0 tx=0x7f203c00eeb0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.048+0000 7f204fe00640 1 -- 172.21.15.112:0/2387136837 shutdown_connections 2024-04-25T18:52:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.048+0000 7f204fe00640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f20300a3e40 0x7f20300a6300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.048+0000 7f204fe00640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2048108060 0x7f204819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.048+0000 7f204fe00640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f204806aaf0 0x7f20481a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.048+0000 7f204fe00640 1 --2- 172.21.15.112:0/2387136837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204806a150 0x7f20481a0570 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.048+0000 7f204fe00640 1 -- 172.21.15.112:0/2387136837 >> 172.21.15.112:0/2387136837 conn(0x7f2048077d30 msgr2=0x7f204810d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.048+0000 7f204fe00640 1 -- 172.21.15.112:0/2387136837 shutdown_connections 2024-04-25T18:52:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:27.048+0000 7f204fe00640 1 -- 172.21.15.112:0/2387136837 wait complete. 2024-04-25T18:52:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:27 smithi171 ceph-mon[33266]: pgmap v156: 97 pgs: 97 active+clean; 44 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 401 op/s 2024-04-25T18:52:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:27 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:52:28.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:27 smithi161 ceph-mon[32609]: pgmap v156: 97 pgs: 97 active+clean; 44 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 401 op/s 2024-04-25T18:52:28.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:27 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:52:28.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:27 smithi112 ceph-mon[27864]: pgmap v156: 97 pgs: 97 active+clean; 44 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 401 op/s 2024-04-25T18:52:28.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:27 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:52:30.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:29 smithi171 ceph-mon[33266]: pgmap v157: 97 pgs: 97 active+clean; 44 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 242 op/s 2024-04-25T18:52:30.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:29 smithi161 ceph-mon[32609]: pgmap v157: 97 pgs: 97 active+clean; 44 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 242 op/s 2024-04-25T18:52:30.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:29 smithi112 ceph-mon[27864]: pgmap v157: 97 pgs: 97 active+clean; 44 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 242 op/s 2024-04-25T18:52:32.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:31 smithi171 ceph-mon[33266]: pgmap v158: 97 pgs: 97 active+clean; 47 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.2 MiB/s wr, 390 op/s 2024-04-25T18:52:32.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:31 smithi161 ceph-mon[32609]: pgmap v158: 97 pgs: 97 active+clean; 47 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.2 MiB/s wr, 390 op/s 2024-04-25T18:52:32.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:31 smithi112 ceph-mon[27864]: pgmap v158: 97 pgs: 97 active+clean; 47 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.2 MiB/s wr, 390 op/s 2024-04-25T18:52:34.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:33 smithi171 ceph-mon[33266]: pgmap v159: 97 pgs: 97 active+clean; 47 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 286 op/s 2024-04-25T18:52:34.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:33 smithi161 ceph-mon[32609]: pgmap v159: 97 pgs: 97 active+clean; 47 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 286 op/s 2024-04-25T18:52:34.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:33 smithi112 ceph-mon[27864]: pgmap v159: 97 pgs: 97 active+clean; 47 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 286 op/s 2024-04-25T18:52:36.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:35 smithi171 ceph-mon[33266]: pgmap v160: 97 pgs: 97 active+clean; 47 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 286 op/s 2024-04-25T18:52:36.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:35 smithi161 ceph-mon[32609]: pgmap v160: 97 pgs: 97 active+clean; 47 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 286 op/s 2024-04-25T18:52:36.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:35 smithi112 ceph-mon[27864]: pgmap v160: 97 pgs: 97 active+clean; 47 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 286 op/s 2024-04-25T18:52:38.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:37 smithi171 ceph-mon[33266]: pgmap v161: 97 pgs: 97 active+clean; 47 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 418 op/s 2024-04-25T18:52:38.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:37 smithi161 ceph-mon[32609]: pgmap v161: 97 pgs: 97 active+clean; 47 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 418 op/s 2024-04-25T18:52:38.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:37 smithi112 ceph-mon[27864]: pgmap v161: 97 pgs: 97 active+clean; 47 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 418 op/s 2024-04-25T18:52:40.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:39 smithi171 ceph-mon[33266]: pgmap v162: 97 pgs: 97 active+clean; 47 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 279 op/s 2024-04-25T18:52:40.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:39 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.207+0000 7f2224800640 1 -- 172.21.15.112:0/1730036516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c113010 msgr2=0x7f221c115400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.207+0000 7f2224800640 1 --2- 172.21.15.112:0/1730036516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c113010 0x7f221c115400 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2218009cd0 tx=0x7f221802a970 comp rx=0 tx=0).stop 2024-04-25T18:52:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.207+0000 7f2224800640 1 -- 172.21.15.112:0/1730036516 shutdown_connections 2024-04-25T18:52:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.207+0000 7f2224800640 1 --2- 172.21.15.112:0/1730036516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c113010 0x7f221c115400 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.207+0000 7f2224800640 1 --2- 172.21.15.112:0/1730036516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c074b70 0x7f221c112ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.207+0000 7f2224800640 1 --2- 172.21.15.112:0/1730036516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f221c074250 0x7f221c074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.207+0000 7f2224800640 1 -- 172.21.15.112:0/1730036516 >> 172.21.15.112:0/1730036516 conn(0x7f221c0793e0 msgr2=0x7f221c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.207+0000 7f2224800640 1 -- 172.21.15.112:0/1730036516 shutdown_connections 2024-04-25T18:52:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.207+0000 7f2224800640 1 -- 172.21.15.112:0/1730036516 wait complete. 2024-04-25T18:52:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.209+0000 7f2224800640 1 Processor -- start 2024-04-25T18:52:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.209+0000 7f2224800640 1 -- start start 2024-04-25T18:52:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.209+0000 7f2224800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c074250 0x7f221c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.209+0000 7f2224800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f221c074b70 0x7f221c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.209+0000 7f2224800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c113010 0x7f221c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.209+0000 7f2224800640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1187f0 con 0x7f221c113010 2024-04-25T18:52:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.209+0000 7f2224800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f221c118670 con 0x7f221c074250 2024-04-25T18:52:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.209+0000 7f2224800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f221c118970 con 0x7f221c074b70 2024-04-25T18:52:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.210+0000 7f2221e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c074250 0x7f221c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.210+0000 7f2221e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c074250 0x7f221c1a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:52814/0 (socket says 172.21.15.112:52814) 2024-04-25T18:52:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.210+0000 7f2221e00640 1 -- 172.21.15.112:0/1604756390 learned_addr learned my addr 172.21.15.112:0/1604756390 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.210+0000 7f2221400640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f221c074b70 0x7f221c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.210+0000 7f2222800640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c113010 0x7f221c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.210+0000 7f2221e00640 1 -- 172.21.15.112:0/1604756390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f221c074b70 msgr2=0x7f221c1a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.210+0000 7f2221e00640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f221c074b70 0x7f221c1a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.210+0000 7f2221e00640 1 -- 172.21.15.112:0/1604756390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c113010 msgr2=0x7f221c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.210+0000 7f2221e00640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c113010 0x7f221c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.210+0000 7f2221e00640 1 -- 172.21.15.112:0/1604756390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221c19f270 con 0x7f221c074250 2024-04-25T18:52:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.210+0000 7f2221e00640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c074250 0x7f221c1a4940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f221000e9f0 tx=0x7f221000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.211+0000 7f2216a00640 1 -- 172.21.15.112:0/1604756390 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221000cd80 con 0x7f221c074250 2024-04-25T18:52:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.211+0000 7f2224800640 1 -- 172.21.15.112:0/1604756390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221c19f560 con 0x7f221c074250 2024-04-25T18:52:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.211+0000 7f2216a00640 1 -- 172.21.15.112:0/1604756390 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22100054d0 con 0x7f221c074250 2024-04-25T18:52:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.211+0000 7f2216a00640 1 -- 172.21.15.112:0/1604756390 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210010690 con 0x7f221c074250 2024-04-25T18:52:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.211+0000 7f2224800640 1 -- 172.21.15.112:0/1604756390 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f221c1ab800 con 0x7f221c074250 2024-04-25T18:52:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.215+0000 7f2216a00640 1 -- 172.21.15.112:0/1604756390 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f2210004e70 con 0x7f221c074250 2024-04-25T18:52:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.216+0000 7f22277f2740 1 Processor -- start 2024-04-25T18:52:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.217+0000 7f22277f2740 1 -- start start 2024-04-25T18:52:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.217+0000 7f2216a00640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f22040a3e10 0x7f22040a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.217+0000 7f22277f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb35fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.217+0000 7f22277f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 0x558ccb36a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.217+0000 7f22277f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb36cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.217+0000 7f22277f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558ccb2c5900 con 0x558ccb360140 2024-04-25T18:52:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.217+0000 7f22277f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558ccb26bef0 con 0x558ccb36aa00 2024-04-25T18:52:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.217+0000 7f22277f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558ccb2c18c0 con 0x558ccb35f820 2024-04-25T18:52:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.217+0000 7f2216a00640 1 -- 172.21.15.112:0/1604756390 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7f22100d0cc0 con 0x7f221c074250 2024-04-25T18:52:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.217+0000 7f2221e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb35fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.218+0000 7f2221e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb35fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:48652/0 (socket says 172.21.15.112:48652) 2024-04-25T18:52:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.218+0000 7f2221e00640 1 -- 172.21.15.112:0/1219798468 learned_addr learned my addr 172.21.15.112:0/1219798468 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.218+0000 7f2222800640 1 --2- 172.21.15.112:0/1219798468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb36cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.218+0000 7f2221e00640 1 -- 172.21.15.112:0/1219798468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 msgr2=0x558ccb36cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.218+0000 7f2221e00640 1 --2- 172.21.15.112:0/1219798468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb36cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.218+0000 7f2221e00640 1 -- 172.21.15.112:0/1219798468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 msgr2=0x558ccb36a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.218+0000 7f2221e00640 1 --2- 172.21.15.112:0/1219798468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 0x558ccb36a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.218+0000 7f2221e00640 1 -- 172.21.15.112:0/1219798468 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ccafd8210 con 0x558ccb35f820 2024-04-25T18:52:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.218+0000 7f2222800640 1 --2- 172.21.15.112:0/1219798468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb36cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:52:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.218+0000 7f2222800640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f22040a3e10 0x7f22040a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.219+0000 7f2221e00640 1 --2- 172.21.15.112:0/1219798468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb35fc00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2210099630 tx=0x7f22100b00e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c86b133139c8fd3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.219+0000 7f220b000640 1 -- 172.21.15.112:0/1219798468 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22100b1ab0 con 0x558ccb35f820 2024-04-25T18:52:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.219+0000 7f220b000640 1 -- 172.21.15.112:0/1219798468 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2210005680 con 0x558ccb35f820 2024-04-25T18:52:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.219+0000 7f220b000640 1 -- 172.21.15.112:0/1219798468 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22100a58d0 con 0x558ccb35f820 2024-04-25T18:52:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.219+0000 7f2222800640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f22040a3e10 0x7f22040a62d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2218006010 tx=0x7f2218009990 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:40.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:39 smithi161 ceph-mon[32609]: pgmap v162: 97 pgs: 97 active+clean; 47 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 279 op/s 2024-04-25T18:52:40.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:39 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.220+0000 7f22277f2740 1 -- 172.21.15.112:0/1219798468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 msgr2=0x558ccb35fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.220+0000 7f22277f2740 1 --2- 172.21.15.112:0/1219798468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb35fc00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2210099630 tx=0x7f22100b00e0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.220+0000 7f22277f2740 1 -- 172.21.15.112:0/1219798468 shutdown_connections 2024-04-25T18:52:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.220+0000 7f22277f2740 1 --2- 172.21.15.112:0/1219798468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb36cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.220+0000 7f22277f2740 1 --2- 172.21.15.112:0/1219798468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 0x558ccb36a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.220+0000 7f22277f2740 1 --2- 172.21.15.112:0/1219798468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb35fc00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.220+0000 7f22277f2740 1 -- 172.21.15.112:0/1219798468 >> 172.21.15.112:0/1219798468 conn(0x558ccb35bd30 msgr2=0x558ccb35c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.220+0000 7f22277f2740 1 -- 172.21.15.112:0/1219798468 shutdown_connections 2024-04-25T18:52:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.220+0000 7f22277f2740 1 -- 172.21.15.112:0/1219798468 wait complete. 2024-04-25T18:52:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.221+0000 7f22277f2740 1 -- start start 2024-04-25T18:52:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.221+0000 7f22277f2740 1 Processor -- start 2024-04-25T18:52:40.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:39 smithi112 ceph-mon[27864]: pgmap v162: 97 pgs: 97 active+clean; 47 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 279 op/s 2024-04-25T18:52:40.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:39 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f22277f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb3679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f22277f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 0x558ccb367f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f22277f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb368450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f22277f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558ccb36d5d0 con 0x558ccb360140 2024-04-25T18:52:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f22277f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558ccb27f310 con 0x558ccb36aa00 2024-04-25T18:52:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f22277f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558ccb36d750 con 0x558ccb35f820 2024-04-25T18:52:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f22277f2740 10 client.0 _ll_register_callbacks cb 0x558ccb24dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:52:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f2222800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb368450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f2222800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb368450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:52830/0 (socket says 172.21.15.112:52830) 2024-04-25T18:52:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f2222800640 1 -- 172.21.15.112:0/3967848462 learned_addr learned my addr 172.21.15.112:0/3967848462 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f2221e00640 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb3679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.222+0000 7f2221400640 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 0x558ccb367f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.223+0000 7f2222800640 1 -- 172.21.15.112:0/3967848462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 msgr2=0x558ccb3679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.223+0000 7f2222800640 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb3679d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.223+0000 7f2222800640 1 -- 172.21.15.112:0/3967848462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 msgr2=0x558ccb367f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.223+0000 7f2222800640 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 0x558ccb367f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.223+0000 7f2222800640 1 -- 172.21.15.112:0/3967848462 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ccb0a4620 con 0x558ccb36aa00 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.223+0000 7f2222800640 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb368450 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f221802ffc0 tx=0x7f221802a970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb9f26f62cf53afe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.223+0000 7f21ff400640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.223+0000 7f21ff400640 1 -- 172.21.15.112:0/3967848462 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218053060 con 0x558ccb36aa00 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.223+0000 7f21ff400640 1 -- 172.21.15.112:0/3967848462 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7f2218004250 con 0x558ccb36aa00 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.223+0000 7f22277f2740 1 -- 172.21.15.112:0/3967848462 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2210005680 con 0x558ccb36aa00 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.224+0000 7f21ff400640 1 -- 172.21.15.112:0/3967848462 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2218004450 con 0x558ccb36aa00 2024-04-25T18:52:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.224+0000 7f22277f2740 10 client.25712 fetch_fsmap learned FSMap version 33 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.224+0000 7f22277f2740 1 -- 172.21.15.112:0/3967848462 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558ccb3797f0 con 0x558ccb36aa00 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.224+0000 7f21ff400640 1 -- 172.21.15.112:0/3967848462 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2210005680 con 0x558ccb36aa00 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.224+0000 7f21ff400640 1 -- 172.21.15.112:0/3967848462 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7f221804e440 con 0x558ccb36aa00 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.224+0000 7f22277f2740 10 client.25712 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.224+0000 7f22277f2740 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.224+0000 7f22277f2740 20 client.25712 populate_metadata read hostname 'smithi112' 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.225+0000 7f22277f2740 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558ccb37ad00 0x558ccb37d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.225+0000 7f22277f2740 4 client.25712 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.225+0000 7f22277f2740 1 -- 172.21.15.112:0/3967848462 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ccb2d98b0 con 0x558ccb37ad00 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.225+0000 7f2221e00640 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558ccb37ad00 0x558ccb37d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.225+0000 7f21ff400640 10 client.25712 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.225+0000 7f2221e00640 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558ccb37ad00 0x558ccb37d120 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94685ab24cd37e8 server_cookie=7cd7e56090cab958 in_seq=0 out_seq=0 2024-04-25T18:52:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.226+0000 7f21ff400640 1 -- 172.21.15.112:0/3967848462 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ccb2d98b0 con 0x558ccb37ad00 2024-04-25T18:52:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.226+0000 7f21ff400640 10 client.25712 handle_command_reply: tid=0 2024-04-25T18:52:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.226+0000 7f22277f2740 1 client.25712 shutdown 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.226+0000 7f22277f2740 1 -- 172.21.15.112:0/3967848462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 msgr2=0x558ccb368450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.227+0000 7f22277f2740 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb368450 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f221802ffc0 tx=0x7f221802a970 comp rx=0 tx=0).stop 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.227+0000 7f22277f2740 1 -- 172.21.15.112:0/3967848462 shutdown_connections 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.227+0000 7f22277f2740 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb368450 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.227+0000 7f22277f2740 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558ccb37ad00 0x558ccb37d120 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.227+0000 7f22277f2740 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 0x558ccb367f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.227+0000 7f22277f2740 1 --2- 172.21.15.112:0/3967848462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb3679d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.227+0000 7f22277f2740 1 -- 172.21.15.112:0/3967848462 >> 172.21.15.112:0/3967848462 conn(0x558ccb35bd30 msgr2=0x558ccb361ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.227+0000 7f21ff400640 0 client.25712 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.227+0000 7f22277f2740 1 -- 172.21.15.112:0/3967848462 shutdown_connections 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.228+0000 7f22277f2740 1 -- 172.21.15.112:0/3967848462 wait complete. 2024-04-25T18:52:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.228+0000 7f22277f2740 20 client.25712 trim_cache size 0 max 16384 2024-04-25T18:52:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.260+0000 7f22277f2740 1 Processor -- start 2024-04-25T18:52:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.260+0000 7f22277f2740 1 -- start start 2024-04-25T18:52:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.261+0000 7f22277f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb35f820 0x558ccb376490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.261+0000 7f22277f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb360140 0x558ccb3621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.261+0000 7f22277f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb36aa00 0x558ccb382a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.261+0000 7f22277f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1b6bb0 con 0x558ccb35f820 2024-04-25T18:52:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.261+0000 7f22277f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1b4450 con 0x558ccb360140 2024-04-25T18:52:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.261+0000 7f22277f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1b22a0 con 0x558ccb36aa00 2024-04-25T18:52:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.261+0000 7f2221400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb360140 0x558ccb3621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.261+0000 7f2221e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb35f820 0x558ccb376490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.261+0000 7f2221400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb360140 0x558ccb3621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:52844/0 (socket says 172.21.15.112:52844) 2024-04-25T18:52:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.261+0000 7f2221400640 1 -- 172.21.15.112:0/2087574330 learned_addr learned my addr 172.21.15.112:0/2087574330 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.261+0000 7f2222800640 1 --2- 172.21.15.112:0/2087574330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb36aa00 0x558ccb382a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.262+0000 7f2222800640 1 -- 172.21.15.112:0/2087574330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb360140 msgr2=0x558ccb3621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.262+0000 7f2222800640 1 --2- 172.21.15.112:0/2087574330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb360140 0x558ccb3621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.262+0000 7f2222800640 1 -- 172.21.15.112:0/2087574330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb35f820 msgr2=0x558ccb376490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.262+0000 7f2222800640 1 --2- 172.21.15.112:0/2087574330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb35f820 0x558ccb376490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.262+0000 7f2222800640 1 -- 172.21.15.112:0/2087574330 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ccb365f20 con 0x558ccb36aa00 2024-04-25T18:52:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.262+0000 7f2221400640 1 --2- 172.21.15.112:0/2087574330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb360140 0x558ccb3621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:52:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.262+0000 7f2221e00640 1 --2- 172.21.15.112:0/2087574330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb35f820 0x558ccb376490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:52:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.262+0000 7f2222800640 1 --2- 172.21.15.112:0/2087574330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb36aa00 0x558ccb382a30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f221802fd10 tx=0x7f22180095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3284b69983ce0bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.262+0000 7f220a600640 1 -- 172.21.15.112:0/2087574330 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221804ead0 con 0x558ccb36aa00 2024-04-25T18:52:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.262+0000 7f220a600640 1 -- 172.21.15.112:0/2087574330 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221804ec70 con 0x558ccb36aa00 2024-04-25T18:52:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.262+0000 7f220a600640 1 -- 172.21.15.112:0/2087574330 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218030d40 con 0x558ccb36aa00 2024-04-25T18:52:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.264+0000 7f22277f2740 1 -- 172.21.15.112:0/2087574330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb36aa00 msgr2=0x558ccb382a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.264+0000 7f22277f2740 1 --2- 172.21.15.112:0/2087574330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb36aa00 0x558ccb382a30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f221802fd10 tx=0x7f22180095c0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.264+0000 7f22277f2740 1 -- 172.21.15.112:0/2087574330 shutdown_connections 2024-04-25T18:52:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.264+0000 7f22277f2740 1 --2- 172.21.15.112:0/2087574330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb36aa00 0x558ccb382a30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.264+0000 7f22277f2740 1 --2- 172.21.15.112:0/2087574330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb360140 0x558ccb3621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.264+0000 7f22277f2740 1 --2- 172.21.15.112:0/2087574330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb35f820 0x558ccb376490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.264+0000 7f22277f2740 1 -- 172.21.15.112:0/2087574330 >> 172.21.15.112:0/2087574330 conn(0x558ccb35bd30 msgr2=0x558ccb367990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.264+0000 7f22277f2740 1 -- 172.21.15.112:0/2087574330 shutdown_connections 2024-04-25T18:52:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.264+0000 7f22277f2740 1 -- 172.21.15.112:0/2087574330 wait complete. 2024-04-25T18:52:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.264+0000 7f22277f2740 1 -- start start 2024-04-25T18:52:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f22277f2740 1 Processor -- start 2024-04-25T18:52:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f22277f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb372b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f22277f2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 0x558ccb36ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f22277f2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb370490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f22277f2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1b2910 con 0x558ccb360140 2024-04-25T18:52:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f22277f2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1b3d80 con 0x558ccb36aa00 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f22277f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1b2fe0 con 0x558ccb35f820 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f22277f2740 10 client.0 _ll_register_callbacks cb 0x7f2218009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f2221e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb372b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f2221e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb372b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:48674/0 (socket says 172.21.15.112:48674) 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f2222800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb370490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.265+0000 7f2221e00640 1 -- 172.21.15.112:0/3537455565 learned_addr learned my addr 172.21.15.112:0/3537455565 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.266+0000 7f2221400640 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 0x558ccb36ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.266+0000 7f2222800640 1 -- 172.21.15.112:0/3537455565 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 msgr2=0x558ccb372b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.266+0000 7f2222800640 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb372b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.266+0000 7f2222800640 1 -- 172.21.15.112:0/3537455565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 msgr2=0x558ccb36ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.266+0000 7f2222800640 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 0x558ccb36ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.266+0000 7f2222800640 1 -- 172.21.15.112:0/3537455565 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ccb370a80 con 0x558ccb36aa00 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.266+0000 7f2221e00640 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb372b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:52:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.266+0000 7f2222800640 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb370490 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2218051040 tx=0x7f221802f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82f04dd0d14a819f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.266+0000 7f21ff400640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.266+0000 7f21ff400640 1 -- 172.21.15.112:0/3537455565 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218053040 con 0x558ccb36aa00 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f21ff400640 1 -- 172.21.15.112:0/3537455565 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7f2218004250 con 0x558ccb36aa00 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f22277f2740 1 -- 172.21.15.112:0/3537455565 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221804ec70 con 0x558ccb36aa00 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f21ff400640 1 -- 172.21.15.112:0/3537455565 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2218070050 con 0x558ccb36aa00 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f22277f2740 10 client.25718 fetch_fsmap learned FSMap version 33 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f22277f2740 1 -- 172.21.15.112:0/3537455565 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558ccb3e10c0 con 0x558ccb36aa00 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f21ff400640 1 -- 172.21.15.112:0/3537455565 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221804ec70 con 0x558ccb36aa00 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f21ff400640 1 -- 172.21.15.112:0/3537455565 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7f221805f070 con 0x558ccb36aa00 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f22277f2740 10 client.25718 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f22277f2740 10 client.25718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f22277f2740 20 client.25718 populate_metadata read hostname 'smithi112' 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f22277f2740 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558ccb3e1750 0x558ccb3e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f22277f2740 4 client.25718 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:52:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.267+0000 7f22277f2740 1 -- 172.21.15.112:0/3537455565 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ccb2d3320 con 0x558ccb3e1750 2024-04-25T18:52:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.268+0000 7f2221e00640 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558ccb3e1750 0x558ccb3e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.269+0000 7f21ff400640 10 client.25718 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.269+0000 7f2221e00640 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558ccb3e1750 0x558ccb3e3b70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36587611f9fe123f server_cookie=532b4c8332ac82c6 in_seq=0 out_seq=0 2024-04-25T18:52:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.283+0000 7f21ff400640 1 -- 172.21.15.112:0/3537455565 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ccb2d3320 con 0x558ccb3e1750 2024-04-25T18:52:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.283+0000 7f21ff400640 10 client.25718 handle_command_reply: tid=0 2024-04-25T18:52:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.283+0000 7f22277f2740 1 client.25718 shutdown 2024-04-25T18:52:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.283+0000 7f22277f2740 1 -- 172.21.15.112:0/3537455565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 msgr2=0x558ccb370490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.283+0000 7f22277f2740 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb370490 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2218051040 tx=0x7f221802f810 comp rx=0 tx=0).stop 2024-04-25T18:52:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.283+0000 7f22277f2740 1 -- 172.21.15.112:0/3537455565 shutdown_connections 2024-04-25T18:52:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.283+0000 7f22277f2740 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ccb36aa00 0x558ccb370490 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.283+0000 7f22277f2740 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558ccb3e1750 0x558ccb3e3b70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.283+0000 7f22277f2740 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ccb360140 0x558ccb36ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.283+0000 7f22277f2740 1 --2- 172.21.15.112:0/3537455565 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558ccb35f820 0x558ccb372b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.283+0000 7f22277f2740 1 -- 172.21.15.112:0/3537455565 >> 172.21.15.112:0/3537455565 conn(0x558ccb35bd30 msgr2=0x558ccb368220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.284+0000 7f21ff400640 0 client.25718 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.284+0000 7f22277f2740 1 -- 172.21.15.112:0/3537455565 shutdown_connections 2024-04-25T18:52:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.284+0000 7f22277f2740 1 -- 172.21.15.112:0/3537455565 wait complete. 2024-04-25T18:52:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.284+0000 7f22277f2740 20 client.25718 trim_cache size 0 max 16384 2024-04-25T18:52:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:52:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:52:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:52:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:52:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 -- 172.21.15.112:0/1604756390 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f22040a3e10 msgr2=0x7f22040a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f22040a3e10 0x7f22040a62d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2218006010 tx=0x7f2218009990 comp rx=0 tx=0).stop 2024-04-25T18:52:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 -- 172.21.15.112:0/1604756390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c074250 msgr2=0x7f221c1a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c074250 0x7f221c1a4940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f221000e9f0 tx=0x7f221000edf0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 -- 172.21.15.112:0/1604756390 shutdown_connections 2024-04-25T18:52:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f22040a3e10 0x7f22040a62d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f221c113010 0x7f221c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f221c074b70 0x7f221c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 --2- 172.21.15.112:0/1604756390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c074250 0x7f221c1a4940 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 -- 172.21.15.112:0/1604756390 >> 172.21.15.112:0/1604756390 conn(0x7f221c0793e0 msgr2=0x7f221c0fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 -- 172.21.15.112:0/1604756390 shutdown_connections 2024-04-25T18:52:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:40.285+0000 7f2224800640 1 -- 172.21.15.112:0/1604756390 wait complete. 2024-04-25T18:52:41.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:40 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:41.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:40 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:41.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:40 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:52:41.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:40 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:41.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:40 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:41.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:40 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:52:41.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:40 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:41.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:40 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:41.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:40 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:52:42.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:41 smithi171 ceph-mon[33266]: pgmap v163: 97 pgs: 97 active+clean; 41 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.4 MiB/s wr, 431 op/s 2024-04-25T18:52:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:41 smithi161 ceph-mon[32609]: pgmap v163: 97 pgs: 97 active+clean; 41 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.4 MiB/s wr, 431 op/s 2024-04-25T18:52:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:41 smithi112 ceph-mon[27864]: pgmap v163: 97 pgs: 97 active+clean; 41 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.4 MiB/s wr, 431 op/s 2024-04-25T18:52:44.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:43 smithi171 ceph-mon[33266]: pgmap v164: 97 pgs: 97 active+clean; 41 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 283 op/s 2024-04-25T18:52:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:43 smithi161 ceph-mon[32609]: pgmap v164: 97 pgs: 97 active+clean; 41 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 283 op/s 2024-04-25T18:52:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:43 smithi112 ceph-mon[27864]: pgmap v164: 97 pgs: 97 active+clean; 41 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 283 op/s 2024-04-25T18:52:46.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:45 smithi171 ceph-mon[33266]: pgmap v165: 97 pgs: 97 active+clean; 41 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 283 op/s 2024-04-25T18:52:46.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:45 smithi161 ceph-mon[32609]: pgmap v165: 97 pgs: 97 active+clean; 41 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 283 op/s 2024-04-25T18:52:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:45 smithi112 ceph-mon[27864]: pgmap v165: 97 pgs: 97 active+clean; 41 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 283 op/s 2024-04-25T18:52:48.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:47 smithi171 ceph-mon[33266]: pgmap v166: 97 pgs: 97 active+clean; 43 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.4 MiB/s wr, 406 op/s 2024-04-25T18:52:48.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:47 smithi161 ceph-mon[32609]: pgmap v166: 97 pgs: 97 active+clean; 43 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.4 MiB/s wr, 406 op/s 2024-04-25T18:52:48.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:47 smithi112 ceph-mon[27864]: pgmap v166: 97 pgs: 97 active+clean; 43 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.4 MiB/s wr, 406 op/s 2024-04-25T18:52:50.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:49 smithi171 ceph-mon[33266]: pgmap v167: 97 pgs: 97 active+clean; 43 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 274 op/s 2024-04-25T18:52:50.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:49 smithi161 ceph-mon[32609]: pgmap v167: 97 pgs: 97 active+clean; 43 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 274 op/s 2024-04-25T18:52:50.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:49 smithi112 ceph-mon[27864]: pgmap v167: 97 pgs: 97 active+clean; 43 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 274 op/s 2024-04-25T18:52:52.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:51 smithi171 ceph-mon[33266]: pgmap v168: 97 pgs: 97 active+clean; 50 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.3 MiB/s wr, 394 op/s 2024-04-25T18:52:52.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:51 smithi161 ceph-mon[32609]: pgmap v168: 97 pgs: 97 active+clean; 50 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.3 MiB/s wr, 394 op/s 2024-04-25T18:52:52.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:51 smithi112 ceph-mon[27864]: pgmap v168: 97 pgs: 97 active+clean; 50 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.3 MiB/s wr, 394 op/s 2024-04-25T18:52:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.443+0000 7ff58b200640 1 -- 172.21.15.112:0/860631803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff58c103830 msgr2=0x7ff58c10bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.443+0000 7ff58b200640 1 --2- 172.21.15.112:0/860631803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff58c103830 0x7ff58c10bd10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff574009900 tx=0x7ff57402a480 comp rx=0 tx=0).stop 2024-04-25T18:52:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.444+0000 7ff58b200640 1 -- 172.21.15.112:0/860631803 shutdown_connections 2024-04-25T18:52:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.444+0000 7ff58b200640 1 --2- 172.21.15.112:0/860631803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58c10c330 0x7ff58c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.444+0000 7ff58b200640 1 --2- 172.21.15.112:0/860631803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff58c103830 0x7ff58c10bd10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.444+0000 7ff58b200640 1 --2- 172.21.15.112:0/860631803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c102f10 0x7ff58c1032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.444+0000 7ff58b200640 1 -- 172.21.15.112:0/860631803 >> 172.21.15.112:0/860631803 conn(0x7ff58c0fc770 msgr2=0x7ff58c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.444+0000 7ff58b200640 1 -- 172.21.15.112:0/860631803 shutdown_connections 2024-04-25T18:52:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.444+0000 7ff58b200640 1 -- 172.21.15.112:0/860631803 wait complete. 2024-04-25T18:52:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.445+0000 7ff58b200640 1 Processor -- start 2024-04-25T18:52:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.445+0000 7ff58b200640 1 -- start start 2024-04-25T18:52:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.446+0000 7ff58b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c102f10 0x7ff58c1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.446+0000 7ff58b200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58c103830 0x7ff58c1a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.446+0000 7ff58b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff58c10c330 0x7ff58c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.446+0000 7ff58b200640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c111ff0 con 0x7ff58c103830 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.446+0000 7ff58b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c111e70 con 0x7ff58c102f10 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.446+0000 7ff58b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c112170 con 0x7ff58c10c330 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.446+0000 7ff589e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c102f10 0x7ff58c1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.446+0000 7ff589e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c102f10 0x7ff58c1a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:60458/0 (socket says 172.21.15.112:60458) 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.446+0000 7ff589e00640 1 -- 172.21.15.112:0/1968728892 learned_addr learned my addr 172.21.15.112:0/1968728892 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.446+0000 7ff58a800640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff58c10c330 0x7ff58c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.446+0000 7ff581400640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58c103830 0x7ff58c1a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.447+0000 7ff58a800640 1 -- 172.21.15.112:0/1968728892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c102f10 msgr2=0x7ff58c1a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.447+0000 7ff58a800640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c102f10 0x7ff58c1a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.447+0000 7ff58a800640 1 -- 172.21.15.112:0/1968728892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58c103830 msgr2=0x7ff58c1a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.447+0000 7ff58a800640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58c103830 0x7ff58c1a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.447+0000 7ff58a800640 1 -- 172.21.15.112:0/1968728892 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c19ad90 con 0x7ff58c10c330 2024-04-25T18:52:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.447+0000 7ff581400640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58c103830 0x7ff58c1a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:52:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.447+0000 7ff58a800640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff58c10c330 0x7ff58c19a610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff58400d6b0 tx=0x7ff58400db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.447+0000 7ff583400640 1 -- 172.21.15.112:0/1968728892 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584017070 con 0x7ff58c10c330 2024-04-25T18:52:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.447+0000 7ff583400640 1 -- 172.21.15.112:0/1968728892 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff584009cd0 con 0x7ff58c10c330 2024-04-25T18:52:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.447+0000 7ff583400640 1 -- 172.21.15.112:0/1968728892 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584003cc0 con 0x7ff58c10c330 2024-04-25T18:52:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.447+0000 7ff58b200640 1 -- 172.21.15.112:0/1968728892 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c19b080 con 0x7ff58c10c330 2024-04-25T18:52:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.448+0000 7ff58b200640 1 -- 172.21.15.112:0/1968728892 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c1a72d0 con 0x7ff58c10c330 2024-04-25T18:52:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.451+0000 7ff593963740 1 Processor -- start 2024-04-25T18:52:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.451+0000 7ff583400640 1 -- 172.21.15.112:0/1968728892 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7ff584009820 con 0x7ff58c10c330 2024-04-25T18:52:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.451+0000 7ff593963740 1 -- start start 2024-04-25T18:52:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff593963740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8aa59b0 0x55aed8aa5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff593963740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8ab0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff593963740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 0x55aed8ab2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff593963740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aed8a0b900 con 0x55aed8ab0b90 2024-04-25T18:52:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff593963740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aed89b1ef0 con 0x55aed8aa59b0 2024-04-25T18:52:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff593963740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55aed8a078c0 con 0x55aed8aa62d0 2024-04-25T18:52:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff589e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8ab0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff589e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8ab0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:45746/0 (socket says 172.21.15.112:45746) 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff589e00640 1 -- 172.21.15.112:0/463364670 learned_addr learned my addr 172.21.15.112:0/463364670 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff581400640 1 --2- 172.21.15.112:0/463364670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8aa59b0 0x55aed8aa5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff581400640 1 --2- 172.21.15.112:0/463364670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 0x55aed8ab2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff589e00640 1 -- 172.21.15.112:0/463364670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8aa59b0 msgr2=0x55aed8aa5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff589e00640 1 --2- 172.21.15.112:0/463364670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8aa59b0 0x55aed8aa5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff589e00640 1 -- 172.21.15.112:0/463364670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 msgr2=0x55aed8ab2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff589e00640 1 --2- 172.21.15.112:0/463364670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 0x55aed8ab2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff589e00640 1 -- 172.21.15.112:0/463364670 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aed871e210 con 0x55aed8aa62d0 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.452+0000 7ff581400640 1 --2- 172.21.15.112:0/463364670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 0x55aed8ab2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.453+0000 7ff589e00640 1 --2- 172.21.15.112:0/463364670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8ab0650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0115c0 tx=0x7ff57c011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c7a7b7b0898537a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.453+0000 7ff583400640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff5700a3d90 0x7ff5700a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.453+0000 7ff57b000640 1 -- 172.21.15.112:0/463364670 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00f6c0 con 0x55aed8aa62d0 2024-04-25T18:52:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.453+0000 7ff57b000640 1 -- 172.21.15.112:0/463364670 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff57c00fd10 con 0x55aed8aa62d0 2024-04-25T18:52:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.453+0000 7ff57b000640 1 -- 172.21.15.112:0/463364670 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c019e30 con 0x55aed8aa62d0 2024-04-25T18:52:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.453+0000 7ff583400640 1 -- 172.21.15.112:0/1968728892 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7ff5840cffd0 con 0x7ff58c10c330 2024-04-25T18:52:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.454+0000 7ff581400640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff5700a3d90 0x7ff5700a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.454+0000 7ff593963740 1 -- 172.21.15.112:0/463364670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 msgr2=0x55aed8ab0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.454+0000 7ff593963740 1 --2- 172.21.15.112:0/463364670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8ab0650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0115c0 tx=0x7ff57c011a80 comp rx=0 tx=0).stop 2024-04-25T18:52:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.454+0000 7ff593963740 1 -- 172.21.15.112:0/463364670 shutdown_connections 2024-04-25T18:52:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.454+0000 7ff593963740 1 --2- 172.21.15.112:0/463364670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 0x55aed8ab2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.454+0000 7ff593963740 1 --2- 172.21.15.112:0/463364670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8ab0650 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.454+0000 7ff593963740 1 --2- 172.21.15.112:0/463364670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8aa59b0 0x55aed8aa5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.454+0000 7ff593963740 1 -- 172.21.15.112:0/463364670 >> 172.21.15.112:0/463364670 conn(0x55aed8aa1ec0 msgr2=0x55aed8aa22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.455+0000 7ff593963740 1 -- 172.21.15.112:0/463364670 shutdown_connections 2024-04-25T18:52:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.455+0000 7ff593963740 1 -- 172.21.15.112:0/463364670 wait complete. 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.455+0000 7ff581400640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff5700a3d90 0x7ff5700a6250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff574000c00 tx=0x7ff574031400 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.456+0000 7ff593963740 1 -- start start 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.456+0000 7ff593963740 1 Processor -- start 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.456+0000 7ff593963740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa59b0 0x55aed8aadba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.456+0000 7ff593963740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8aa62d0 0x55aed8aae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.456+0000 7ff593963740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8aae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.456+0000 7ff593963740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55aed8ab37c0 con 0x55aed8aa62d0 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.456+0000 7ff593963740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aed89c5310 con 0x55aed8ab0b90 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.456+0000 7ff593963740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55aed8ab3940 con 0x55aed8aa59b0 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.456+0000 7ff593963740 10 client.0 _ll_register_callbacks cb 0x55aed8993cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff58a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8aae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff589e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa59b0 0x55aed8aadba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff589e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa59b0 0x55aed8aadba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:45752/0 (socket says 172.21.15.112:45752) 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff58a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8aae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:60468/0 (socket says 172.21.15.112:60468) 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff58a800640 1 -- 172.21.15.112:0/4275894721 learned_addr learned my addr 172.21.15.112:0/4275894721 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff581400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8aa62d0 0x55aed8aae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff58a800640 1 -- 172.21.15.112:0/4275894721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa59b0 msgr2=0x55aed8aadba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff58a800640 1 --2- 172.21.15.112:0/4275894721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa59b0 0x55aed8aadba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff58a800640 1 -- 172.21.15.112:0/4275894721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8aa62d0 msgr2=0x55aed8aae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff58a800640 1 --2- 172.21.15.112:0/4275894721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8aa62d0 0x55aed8aae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff58a800640 1 -- 172.21.15.112:0/4275894721 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aed87ea620 con 0x55aed8ab0b90 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff58a800640 1 --2- 172.21.15.112:0/4275894721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8aae620 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff5840988e0 tx=0x7ff58409b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7ddde490ee55c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff56f400640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.457+0000 7ff56f400640 1 -- 172.21.15.112:0/4275894721 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584020040 con 0x55aed8ab0b90 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff56f400640 1 -- 172.21.15.112:0/4275894721 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7ff584098580 con 0x55aed8ab0b90 2024-04-25T18:52:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff593963740 1 -- 172.21.15.112:0/4275894721 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff57c00fd10 con 0x55aed8ab0b90 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff56f400640 1 -- 172.21.15.112:0/4275894721 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff58409c610 con 0x55aed8ab0b90 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff593963740 10 client.25727 fetch_fsmap learned FSMap version 33 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff593963740 1 -- 172.21.15.112:0/4275894721 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aed8abfa20 con 0x55aed8ab0b90 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff56f400640 1 -- 172.21.15.112:0/4275894721 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff57c00fd10 con 0x55aed8ab0b90 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff56f400640 1 -- 172.21.15.112:0/4275894721 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7ff5840b5200 con 0x55aed8ab0b90 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff593963740 10 client.25727 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff593963740 10 client.25727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff593963740 20 client.25727 populate_metadata read hostname 'smithi112' 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff593963740 1 --2- 172.21.15.112:0/4275894721 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aed8ac0ee0 0x55aed8ac32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff593963740 4 client.25727 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.458+0000 7ff593963740 1 -- 172.21.15.112:0/4275894721 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aed8a1f8b0 con 0x55aed8ac0ee0 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.459+0000 7ff589e00640 1 --2- 172.21.15.112:0/4275894721 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aed8ac0ee0 0x55aed8ac32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.459+0000 7ff589e00640 1 --2- 172.21.15.112:0/4275894721 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aed8ac0ee0 0x55aed8ac32f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b7d3da468c05315 server_cookie=e9080a50799dedf0 in_seq=0 out_seq=0 2024-04-25T18:52:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.459+0000 7ff56f400640 10 client.25727 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff56f400640 1 -- 172.21.15.112:0/4275894721 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aed8a1f8b0 con 0x55aed8ac0ee0 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff56f400640 10 client.25727 handle_command_reply: tid=0 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 1 client.25727 shutdown 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 1 -- 172.21.15.112:0/4275894721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 msgr2=0x55aed8aae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 1 --2- 172.21.15.112:0/4275894721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8aae620 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff5840988e0 tx=0x7ff58409b870 comp rx=0 tx=0).stop 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 1 -- 172.21.15.112:0/4275894721 shutdown_connections 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 1 --2- 172.21.15.112:0/4275894721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8aae620 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 1 --2- 172.21.15.112:0/4275894721 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aed8ac0ee0 0x55aed8ac32f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 1 --2- 172.21.15.112:0/4275894721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8aa62d0 0x55aed8aae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 1 --2- 172.21.15.112:0/4275894721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa59b0 0x55aed8aadba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff56f400640 0 client.25727 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 1 -- 172.21.15.112:0/4275894721 >> 172.21.15.112:0/4275894721 conn(0x55aed8aa1ec0 msgr2=0x55aed8aa7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 1 -- 172.21.15.112:0/4275894721 shutdown_connections 2024-04-25T18:52:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 1 -- 172.21.15.112:0/4275894721 wait complete. 2024-04-25T18:52:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.460+0000 7ff593963740 20 client.25727 trim_cache size 0 max 16384 2024-04-25T18:52:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.478+0000 7ff593963740 1 Processor -- start 2024-04-25T18:52:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.478+0000 7ff593963740 1 -- start start 2024-04-25T18:52:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.478+0000 7ff593963740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8aa59b0 0x55aed8abc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.478+0000 7ff593963740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8aa17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.478+0000 7ff593963740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 0x55aed8ac8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.478+0000 7ff593963740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b2640 con 0x55aed8ab0b90 2024-04-25T18:52:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.478+0000 7ff593963740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1afee0 con 0x55aed8aa59b0 2024-04-25T18:52:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.478+0000 7ff593963740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1add30 con 0x55aed8aa62d0 2024-04-25T18:52:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.478+0000 7ff581400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8aa17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff589e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8aa59b0 0x55aed8abc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff581400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8aa17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:45768/0 (socket says 172.21.15.112:45768) 2024-04-25T18:52:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff581400640 1 -- 172.21.15.112:0/2193843891 learned_addr learned my addr 172.21.15.112:0/2193843891 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff58a800640 1 --2- 172.21.15.112:0/2193843891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 0x55aed8ac8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff589e00640 1 -- 172.21.15.112:0/2193843891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 msgr2=0x55aed8aa17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff589e00640 1 --2- 172.21.15.112:0/2193843891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8aa17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff589e00640 1 -- 172.21.15.112:0/2193843891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 msgr2=0x55aed8ac8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff589e00640 1 --2- 172.21.15.112:0/2193843891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 0x55aed8ac8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff589e00640 1 -- 172.21.15.112:0/2193843891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aed8aa1ce0 con 0x55aed8aa59b0 2024-04-25T18:52:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff581400640 1 --2- 172.21.15.112:0/2193843891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8aa17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:52:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff58a800640 1 --2- 172.21.15.112:0/2193843891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 0x55aed8ac8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:52:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff589e00640 1 --2- 172.21.15.112:0/2193843891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8aa59b0 0x55aed8abc680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0199b0 tx=0x7ff57c005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18c66375ae3e7d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.479+0000 7ff57a600640 1 -- 172.21.15.112:0/2193843891 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c030070 con 0x55aed8aa59b0 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.480+0000 7ff57a600640 1 -- 172.21.15.112:0/2193843891 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff57c019dc0 con 0x55aed8aa59b0 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.480+0000 7ff57a600640 1 -- 172.21.15.112:0/2193843891 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00f6c0 con 0x55aed8aa59b0 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.481+0000 7ff593963740 1 -- 172.21.15.112:0/2193843891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8aa59b0 msgr2=0x55aed8abc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.481+0000 7ff593963740 1 --2- 172.21.15.112:0/2193843891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8aa59b0 0x55aed8abc680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0199b0 tx=0x7ff57c005040 comp rx=0 tx=0).stop 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.481+0000 7ff593963740 1 -- 172.21.15.112:0/2193843891 shutdown_connections 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.481+0000 7ff593963740 1 --2- 172.21.15.112:0/2193843891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8ab0b90 0x55aed8ac8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.481+0000 7ff593963740 1 --2- 172.21.15.112:0/2193843891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8aa17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.481+0000 7ff593963740 1 --2- 172.21.15.112:0/2193843891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8aa59b0 0x55aed8abc680 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.481+0000 7ff593963740 1 -- 172.21.15.112:0/2193843891 >> 172.21.15.112:0/2193843891 conn(0x55aed8aa1ec0 msgr2=0x55aed8aa8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.481+0000 7ff593963740 1 -- 172.21.15.112:0/2193843891 shutdown_connections 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.481+0000 7ff593963740 1 -- 172.21.15.112:0/2193843891 wait complete. 2024-04-25T18:52:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.481+0000 7ff593963740 1 -- start start 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.482+0000 7ff593963740 1 Processor -- start 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.482+0000 7ff593963740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8aa59b0 0x55aed8ab8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.482+0000 7ff593963740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8ab8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.482+0000 7ff593963740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8ab5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.482+0000 7ff593963740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1ae3a0 con 0x55aed8aa59b0 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.482+0000 7ff593963740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1af810 con 0x55aed8ab0b90 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.482+0000 7ff593963740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1aea70 con 0x55aed8aa62d0 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.482+0000 7ff593963740 10 client.0 _ll_register_callbacks cb 0x7ff58409bce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff58a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8ab5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff581400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8ab8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff58a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8ab5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:60476/0 (socket says 172.21.15.112:60476) 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff58a800640 1 -- 172.21.15.112:0/1423551389 learned_addr learned my addr 172.21.15.112:0/1423551389 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff581400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8ab8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:45776/0 (socket says 172.21.15.112:45776) 2024-04-25T18:52:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff589e00640 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8aa59b0 0x55aed8ab8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff58a800640 1 -- 172.21.15.112:0/1423551389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 msgr2=0x55aed8ab8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff58a800640 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8ab8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff58a800640 1 -- 172.21.15.112:0/1423551389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8aa59b0 msgr2=0x55aed8ab8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff58a800640 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8aa59b0 0x55aed8ab8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff58a800640 1 -- 172.21.15.112:0/1423551389 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aed8ab6610 con 0x55aed8ab0b90 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff58a800640 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8ab5f70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff5840b7050 tx=0x7ff584003760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f393cac3097e315a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff56f400640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff56f400640 1 -- 172.21.15.112:0/1423551389 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58407e070 con 0x55aed8ab0b90 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff593963740 1 -- 172.21.15.112:0/1423551389 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff57c019dc0 con 0x55aed8ab0b90 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.483+0000 7ff56f400640 1 -- 172.21.15.112:0/1423551389 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8527+0+0 (secure 0 0 0) 0x7ff58409b360 con 0x55aed8ab0b90 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff56f400640 1 -- 172.21.15.112:0/1423551389 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff584014070 con 0x55aed8ab0b90 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff593963740 10 client.25733 fetch_fsmap learned FSMap version 33 2024-04-25T18:52:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff593963740 1 -- 172.21.15.112:0/1423551389 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aed8b26f00 con 0x55aed8ab0b90 2024-04-25T18:52:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff56f400640 1 -- 172.21.15.112:0/1423551389 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff57c019dc0 con 0x55aed8ab0b90 2024-04-25T18:52:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff56f400640 1 -- 172.21.15.112:0/1423551389 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7ff584081020 con 0x55aed8ab0b90 2024-04-25T18:52:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff593963740 10 client.25733 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T18:52:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff593963740 10 client.25733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:52:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff593963740 20 client.25733 populate_metadata read hostname 'smithi112' 2024-04-25T18:52:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff593963740 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aed8b276d0 0x55aed8b29af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff593963740 4 client.25733 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:52:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff593963740 1 -- 172.21.15.112:0/1423551389 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aed8a19320 con 0x55aed8b276d0 2024-04-25T18:52:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.484+0000 7ff589e00640 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aed8b276d0 0x55aed8b29af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.485+0000 7ff589e00640 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aed8b276d0 0x55aed8b29af0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e727f9bcc65d653b server_cookie=4250f6d219b61953 in_seq=0 out_seq=0 2024-04-25T18:52:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.485+0000 7ff56f400640 10 client.25733 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.499+0000 7ff56f400640 1 -- 172.21.15.112:0/1423551389 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aed8a19320 con 0x55aed8b276d0 2024-04-25T18:52:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.499+0000 7ff56f400640 10 client.25733 handle_command_reply: tid=0 2024-04-25T18:52:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.499+0000 7ff593963740 1 client.25733 shutdown 2024-04-25T18:52:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff593963740 1 -- 172.21.15.112:0/1423551389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 msgr2=0x55aed8ab5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff593963740 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8ab5f70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff5840b7050 tx=0x7ff584003760 comp rx=0 tx=0).stop 2024-04-25T18:52:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff593963740 1 -- 172.21.15.112:0/1423551389 shutdown_connections 2024-04-25T18:52:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff593963740 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aed8ab0b90 0x55aed8ab5f70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff593963740 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55aed8aa62d0 0x55aed8ab8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff593963740 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55aed8b276d0 0x55aed8b29af0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff593963740 1 --2- 172.21.15.112:0/1423551389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55aed8aa59b0 0x55aed8ab8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff56f400640 0 client.25733 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:52:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff593963740 1 -- 172.21.15.112:0/1423551389 >> 172.21.15.112:0/1423551389 conn(0x55aed8aa1ec0 msgr2=0x55aed8aa7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff593963740 1 -- 172.21.15.112:0/1423551389 shutdown_connections 2024-04-25T18:52:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff593963740 1 -- 172.21.15.112:0/1423551389 wait complete. 2024-04-25T18:52:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.500+0000 7ff593963740 20 client.25733 trim_cache size 0 max 16384 2024-04-25T18:52:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:52:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:52:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:52:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:52:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.501+0000 7ff58b200640 1 -- 172.21.15.112:0/1968728892 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff5700a3d90 msgr2=0x7ff5700a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.501+0000 7ff58b200640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff5700a3d90 0x7ff5700a6250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff574000c00 tx=0x7ff574031400 comp rx=0 tx=0).stop 2024-04-25T18:52:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.501+0000 7ff58b200640 1 -- 172.21.15.112:0/1968728892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff58c10c330 msgr2=0x7ff58c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.501+0000 7ff58b200640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff58c10c330 0x7ff58c19a610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff58400d6b0 tx=0x7ff58400db70 comp rx=0 tx=0).stop 2024-04-25T18:52:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.501+0000 7ff58b200640 1 -- 172.21.15.112:0/1968728892 shutdown_connections 2024-04-25T18:52:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.501+0000 7ff58b200640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff5700a3d90 0x7ff5700a6250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.501+0000 7ff58b200640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff58c10c330 0x7ff58c19a610 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.501+0000 7ff58b200640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff58c103830 0x7ff58c1a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.501+0000 7ff58b200640 1 --2- 172.21.15.112:0/1968728892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c102f10 0x7ff58c1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.501+0000 7ff58b200640 1 -- 172.21.15.112:0/1968728892 >> 172.21.15.112:0/1968728892 conn(0x7ff58c0fc770 msgr2=0x7ff58c101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.501+0000 7ff58b200640 1 -- 172.21.15.112:0/1968728892 shutdown_connections 2024-04-25T18:52:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:52:53.502+0000 7ff58b200640 1 -- 172.21.15.112:0/1968728892 wait complete. 2024-04-25T18:52:53.521 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x9ff773 2024-04-25T18:52:54.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:53 smithi171 ceph-mon[33266]: pgmap v169: 97 pgs: 97 active+clean; 50 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-25T18:52:54.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:53 smithi161 ceph-mon[32609]: pgmap v169: 97 pgs: 97 active+clean; 50 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-25T18:52:54.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:53 smithi112 ceph-mon[27864]: pgmap v169: 97 pgs: 97 active+clean; 50 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-25T18:52:54.391 INFO:tasks.workunit.client.0.smithi112.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-04-25T18:52:54.724 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:52:54 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: ignoring --setuser ceph since I am not root 2024-04-25T18:52:54.724 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:52:54 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: ignoring --setgroup ceph since I am not root 2024-04-25T18:52:54.724 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:52:54 smithi112 ceph-mds[86791]: -- 172.21.15.112:0/1382771646 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a5431ba40 con 0x557a542ef000 2024-04-25T18:52:54.724 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:52:54 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: starting mds.g at 2024-04-25T18:52:55.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:54 smithi171 ceph-mon[33266]: daemon mds.g restarted 2024-04-25T18:52:55.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:54 smithi161 ceph-mon[32609]: daemon mds.g restarted 2024-04-25T18:52:55.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:54 smithi112 ceph-mon[27864]: daemon mds.g restarted 2024-04-25T18:52:56.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:55 smithi171 ceph-mon[33266]: pgmap v170: 97 pgs: 97 active+clean; 50 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-25T18:52:56.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:55 smithi171 ceph-mon[33266]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T18:52:56.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:55 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:52:56.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:55 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.112:6838/3065773788,v1:172.21.15.112:6839/3065773788] up:boot 2024-04-25T18:52:56.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:55 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T18:52:56.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:55 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:52:56.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:55 smithi161 ceph-mon[32609]: pgmap v170: 97 pgs: 97 active+clean; 50 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-25T18:52:56.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:55 smithi161 ceph-mon[32609]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T18:52:56.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:55 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:52:56.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:55 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.112:6838/3065773788,v1:172.21.15.112:6839/3065773788] up:boot 2024-04-25T18:52:56.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:55 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T18:52:56.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:55 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:52:56.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:55 smithi112 ceph-mon[27864]: pgmap v170: 97 pgs: 97 active+clean; 50 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-25T18:52:56.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:55 smithi112 ceph-mon[27864]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T18:52:56.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:55 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:52:56.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:55 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.112:6838/3065773788,v1:172.21.15.112:6839/3065773788] up:boot 2024-04-25T18:52:56.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:55 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T18:52:56.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:55 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:52:58.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:57 smithi171 ceph-mon[33266]: pgmap v172: 97 pgs: 97 active+clean; 36 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 305 op/s 2024-04-25T18:52:58.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:57 smithi161 ceph-mon[32609]: pgmap v172: 97 pgs: 97 active+clean; 36 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 305 op/s 2024-04-25T18:52:58.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:57 smithi112 ceph-mon[27864]: pgmap v172: 97 pgs: 97 active+clean; 36 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 305 op/s 2024-04-25T18:53:00.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:52:59 smithi171 ceph-mon[33266]: pgmap v173: 97 pgs: 97 active+clean; 36 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 305 op/s 2024-04-25T18:53:00.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:52:59 smithi161 ceph-mon[32609]: pgmap v173: 97 pgs: 97 active+clean; 36 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 305 op/s 2024-04-25T18:53:00.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:52:59 smithi112 ceph-mon[27864]: pgmap v173: 97 pgs: 97 active+clean; 36 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 305 op/s 2024-04-25T18:53:02.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:01 smithi171 ceph-mon[33266]: pgmap v174: 97 pgs: 97 active+clean; 45 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-04-25T18:53:02.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:01 smithi161 ceph-mon[32609]: pgmap v174: 97 pgs: 97 active+clean; 45 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-04-25T18:53:02.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:01 smithi112 ceph-mon[27864]: pgmap v174: 97 pgs: 97 active+clean; 45 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-04-25T18:53:04.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:03 smithi171 ceph-mon[33266]: pgmap v175: 97 pgs: 97 active+clean; 45 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-04-25T18:53:04.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:03 smithi161 ceph-mon[32609]: pgmap v175: 97 pgs: 97 active+clean; 45 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-04-25T18:53:04.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:03 smithi112 ceph-mon[27864]: pgmap v175: 97 pgs: 97 active+clean; 45 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-04-25T18:53:06.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:05 smithi161 ceph-mon[32609]: pgmap v176: 97 pgs: 97 active+clean; 45 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-04-25T18:53:06.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:05 smithi112 ceph-mon[27864]: pgmap v176: 97 pgs: 97 active+clean; 45 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-04-25T18:53:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:05 smithi171 ceph-mon[33266]: pgmap v176: 97 pgs: 97 active+clean; 45 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 324 op/s 2024-04-25T18:53:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.659+0000 7fec08c00640 1 -- 172.21.15.112:0/3420310104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec041041f0 msgr2=0x7fec041045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.659+0000 7fec08c00640 1 --2- 172.21.15.112:0/3420310104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec041041f0 0x7fec041045d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7febf4009900 tx=0x7febf402a3f0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.659+0000 7fec08c00640 1 -- 172.21.15.112:0/3420310104 shutdown_connections 2024-04-25T18:53:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.659+0000 7fec08c00640 1 --2- 172.21.15.112:0/3420310104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec041092c0 0x7fec04110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.659+0000 7fec08c00640 1 --2- 172.21.15.112:0/3420310104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec04104ba0 0x7fec04108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.659+0000 7fec08c00640 1 --2- 172.21.15.112:0/3420310104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec041041f0 0x7fec041045d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.659+0000 7fec08c00640 1 -- 172.21.15.112:0/3420310104 >> 172.21.15.112:0/3420310104 conn(0x7fec040ffa30 msgr2=0x7fec04101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.659+0000 7fec08c00640 1 -- 172.21.15.112:0/3420310104 shutdown_connections 2024-04-25T18:53:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.659+0000 7fec08c00640 1 -- 172.21.15.112:0/3420310104 wait complete. 2024-04-25T18:53:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.661+0000 7fec08c00640 1 Processor -- start 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.661+0000 7fec08c00640 1 -- start start 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.661+0000 7fec08c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec041041f0 0x7fec041a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.661+0000 7fec08c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec041092c0 0x7fec041a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.661+0000 7fec08c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec0419a650 0x7fec0419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.661+0000 7fec08c00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec04114250 con 0x7fec041092c0 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.661+0000 7fec08c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec041140d0 con 0x7fec041041f0 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.661+0000 7fec08c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fec041143d0 con 0x7fec0419a650 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.662+0000 7fec03400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec041041f0 0x7fec041a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.662+0000 7fec03e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec0419a650 0x7fec0419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.662+0000 7fec03400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec041041f0 0x7fec041a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:54436/0 (socket says 172.21.15.112:54436) 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.662+0000 7febfbe00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec041092c0 0x7fec041a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.662+0000 7fec03400640 1 -- 172.21.15.112:0/2399654749 learned_addr learned my addr 172.21.15.112:0/2399654749 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.662+0000 7fec03e00640 1 -- 172.21.15.112:0/2399654749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec041041f0 msgr2=0x7fec041a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.662+0000 7fec03e00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec041041f0 0x7fec041a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.662+0000 7fec03e00640 1 -- 172.21.15.112:0/2399654749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec041092c0 msgr2=0x7fec041a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.662+0000 7fec03e00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec041092c0 0x7fec041a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.662+0000 7fec03e00640 1 -- 172.21.15.112:0/2399654749 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0419b250 con 0x7fec0419a650 2024-04-25T18:53:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.662+0000 7febfbe00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec041092c0 0x7fec041a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.663+0000 7fec03e00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec0419a650 0x7fec0419aad0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7febfc00da10 tx=0x7febfc00ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.663+0000 7fec00c00640 1 -- 172.21.15.112:0/2399654749 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc00bd70 con 0x7fec0419a650 2024-04-25T18:53:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.663+0000 7fec00c00640 1 -- 172.21.15.112:0/2399654749 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc003e20 con 0x7fec0419a650 2024-04-25T18:53:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.663+0000 7fec08c00640 1 -- 172.21.15.112:0/2399654749 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec04076390 con 0x7fec0419a650 2024-04-25T18:53:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.663+0000 7fec00c00640 1 -- 172.21.15.112:0/2399654749 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc0106c0 con 0x7fec0419a650 2024-04-25T18:53:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.663+0000 7fec08c00640 1 -- 172.21.15.112:0/2399654749 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec040768e0 con 0x7fec0419a650 2024-04-25T18:53:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.667+0000 7fec00c00640 1 -- 172.21.15.112:0/2399654749 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7febfc004270 con 0x7fec0419a650 2024-04-25T18:53:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.667+0000 7fec0cfff740 1 Processor -- start 2024-04-25T18:53:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.668+0000 7fec0cfff740 1 -- start start 2024-04-25T18:53:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.668+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce6ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.668+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce75650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.668+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce75b90 0x55fedce77f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.668+0000 7fec0cfff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fedcdd0900 con 0x55fedce6a9b0 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.668+0000 7fec0cfff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fedcd76ef0 con 0x55fedce75b90 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.668+0000 7fec0cfff740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fedcdcc8c0 con 0x55fedce6b2d0 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.668+0000 7fec03400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce75650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.668+0000 7fec03400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce75650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:32834/0 (socket says 172.21.15.112:32834) 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.668+0000 7fec03400640 1 -- 172.21.15.112:0/1563329761 learned_addr learned my addr 172.21.15.112:0/1563329761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.668+0000 7febfbe00640 1 --2- 172.21.15.112:0/1563329761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce6ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.669+0000 7febfbe00640 1 --2- 172.21.15.112:0/1563329761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce75b90 0x55fedce77f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.669+0000 7fec03400640 1 -- 172.21.15.112:0/1563329761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce75b90 msgr2=0x55fedce77f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.669+0000 7fec03400640 1 --2- 172.21.15.112:0/1563329761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce75b90 0x55fedce77f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.669+0000 7fec03400640 1 -- 172.21.15.112:0/1563329761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 msgr2=0x55fedce6ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.669+0000 7fec03400640 1 --2- 172.21.15.112:0/1563329761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce6ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.669+0000 7fec03400640 1 -- 172.21.15.112:0/1563329761 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fedcae3210 con 0x55fedce6b2d0 2024-04-25T18:53:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.669+0000 7febfbe00640 1 --2- 172.21.15.112:0/1563329761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce6ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:53:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.669+0000 7fec00c00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7febe80ac5a0 0x7febe80aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.669+0000 7fec03400640 1 --2- 172.21.15.112:0/1563329761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce75650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7febf4006010 tx=0x7febf4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7758691cbd59fc0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.669+0000 7fec00c00640 1 -- 172.21.15.112:0/2399654749 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7febfc0d0aa0 con 0x7fec0419a650 2024-04-25T18:53:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.669+0000 7febfbe00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7febe80ac5a0 0x7febe80aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.670+0000 7febf3e00640 1 -- 172.21.15.112:0/1563329761 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4040070 con 0x55fedce6b2d0 2024-04-25T18:53:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.670+0000 7febf3e00640 1 -- 172.21.15.112:0/1563329761 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf40054d0 con 0x55fedce6b2d0 2024-04-25T18:53:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.670+0000 7febf3e00640 1 -- 172.21.15.112:0/1563329761 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4035040 con 0x55fedce6b2d0 2024-04-25T18:53:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.670+0000 7fec0cfff740 1 -- 172.21.15.112:0/1563329761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 msgr2=0x55fedce75650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.670+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1563329761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce75650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7febf4006010 tx=0x7febf4005c00 comp rx=0 tx=0).stop 2024-04-25T18:53:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.670+0000 7febfbe00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7febe80ac5a0 0x7febe80aea60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55fedce682e0 tx=0x7febec015040 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.671+0000 7fec0cfff740 1 -- 172.21.15.112:0/1563329761 shutdown_connections 2024-04-25T18:53:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.671+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1563329761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce75b90 0x55fedce77f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.671+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1563329761 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce75650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.671+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1563329761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce6ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.671+0000 7fec0cfff740 1 -- 172.21.15.112:0/1563329761 >> 172.21.15.112:0/1563329761 conn(0x55fedce66ec0 msgr2=0x55fedce672a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.671+0000 7fec0cfff740 1 -- 172.21.15.112:0/1563329761 shutdown_connections 2024-04-25T18:53:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.671+0000 7fec0cfff740 1 -- 172.21.15.112:0/1563329761 wait complete. 2024-04-25T18:53:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.671+0000 7fec0cfff740 1 -- start start 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.671+0000 7fec0cfff740 1 Processor -- start 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6a9b0 0x55fedce72b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce73080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce75b90 0x55fedce735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7fec0cfff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fedce78760 con 0x55fedce75b90 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7fec0cfff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fedcd8a310 con 0x55fedce6a9b0 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7fec0cfff740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fedce788e0 con 0x55fedce6b2d0 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7fec0cfff740 10 client.0 _ll_register_callbacks cb 0x55fedcd58cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7fec03e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce75b90 0x55fedce735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7febfbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce73080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7fec03400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6a9b0 0x55fedce72b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7febfbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce73080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:32846/0 (socket says 172.21.15.112:32846) 2024-04-25T18:53:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7fec03400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6a9b0 0x55fedce72b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:54462/0 (socket says 172.21.15.112:54462) 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.672+0000 7febfbe00640 1 -- 172.21.15.112:0/1298206251 learned_addr learned my addr 172.21.15.112:0/1298206251 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.673+0000 7febfbe00640 1 -- 172.21.15.112:0/1298206251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6a9b0 msgr2=0x55fedce72b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.673+0000 7febfbe00640 1 --2- 172.21.15.112:0/1298206251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6a9b0 0x55fedce72b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.673+0000 7febfbe00640 1 -- 172.21.15.112:0/1298206251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce75b90 msgr2=0x55fedce735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.673+0000 7febfbe00640 1 --2- 172.21.15.112:0/1298206251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce75b90 0x55fedce735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.673+0000 7febfbe00640 1 -- 172.21.15.112:0/1298206251 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fedcbaf620 con 0x55fedce6b2d0 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.673+0000 7febfbe00640 1 --2- 172.21.15.112:0/1298206251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce73080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7febec0206b0 tx=0x7febec005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3215d1c5a4e4b44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.673+0000 7febf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.673+0000 7febf0c00640 1 -- 172.21.15.112:0/1298206251 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec019030 con 0x55fedce6b2d0 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.674+0000 7febf0c00640 1 -- 172.21.15.112:0/1298206251 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7febec021710 con 0x55fedce6b2d0 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.674+0000 7fec0cfff740 1 -- 172.21.15.112:0/1298206251 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf40054d0 con 0x55fedce6b2d0 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.674+0000 7febf0c00640 1 -- 172.21.15.112:0/1298206251 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7febec021910 con 0x55fedce6b2d0 2024-04-25T18:53:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.674+0000 7fec0cfff740 10 client.25777 fetch_fsmap learned FSMap version 35 2024-04-25T18:53:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.674+0000 7fec0cfff740 1 -- 172.21.15.112:0/1298206251 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fedce95a50 con 0x55fedce6b2d0 2024-04-25T18:53:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.674+0000 7febf0c00640 1 -- 172.21.15.112:0/1298206251 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf40054d0 con 0x55fedce6b2d0 2024-04-25T18:53:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.674+0000 7febf0c00640 1 -- 172.21.15.112:0/1298206251 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7febec034b20 con 0x55fedce6b2d0 2024-04-25T18:53:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.674+0000 7fec0cfff740 10 client.25777 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T18:53:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.674+0000 7fec0cfff740 10 client.25777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:53:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.674+0000 7fec0cfff740 20 client.25777 populate_metadata read hostname 'smithi112' 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.674+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1298206251 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55fedce96f60 0x55fedce993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.675+0000 7fec0cfff740 4 client.25777 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.675+0000 7fec0cfff740 1 -- 172.21.15.112:0/1298206251 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fedcde48b0 con 0x55fedce96f60 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.675+0000 7fec03400640 1 --2- 172.21.15.112:0/1298206251 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55fedce96f60 0x55fedce993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.675+0000 7fec03400640 1 --2- 172.21.15.112:0/1298206251 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55fedce96f60 0x55fedce993b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=975aebc746f4358 server_cookie=801f8c1d2dd3a659 in_seq=0 out_seq=0 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.675+0000 7febf0c00640 10 client.25777 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.676+0000 7febf0c00640 1 -- 172.21.15.112:0/1298206251 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fedcde48b0 con 0x55fedce96f60 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.676+0000 7febf0c00640 10 client.25777 handle_command_reply: tid=0 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.676+0000 7fec0cfff740 1 client.25777 shutdown 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.677+0000 7fec0cfff740 1 -- 172.21.15.112:0/1298206251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 msgr2=0x55fedce73080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.677+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1298206251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce73080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7febec0206b0 tx=0x7febec005850 comp rx=0 tx=0).stop 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.677+0000 7fec0cfff740 1 -- 172.21.15.112:0/1298206251 shutdown_connections 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.677+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1298206251 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55fedce96f60 0x55fedce993b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.677+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1298206251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce75b90 0x55fedce735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.677+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1298206251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce73080 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.677+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1298206251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6a9b0 0x55fedce72b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.677+0000 7febf0c00640 0 client.25777 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.677+0000 7fec0cfff740 1 -- 172.21.15.112:0/1298206251 >> 172.21.15.112:0/1298206251 conn(0x55fedce66ec0 msgr2=0x55fedce6cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.677+0000 7fec0cfff740 1 -- 172.21.15.112:0/1298206251 shutdown_connections 2024-04-25T18:53:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.678+0000 7fec0cfff740 1 -- 172.21.15.112:0/1298206251 wait complete. 2024-04-25T18:53:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.678+0000 7fec0cfff740 20 client.25777 trim_cache size 0 max 16384 2024-04-25T18:53:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.703+0000 7fec0cfff740 1 Processor -- start 2024-04-25T18:53:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.703+0000 7fec0cfff740 1 -- start start 2024-04-25T18:53:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.703+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce81620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.703+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6b2d0 0x55fedce667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.703+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce75b90 0x55fedce74a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.703+0000 7fec0cfff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec04072810 con 0x55fedce6a9b0 2024-04-25T18:53:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.703+0000 7fec0cfff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec040700b0 con 0x55fedce6b2d0 2024-04-25T18:53:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.703+0000 7fec0cfff740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fec0406df00 con 0x55fedce75b90 2024-04-25T18:53:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.703+0000 7fec03400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce81620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.704+0000 7febfbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6b2d0 0x55fedce667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.704+0000 7febfbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6b2d0 0x55fedce667a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:54468/0 (socket says 172.21.15.112:54468) 2024-04-25T18:53:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.704+0000 7febfbe00640 1 -- 172.21.15.112:0/3832044060 learned_addr learned my addr 172.21.15.112:0/3832044060 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.704+0000 7fec03e00640 1 --2- 172.21.15.112:0/3832044060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce75b90 0x55fedce74a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.704+0000 7febfbe00640 1 -- 172.21.15.112:0/3832044060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce75b90 msgr2=0x55fedce74a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.704+0000 7febfbe00640 1 --2- 172.21.15.112:0/3832044060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce75b90 0x55fedce74a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.704+0000 7febfbe00640 1 -- 172.21.15.112:0/3832044060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 msgr2=0x55fedce81620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.704+0000 7febfbe00640 1 --2- 172.21.15.112:0/3832044060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce81620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.704+0000 7febfbe00640 1 -- 172.21.15.112:0/3832044060 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fedce680f0 con 0x55fedce6b2d0 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.704+0000 7febfbe00640 1 --2- 172.21.15.112:0/3832044060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6b2d0 0x55fedce667a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7febec020f40 tx=0x7febec0218a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f76808094032ff29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.705+0000 7febf3400640 1 -- 172.21.15.112:0/3832044060 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec01e730 con 0x55fedce6b2d0 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.705+0000 7febf3400640 1 -- 172.21.15.112:0/3832044060 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febec039070 con 0x55fedce6b2d0 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.705+0000 7febf3400640 1 -- 172.21.15.112:0/3832044060 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec005290 con 0x55fedce6b2d0 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.706+0000 7fec0cfff740 1 -- 172.21.15.112:0/3832044060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6b2d0 msgr2=0x55fedce667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.706+0000 7fec0cfff740 1 --2- 172.21.15.112:0/3832044060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6b2d0 0x55fedce667a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7febec020f40 tx=0x7febec0218a0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.706+0000 7fec0cfff740 1 -- 172.21.15.112:0/3832044060 shutdown_connections 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.706+0000 7fec0cfff740 1 --2- 172.21.15.112:0/3832044060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce75b90 0x55fedce74a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.706+0000 7fec0cfff740 1 --2- 172.21.15.112:0/3832044060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce6b2d0 0x55fedce667a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.706+0000 7fec0cfff740 1 --2- 172.21.15.112:0/3832044060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce81620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.706+0000 7fec0cfff740 1 -- 172.21.15.112:0/3832044060 >> 172.21.15.112:0/3832044060 conn(0x55fedce66ec0 msgr2=0x55fedce74370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.706+0000 7fec0cfff740 1 -- 172.21.15.112:0/3832044060 shutdown_connections 2024-04-25T18:53:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.706+0000 7fec0cfff740 1 -- 172.21.15.112:0/3832044060 wait complete. 2024-04-25T18:53:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.706+0000 7fec0cfff740 1 -- start start 2024-04-25T18:53:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.706+0000 7fec0cfff740 1 Processor -- start 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.707+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce7b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.707+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce7bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.707+0000 7fec0cfff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce75b90 0x55fedce7c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.707+0000 7fec0cfff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec0406e570 con 0x55fedce6a9b0 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.707+0000 7fec0cfff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec0406f9e0 con 0x55fedce75b90 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.707+0000 7fec0cfff740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fec0406ec40 con 0x55fedce6b2d0 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.707+0000 7fec0cfff740 10 client.0 _ll_register_callbacks cb 0x7febec011e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.707+0000 7fec03400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce7b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.707+0000 7fec03400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce7b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44746/0 (socket says 172.21.15.112:44746) 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.707+0000 7fec03400640 1 -- 172.21.15.112:0/1347488494 learned_addr learned my addr 172.21.15.112:0/1347488494 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.707+0000 7febfbe00640 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce7bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.708+0000 7fec03400640 1 -- 172.21.15.112:0/1347488494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 msgr2=0x55fedce7bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.708+0000 7fec03400640 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce7bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.708+0000 7fec03400640 1 -- 172.21.15.112:0/1347488494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce75b90 msgr2=0x55fedce7c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.708+0000 7fec03400640 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce75b90 0x55fedce7c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.708+0000 7fec03400640 1 -- 172.21.15.112:0/1347488494 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fedcefc660 con 0x55fedce6a9b0 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.708+0000 7febfbe00640 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce7bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.708+0000 7fec03400640 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce7b6d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7febf4004cf0 tx=0x7febf4004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13a29b51e2b0403 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.708+0000 7febf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.709+0000 7febf0c00640 1 -- 172.21.15.112:0/1347488494 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4040030 con 0x55fedce6a9b0 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.709+0000 7febf0c00640 1 -- 172.21.15.112:0/1347488494 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7febf4051410 con 0x55fedce6a9b0 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.709+0000 7fec0cfff740 1 -- 172.21.15.112:0/1347488494 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febec039070 con 0x55fedce6a9b0 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.709+0000 7febf0c00640 1 -- 172.21.15.112:0/1347488494 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7febf4060050 con 0x55fedce6a9b0 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.709+0000 7fec0cfff740 10 client.15618 fetch_fsmap learned FSMap version 35 2024-04-25T18:53:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.709+0000 7fec0cfff740 1 -- 172.21.15.112:0/1347488494 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fedcefd950 con 0x55fedce6a9b0 2024-04-25T18:53:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.710+0000 7febf0c00640 1 -- 172.21.15.112:0/1347488494 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febec039070 con 0x55fedce6a9b0 2024-04-25T18:53:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.710+0000 7febf0c00640 1 -- 172.21.15.112:0/1347488494 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7febf40052d0 con 0x55fedce6a9b0 2024-04-25T18:53:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.710+0000 7fec0cfff740 10 client.15618 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T18:53:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.710+0000 7fec0cfff740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:53:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.710+0000 7fec0cfff740 20 client.15618 populate_metadata read hostname 'smithi112' 2024-04-25T18:53:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.710+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55fedcefdc90 0x55fedcf000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.710+0000 7fec0cfff740 4 client.15618 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:53:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.710+0000 7fec0cfff740 1 -- 172.21.15.112:0/1347488494 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fedcdde320 con 0x55fedcefdc90 2024-04-25T18:53:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.711+0000 7febfbe00640 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55fedcefdc90 0x55fedcf000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.711+0000 7febfbe00640 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55fedcefdc90 0x55fedcf000d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aa9133a59b72d78 server_cookie=d5ab6a2efae62a3f in_seq=0 out_seq=0 2024-04-25T18:53:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.711+0000 7febf0c00640 10 client.15618 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7febf0c00640 1 -- 172.21.15.112:0/1347488494 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fedcdde320 con 0x55fedcefdc90 2024-04-25T18:53:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7febf0c00640 10 client.15618 handle_command_reply: tid=0 2024-04-25T18:53:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 1 client.15618 shutdown 2024-04-25T18:53:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 1 -- 172.21.15.112:0/1347488494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 msgr2=0x55fedce7b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce7b6d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7febf4004cf0 tx=0x7febf4004d20 comp rx=0 tx=0).stop 2024-04-25T18:53:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 1 -- 172.21.15.112:0/1347488494 shutdown_connections 2024-04-25T18:53:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fedce75b90 0x55fedce7c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fedce6b2d0 0x55fedce7bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55fedcefdc90 0x55fedcf000d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 1 --2- 172.21.15.112:0/1347488494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fedce6a9b0 0x55fedce7b6d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 1 -- 172.21.15.112:0/1347488494 >> 172.21.15.112:0/1347488494 conn(0x55fedce66ec0 msgr2=0x55fedce74c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7febf0c00640 0 client.15618 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 1 -- 172.21.15.112:0/1347488494 shutdown_connections 2024-04-25T18:53:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 1 -- 172.21.15.112:0/1347488494 wait complete. 2024-04-25T18:53:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.731+0000 7fec0cfff740 20 client.15618 trim_cache size 0 max 16384 2024-04-25T18:53:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:53:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:53:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:53:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:53:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 -- 172.21.15.112:0/2399654749 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7febe80ac5a0 msgr2=0x7febe80aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7febe80ac5a0 0x7febe80aea60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55fedce682e0 tx=0x7febec015040 comp rx=0 tx=0).stop 2024-04-25T18:53:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 -- 172.21.15.112:0/2399654749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec0419a650 msgr2=0x7fec0419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec0419a650 0x7fec0419aad0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7febfc00da10 tx=0x7febfc00ded0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 -- 172.21.15.112:0/2399654749 shutdown_connections 2024-04-25T18:53:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7febe80ac5a0 0x7febe80aea60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec0419a650 0x7fec0419aad0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec041092c0 0x7fec041a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 --2- 172.21.15.112:0/2399654749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec041041f0 0x7fec041a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 -- 172.21.15.112:0/2399654749 >> 172.21.15.112:0/2399654749 conn(0x7fec040ffa30 msgr2=0x7fec04100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 -- 172.21.15.112:0/2399654749 shutdown_connections 2024-04-25T18:53:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:06.732+0000 7fec08c00640 1 -- 172.21.15.112:0/2399654749 wait complete. 2024-04-25T18:53:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:07 smithi161 ceph-mon[32609]: pgmap v177: 97 pgs: 97 active+clean; 48 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 414 op/s 2024-04-25T18:53:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:07 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:53:08.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:07 smithi112 ceph-mon[27864]: pgmap v177: 97 pgs: 97 active+clean; 48 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 414 op/s 2024-04-25T18:53:08.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:07 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:53:08.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:07 smithi171 ceph-mon[33266]: pgmap v177: 97 pgs: 97 active+clean; 48 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 414 op/s 2024-04-25T18:53:08.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:07 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:53:10.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:09 smithi161 ceph-mon[32609]: pgmap v178: 97 pgs: 97 active+clean; 48 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-04-25T18:53:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:09 smithi112 ceph-mon[27864]: pgmap v178: 97 pgs: 97 active+clean; 48 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-04-25T18:53:10.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:09 smithi171 ceph-mon[33266]: pgmap v178: 97 pgs: 97 active+clean; 48 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-04-25T18:53:12.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:11 smithi171 ceph-mon[33266]: pgmap v179: 97 pgs: 97 active+clean; 51 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 393 op/s 2024-04-25T18:53:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:11 smithi161 ceph-mon[32609]: pgmap v179: 97 pgs: 97 active+clean; 51 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 393 op/s 2024-04-25T18:53:12.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:11 smithi112 ceph-mon[27864]: pgmap v179: 97 pgs: 97 active+clean; 51 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 393 op/s 2024-04-25T18:53:12.670 INFO:tasks.workunit.client.0.smithi112.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-04-25T18:53:14.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:13 smithi171 ceph-mon[33266]: pgmap v180: 97 pgs: 97 active+clean; 51 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-25T18:53:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:13 smithi161 ceph-mon[32609]: pgmap v180: 97 pgs: 97 active+clean; 51 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-25T18:53:14.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:13 smithi112 ceph-mon[27864]: pgmap v180: 97 pgs: 97 active+clean; 51 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-25T18:53:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:15 smithi171 ceph-mon[33266]: pgmap v181: 97 pgs: 97 active+clean; 51 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-25T18:53:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:15 smithi161 ceph-mon[32609]: pgmap v181: 97 pgs: 97 active+clean; 51 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-25T18:53:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:15 smithi112 ceph-mon[27864]: pgmap v181: 97 pgs: 97 active+clean; 51 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-25T18:53:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:17 smithi171 ceph-mon[33266]: pgmap v182: 97 pgs: 97 active+clean; 52 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 365 op/s 2024-04-25T18:53:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:17 smithi161 ceph-mon[32609]: pgmap v182: 97 pgs: 97 active+clean; 52 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 365 op/s 2024-04-25T18:53:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:17 smithi112 ceph-mon[27864]: pgmap v182: 97 pgs: 97 active+clean; 52 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 365 op/s 2024-04-25T18:53:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.888+0000 7f55c3200640 1 -- 172.21.15.112:0/2309049623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55c4100eb0 msgr2=0x7f55c4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.888+0000 7f55c3200640 1 --2- 172.21.15.112:0/2309049623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55c4100eb0 0x7f55c4101290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009900 tx=0x7f55b402a3f0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.889+0000 7f55c3200640 1 -- 172.21.15.112:0/2309049623 shutdown_connections 2024-04-25T18:53:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.889+0000 7f55c3200640 1 --2- 172.21.15.112:0/2309049623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c4105f80 0x7f55c410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.889+0000 7f55c3200640 1 --2- 172.21.15.112:0/2309049623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c4101860 0x7f55c41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.889+0000 7f55c3200640 1 --2- 172.21.15.112:0/2309049623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55c4100eb0 0x7f55c4101290 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.889+0000 7f55c3200640 1 -- 172.21.15.112:0/2309049623 >> 172.21.15.112:0/2309049623 conn(0x7f55c40fc770 msgr2=0x7f55c40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.889+0000 7f55c3200640 1 -- 172.21.15.112:0/2309049623 shutdown_connections 2024-04-25T18:53:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.889+0000 7f55c3200640 1 -- 172.21.15.112:0/2309049623 wait complete. 2024-04-25T18:53:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.890+0000 7f55c3200640 1 Processor -- start 2024-04-25T18:53:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.890+0000 7f55c3200640 1 -- start start 2024-04-25T18:53:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.890+0000 7f55c3200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c4100eb0 0x7f55c41a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.890+0000 7f55c3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55c4105f80 0x7f55c41a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.890+0000 7f55c3200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c419a5d0 0x7f55c419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.890+0000 7f55c3200640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4112030 con 0x7f55c4100eb0 2024-04-25T18:53:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.890+0000 7f55c3200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4111eb0 con 0x7f55c419a5d0 2024-04-25T18:53:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.890+0000 7f55c3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41121b0 con 0x7f55c4105f80 2024-04-25T18:53:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c1e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c4100eb0 0x7f55c41a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55c4105f80 0x7f55c41a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55c4105f80 0x7f55c41a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:49022/0 (socket says 172.21.15.112:49022) 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c1400640 1 -- 172.21.15.112:0/3806974176 learned_addr learned my addr 172.21.15.112:0/3806974176 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c1e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c4100eb0 0x7f55c41a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48012/0 (socket says 172.21.15.112:48012) 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c2800640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c419a5d0 0x7f55c419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c1400640 1 -- 172.21.15.112:0/3806974176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c419a5d0 msgr2=0x7f55c419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c1400640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c419a5d0 0x7f55c419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c1400640 1 -- 172.21.15.112:0/3806974176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c4100eb0 msgr2=0x7f55c41a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c1400640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c4100eb0 0x7f55c41a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c1400640 1 -- 172.21.15.112:0/3806974176 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c419b0f0 con 0x7f55c4105f80 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.891+0000 7f55c1e00640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c4100eb0 0x7f55c41a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.892+0000 7f55c1400640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55c4105f80 0x7f55c41a0960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c970 tx=0x7f55b000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.892+0000 7f55baa00640 1 -- 172.21.15.112:0/3806974176 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0007bb0 con 0x7f55c4105f80 2024-04-25T18:53:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.892+0000 7f55c3200640 1 -- 172.21.15.112:0/3806974176 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c419b3e0 con 0x7f55c4105f80 2024-04-25T18:53:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.892+0000 7f55baa00640 1 -- 172.21.15.112:0/3806974176 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55b0007d50 con 0x7f55c4105f80 2024-04-25T18:53:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.892+0000 7f55baa00640 1 -- 172.21.15.112:0/3806974176 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b000f450 con 0x7f55c4105f80 2024-04-25T18:53:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.892+0000 7f55c3200640 1 -- 172.21.15.112:0/3806974176 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41a7630 con 0x7f55c4105f80 2024-04-25T18:53:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.896+0000 7f55cbf8c740 1 Processor -- start 2024-04-25T18:53:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.896+0000 7f55baa00640 1 -- 172.21.15.112:0/3806974176 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f55b0004ec0 con 0x7f55c4105f80 2024-04-25T18:53:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.896+0000 7f55cbf8c740 1 -- start start 2024-04-25T18:53:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.897+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd19b0 0x565042fd1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.897+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd22d0 0x565042fdc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.897+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x565042fdef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.897+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565042f37900 con 0x565042fd22d0 2024-04-25T18:53:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.897+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565042eddef0 con 0x565042fdcb90 2024-04-25T18:53:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.897+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565042f338c0 con 0x565042fd19b0 2024-04-25T18:53:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.897+0000 7f55c1e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd22d0 0x565042fdc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.897+0000 7f55c2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd19b0 0x565042fd1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.897+0000 7f55c1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x565042fdef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.897+0000 7f55c2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd19b0 0x565042fd1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:49032/0 (socket says 172.21.15.112:49032) 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.897+0000 7f55c2800640 1 -- 172.21.15.112:0/4245510835 learned_addr learned my addr 172.21.15.112:0/4245510835 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.898+0000 7f55c2800640 1 -- 172.21.15.112:0/4245510835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 msgr2=0x565042fdef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.898+0000 7f55c2800640 1 --2- 172.21.15.112:0/4245510835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x565042fdef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.898+0000 7f55c2800640 1 -- 172.21.15.112:0/4245510835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd22d0 msgr2=0x565042fdc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.898+0000 7f55c2800640 1 --2- 172.21.15.112:0/4245510835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd22d0 0x565042fdc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.898+0000 7f55c2800640 1 -- 172.21.15.112:0/4245510835 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565042c4a210 con 0x565042fd19b0 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.898+0000 7f55c1e00640 1 --2- 172.21.15.112:0/4245510835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd22d0 0x565042fdc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.898+0000 7f55c2800640 1 --2- 172.21.15.112:0/4245510835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd19b0 0x565042fd1d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009f70 tx=0x7f55bc010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72cfbc1cfd50f565 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.899+0000 7f55baa00640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f55a80a3d10 0x7f55a80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.899+0000 7f55af400640 1 -- 172.21.15.112:0/4245510835 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc016070 con 0x565042fd19b0 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.899+0000 7f55af400640 1 -- 172.21.15.112:0/4245510835 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55bc004d20 con 0x565042fd19b0 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.899+0000 7f55af400640 1 -- 172.21.15.112:0/4245510835 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0127d0 con 0x565042fd19b0 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.899+0000 7f55baa00640 1 -- 172.21.15.112:0/3806974176 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7f55b00d04d0 con 0x7f55c4105f80 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.899+0000 7f55c1e00640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f55a80a3d10 0x7f55a80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.900+0000 7f55c1e00640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f55a80a3d10 0x7f55a80a61d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005ab0 tx=0x7f55b4005a20 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.900+0000 7f55cbf8c740 1 -- 172.21.15.112:0/4245510835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd19b0 msgr2=0x565042fd1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.900+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/4245510835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd19b0 0x565042fd1d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009f70 tx=0x7f55bc010800 comp rx=0 tx=0).stop 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.900+0000 7f55cbf8c740 1 -- 172.21.15.112:0/4245510835 shutdown_connections 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.900+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/4245510835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x565042fdef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.900+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/4245510835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd22d0 0x565042fdc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.900+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/4245510835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd19b0 0x565042fd1d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.900+0000 7f55cbf8c740 1 -- 172.21.15.112:0/4245510835 >> 172.21.15.112:0/4245510835 conn(0x565042fcdec0 msgr2=0x565042fce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.900+0000 7f55cbf8c740 1 -- 172.21.15.112:0/4245510835 shutdown_connections 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.900+0000 7f55cbf8c740 1 -- 172.21.15.112:0/4245510835 wait complete. 2024-04-25T18:53:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.901+0000 7f55cbf8c740 1 -- start start 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.901+0000 7f55cbf8c740 1 Processor -- start 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.901+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fd19b0 0x565042fd9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.901+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fda130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.901+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fdcb90 0x565042fda670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.901+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565042fdf7c0 con 0x565042fdcb90 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.901+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565042ef1310 con 0x565042fd19b0 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.901+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565042fdf940 con 0x565042fd22d0 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.901+0000 7f55cbf8c740 10 client.0 _ll_register_callbacks cb 0x565042ebfcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fd19b0 0x565042fd9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fd19b0 0x565042fd9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:49710/0 (socket says 172.21.15.112:49710) 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c1e00640 1 -- 172.21.15.112:0/3941304312 learned_addr learned my addr 172.21.15.112:0/3941304312 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c2800640 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fdcb90 0x565042fda670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c1400640 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fda130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c1e00640 1 -- 172.21.15.112:0/3941304312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 msgr2=0x565042fda130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c1e00640 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fda130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c1e00640 1 -- 172.21.15.112:0/3941304312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fdcb90 msgr2=0x565042fda670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c1e00640 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fdcb90 0x565042fda670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c1e00640 1 -- 172.21.15.112:0/3941304312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565042d16620 con 0x565042fd19b0 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c1400640 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fda130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.902+0000 7f55c1e00640 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fd19b0 0x565042fd9bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009900 tx=0x7f55b4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbe06879a4576f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.903+0000 7f55a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.903+0000 7f55a3e00640 1 -- 172.21.15.112:0/3941304312 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4035040 con 0x565042fd19b0 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.903+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3941304312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55bc004d20 con 0x565042fd19b0 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.903+0000 7f55a3e00640 1 -- 172.21.15.112:0/3941304312 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7f55b4003a00 con 0x565042fd19b0 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.903+0000 7f55a3e00640 1 -- 172.21.15.112:0/3941304312 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f55b4004e80 con 0x565042fd19b0 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.903+0000 7f55cbf8c740 10 client.25766 fetch_fsmap learned FSMap version 36 2024-04-25T18:53:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.903+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3941304312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x565042febb00 con 0x565042fd19b0 2024-04-25T18:53:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.903+0000 7f55a3e00640 1 -- 172.21.15.112:0/3941304312 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55bc004d20 con 0x565042fd19b0 2024-04-25T18:53:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.904+0000 7f55a3e00640 1 -- 172.21.15.112:0/3941304312 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f55b4044030 con 0x565042fd19b0 2024-04-25T18:53:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.904+0000 7f55cbf8c740 10 client.25766 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:53:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.904+0000 7f55cbf8c740 10 client.25766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:53:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.904+0000 7f55cbf8c740 20 client.25766 populate_metadata read hostname 'smithi112' 2024-04-25T18:53:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.904+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565042fecfc0 0x565042fef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.904+0000 7f55cbf8c740 4 client.25766 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:53:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.904+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3941304312 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565042f4b8b0 con 0x565042fecfc0 2024-04-25T18:53:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.904+0000 7f55c1400640 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565042fecfc0 0x565042fef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.905+0000 7f55c1400640 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565042fecfc0 0x565042fef3f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b821bb3c7c9de5f server_cookie=96472151d69efd65 in_seq=0 out_seq=0 2024-04-25T18:53:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.905+0000 7f55a3e00640 10 client.25766 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.905+0000 7f55a3e00640 1 -- 172.21.15.112:0/3941304312 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565042f4b8b0 con 0x565042fecfc0 2024-04-25T18:53:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.905+0000 7f55a3e00640 10 client.25766 handle_command_reply: tid=0 2024-04-25T18:53:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.905+0000 7f55cbf8c740 1 client.25766 shutdown 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3941304312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fd19b0 msgr2=0x565042fd9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fd19b0 0x565042fd9bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009900 tx=0x7f55b4009260 comp rx=0 tx=0).stop 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3941304312 shutdown_connections 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565042fecfc0 0x565042fef3f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fdcb90 0x565042fda670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fda130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3941304312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fd19b0 0x565042fd9bf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3941304312 >> 172.21.15.112:0/3941304312 conn(0x565042fcdec0 msgr2=0x565042fd3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55a3e00640 0 client.25766 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3941304312 shutdown_connections 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3941304312 wait complete. 2024-04-25T18:53:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.906+0000 7f55cbf8c740 20 client.25766 trim_cache size 0 max 16384 2024-04-25T18:53:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.924+0000 7f55cbf8c740 1 Processor -- start 2024-04-25T18:53:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.924+0000 7f55cbf8c740 1 -- start start 2024-04-25T18:53:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd19b0 0x565042fe8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fd43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x565042ff4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41b2580 con 0x565042fd19b0 2024-04-25T18:53:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41afe50 con 0x565042fdcb90 2024-04-25T18:53:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41adca0 con 0x565042fd22d0 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55c1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fd43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55c1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fd43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:49060/0 (socket says 172.21.15.112:49060) 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55c1400640 1 -- 172.21.15.112:0/2680481605 learned_addr learned my addr 172.21.15.112:0/2680481605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55c1e00640 1 --2- 172.21.15.112:0/2680481605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd19b0 0x565042fe8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55c2800640 1 --2- 172.21.15.112:0/2680481605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x565042ff4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55c1400640 1 -- 172.21.15.112:0/2680481605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 msgr2=0x565042ff4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55c1400640 1 --2- 172.21.15.112:0/2680481605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x565042ff4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55c1400640 1 -- 172.21.15.112:0/2680481605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd19b0 msgr2=0x565042fe8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55c1400640 1 --2- 172.21.15.112:0/2680481605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd19b0 0x565042fe8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.925+0000 7f55c1400640 1 -- 172.21.15.112:0/2680481605 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565042fd4900 con 0x565042fd22d0 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.926+0000 7f55c1400640 1 --2- 172.21.15.112:0/2680481605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fd43c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f55b009c450 tx=0x7f55b00d2990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd5fc097fd3ee780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.926+0000 7f55aea00640 1 -- 172.21.15.112:0/2680481605 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b001d040 con 0x565042fd22d0 2024-04-25T18:53:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.926+0000 7f55aea00640 1 -- 172.21.15.112:0/2680481605 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55b00046c0 con 0x565042fd22d0 2024-04-25T18:53:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.926+0000 7f55aea00640 1 -- 172.21.15.112:0/2680481605 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0098b30 con 0x565042fd22d0 2024-04-25T18:53:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.927+0000 7f55cbf8c740 1 -- 172.21.15.112:0/2680481605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 msgr2=0x565042fd43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.927+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/2680481605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fd43c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f55b009c450 tx=0x7f55b00d2990 comp rx=0 tx=0).stop 2024-04-25T18:53:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.927+0000 7f55cbf8c740 1 -- 172.21.15.112:0/2680481605 shutdown_connections 2024-04-25T18:53:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.927+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/2680481605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x565042ff4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.927+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/2680481605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fd43c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.927+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/2680481605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd19b0 0x565042fe8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.927+0000 7f55cbf8c740 1 -- 172.21.15.112:0/2680481605 >> 172.21.15.112:0/2680481605 conn(0x565042fcdec0 msgr2=0x565042fcd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.927+0000 7f55cbf8c740 1 -- 172.21.15.112:0/2680481605 shutdown_connections 2024-04-25T18:53:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.927+0000 7f55cbf8c740 1 -- 172.21.15.112:0/2680481605 wait complete. 2024-04-25T18:53:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.927+0000 7f55cbf8c740 1 -- start start 2024-04-25T18:53:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.928+0000 7f55cbf8c740 1 Processor -- start 2024-04-25T18:53:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.928+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd19b0 0x565042fe2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.928+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fe33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.928+0000 7f55cbf8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x56504304c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.928+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41ae310 con 0x565042fd19b0 2024-04-25T18:53:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.928+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41af780 con 0x565042fdcb90 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.928+0000 7f55cbf8c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41ae9e0 con 0x565042fd22d0 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.928+0000 7f55cbf8c740 10 client.0 _ll_register_callbacks cb 0x7f55b40054d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fe33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c1e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd19b0 0x565042fe2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fe33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:49070/0 (socket says 172.21.15.112:49070) 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c1400640 1 -- 172.21.15.112:0/3638266017 learned_addr learned my addr 172.21.15.112:0/3638266017 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c2800640 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x56504304c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c1400640 1 -- 172.21.15.112:0/3638266017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 msgr2=0x56504304c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c1400640 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x56504304c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c1400640 1 -- 172.21.15.112:0/3638266017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd19b0 msgr2=0x565042fe2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c1400640 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd19b0 0x565042fe2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c1400640 1 -- 172.21.15.112:0/3638266017 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504304cc30 con 0x565042fd22d0 2024-04-25T18:53:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c1e00640 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd19b0 0x565042fe2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55c1400640 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fe33f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f55b0098920 tx=0x7f55b0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cfaf9a3ce4d81f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.929+0000 7f55a3e00640 1 -- 172.21.15.112:0/3638266017 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b001d070 con 0x565042fd22d0 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55a3e00640 1 -- 172.21.15.112:0/3638266017 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7f55b0098b30 con 0x565042fd22d0 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3638266017 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565042f45320 con 0x565042fd22d0 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55a3e00640 1 -- 172.21.15.112:0/3638266017 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f55b009c4c0 con 0x565042fd22d0 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55cbf8c740 10 client.25801 fetch_fsmap learned FSMap version 36 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3638266017 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x565043053fe0 con 0x565042fd22d0 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55a3e00640 1 -- 172.21.15.112:0/3638266017 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x565042f45320 con 0x565042fd22d0 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55a3e00640 1 -- 172.21.15.112:0/3638266017 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f55b00982a0 con 0x565042fd22d0 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55cbf8c740 10 client.25801 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55cbf8c740 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55cbf8c740 20 client.25801 populate_metadata read hostname 'smithi112' 2024-04-25T18:53:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565043054780 0x565043056ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55cbf8c740 4 client.25801 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:53:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.930+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3638266017 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565042f3f190 con 0x565043054780 2024-04-25T18:53:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.931+0000 7f55c1e00640 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565043054780 0x565043056ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.931+0000 7f55c1e00640 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565043054780 0x565043056ba0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62243291e458bf78 server_cookie=825c31165bc6c39e in_seq=0 out_seq=0 2024-04-25T18:53:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.931+0000 7f55a3e00640 10 client.25801 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55a3e00640 1 -- 172.21.15.112:0/3638266017 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565042f3f190 con 0x565043054780 2024-04-25T18:53:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55a3e00640 10 client.25801 handle_command_reply: tid=0 2024-04-25T18:53:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55cbf8c740 1 client.25801 shutdown 2024-04-25T18:53:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3638266017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 msgr2=0x565042fe33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fe33f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f55b0098920 tx=0x7f55b0000bc0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3638266017 shutdown_connections 2024-04-25T18:53:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565042fdcb90 0x56504304c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565042fd22d0 0x565042fe33f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565043054780 0x565043056ba0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55cbf8c740 1 --2- 172.21.15.112:0/3638266017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565042fd19b0 0x565042fe2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55a3e00640 0 client.25801 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3638266017 >> 172.21.15.112:0/3638266017 conn(0x565042fcdec0 msgr2=0x565042fd9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3638266017 shutdown_connections 2024-04-25T18:53:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.943+0000 7f55cbf8c740 1 -- 172.21.15.112:0/3638266017 wait complete. 2024-04-25T18:53:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.944+0000 7f55cbf8c740 20 client.25801 trim_cache size 0 max 16384 2024-04-25T18:53:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:53:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:53:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:53:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:53:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 -- 172.21.15.112:0/3806974176 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f55a80a3d10 msgr2=0x7f55a80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f55a80a3d10 0x7f55a80a61d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005ab0 tx=0x7f55b4005a20 comp rx=0 tx=0).stop 2024-04-25T18:53:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 -- 172.21.15.112:0/3806974176 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55c4105f80 msgr2=0x7f55c41a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55c4105f80 0x7f55c41a0960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c970 tx=0x7f55b000ce30 comp rx=0 tx=0).stop 2024-04-25T18:53:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 -- 172.21.15.112:0/3806974176 shutdown_connections 2024-04-25T18:53:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f55a80a3d10 0x7f55a80a61d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c419a5d0 0x7f55c419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55c4105f80 0x7f55c41a0960 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 --2- 172.21.15.112:0/3806974176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55c4100eb0 0x7f55c41a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 -- 172.21.15.112:0/3806974176 >> 172.21.15.112:0/3806974176 conn(0x7f55c40fc770 msgr2=0x7f55c40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 -- 172.21.15.112:0/3806974176 shutdown_connections 2024-04-25T18:53:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:19.945+0000 7f55c3200640 1 -- 172.21.15.112:0/3806974176 wait complete. 2024-04-25T18:53:20.273 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:19 smithi161 ceph-mon[32609]: pgmap v183: 97 pgs: 97 active+clean; 52 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-04-25T18:53:20.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:19 smithi171 ceph-mon[33266]: pgmap v183: 97 pgs: 97 active+clean; 52 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-04-25T18:53:20.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:19 smithi112 ceph-mon[27864]: pgmap v183: 97 pgs: 97 active+clean; 52 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-04-25T18:53:22.248 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x9ffb01 2024-04-25T18:53:22.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:22 smithi171 ceph-mon[33266]: pgmap v184: 97 pgs: 97 active+clean; 49 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 352 op/s 2024-04-25T18:53:22.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:22 smithi161 ceph-mon[32609]: pgmap v184: 97 pgs: 97 active+clean; 49 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 352 op/s 2024-04-25T18:53:22.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:22 smithi112 ceph-mon[27864]: pgmap v184: 97 pgs: 97 active+clean; 49 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 352 op/s 2024-04-25T18:53:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:24 smithi171 ceph-mon[33266]: pgmap v185: 97 pgs: 97 active+clean; 49 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:24 smithi161 ceph-mon[32609]: pgmap v185: 97 pgs: 97 active+clean; 49 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:24.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:24 smithi112 ceph-mon[27864]: pgmap v185: 97 pgs: 97 active+clean; 49 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:26.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:26 smithi171 ceph-mon[33266]: pgmap v186: 97 pgs: 97 active+clean; 49 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:26.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:26 smithi161 ceph-mon[32609]: pgmap v186: 97 pgs: 97 active+clean; 49 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:26.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:26 smithi112 ceph-mon[27864]: pgmap v186: 97 pgs: 97 active+clean; 49 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:28.276 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:28 smithi112 ceph-mon[27864]: pgmap v187: 97 pgs: 97 active+clean; 49 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.3 MiB/s wr, 361 op/s 2024-04-25T18:53:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:28 smithi171 ceph-mon[33266]: pgmap v187: 97 pgs: 97 active+clean; 49 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.3 MiB/s wr, 361 op/s 2024-04-25T18:53:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:28 smithi161 ceph-mon[32609]: pgmap v187: 97 pgs: 97 active+clean; 49 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.3 MiB/s wr, 361 op/s 2024-04-25T18:53:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:30 smithi171 ceph-mon[33266]: pgmap v188: 97 pgs: 97 active+clean; 49 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 251 op/s 2024-04-25T18:53:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:30 smithi161 ceph-mon[32609]: pgmap v188: 97 pgs: 97 active+clean; 49 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 251 op/s 2024-04-25T18:53:30.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:30 smithi112 ceph-mon[27864]: pgmap v188: 97 pgs: 97 active+clean; 49 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 251 op/s 2024-04-25T18:53:31.111 INFO:tasks.workunit.client.0.smithi112.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-04-25T18:53:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:32 smithi171 ceph-mon[33266]: pgmap v189: 97 pgs: 97 active+clean; 55 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.3 MiB/s wr, 378 op/s 2024-04-25T18:53:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:32 smithi161 ceph-mon[32609]: pgmap v189: 97 pgs: 97 active+clean; 55 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.3 MiB/s wr, 378 op/s 2024-04-25T18:53:32.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:32 smithi112 ceph-mon[27864]: pgmap v189: 97 pgs: 97 active+clean; 55 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.3 MiB/s wr, 378 op/s 2024-04-25T18:53:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.102+0000 7f8fd9400640 1 -- 172.21.15.112:0/3348620428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd410c520 msgr2=0x7f8fd410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.102+0000 7f8fd9400640 1 --2- 172.21.15.112:0/3348620428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd410c520 0x7f8fd410e9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009ca0 tx=0x7f8fcc02a9e0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.102+0000 7f8fd9400640 1 -- 172.21.15.112:0/3348620428 shutdown_connections 2024-04-25T18:53:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.102+0000 7f8fd9400640 1 --2- 172.21.15.112:0/3348620428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd410c520 0x7f8fd410e9c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.102+0000 7f8fd9400640 1 --2- 172.21.15.112:0/3348620428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd4103b00 0x7f8fd410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.102+0000 7f8fd9400640 1 --2- 172.21.15.112:0/3348620428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd41031e0 0x7f8fd41035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.102+0000 7f8fd9400640 1 -- 172.21.15.112:0/3348620428 >> 172.21.15.112:0/3348620428 conn(0x7f8fd4077fd0 msgr2=0x7f8fd40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.102+0000 7f8fd9400640 1 -- 172.21.15.112:0/3348620428 shutdown_connections 2024-04-25T18:53:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.102+0000 7f8fd9400640 1 -- 172.21.15.112:0/3348620428 wait complete. 2024-04-25T18:53:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.103+0000 7f8fd9400640 1 Processor -- start 2024-04-25T18:53:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.103+0000 7f8fd9400640 1 -- start start 2024-04-25T18:53:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.104+0000 7f8fd9400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd41031e0 0x7f8fd41a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.104+0000 7f8fd9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd4103b00 0x7f8fd41a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.104+0000 7f8fd9400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd410c520 0x7f8fd419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.104+0000 7f8fd9400640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd4112200 con 0x7f8fd410c520 2024-04-25T18:53:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.104+0000 7f8fd9400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd4112080 con 0x7f8fd4103b00 2024-04-25T18:53:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.104+0000 7f8fd9400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd4112380 con 0x7f8fd41031e0 2024-04-25T18:53:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.104+0000 7f8fd2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd41031e0 0x7f8fd41a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.104+0000 7f8fd2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd41031e0 0x7f8fd41a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:35492/0 (socket says 172.21.15.112:35492) 2024-04-25T18:53:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.104+0000 7f8fd2800640 1 -- 172.21.15.112:0/1829133898 learned_addr learned my addr 172.21.15.112:0/1829133898 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.104+0000 7f8fd3200640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd410c520 0x7f8fd419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.104+0000 7f8fc9e00640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd4103b00 0x7f8fd41a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.105+0000 7f8fd3200640 1 -- 172.21.15.112:0/1829133898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd41031e0 msgr2=0x7f8fd41a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.105+0000 7f8fd3200640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd41031e0 0x7f8fd41a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.105+0000 7f8fd3200640 1 -- 172.21.15.112:0/1829133898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd4103b00 msgr2=0x7f8fd41a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.105+0000 7f8fd3200640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd4103b00 0x7f8fd41a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.105+0000 7f8fd3200640 1 -- 172.21.15.112:0/1829133898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd419af70 con 0x7f8fd410c520 2024-04-25T18:53:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.105+0000 7f8fd2800640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd41031e0 0x7f8fd41a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.105+0000 7f8fd3200640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd410c520 0x7f8fd419a6b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc02af00 tx=0x7f8fcc02fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.106+0000 7f8fcbe00640 1 -- 172.21.15.112:0/1829133898 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc033040 con 0x7f8fd410c520 2024-04-25T18:53:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.106+0000 7f8fcbe00640 1 -- 172.21.15.112:0/1829133898 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fcc02e7a0 con 0x7f8fd410c520 2024-04-25T18:53:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.106+0000 7f8fcbe00640 1 -- 172.21.15.112:0/1829133898 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc042600 con 0x7f8fd410c520 2024-04-25T18:53:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.106+0000 7f8fd9400640 1 -- 172.21.15.112:0/1829133898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd419b200 con 0x7f8fd410c520 2024-04-25T18:53:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.106+0000 7f8fd9400640 1 -- 172.21.15.112:0/1829133898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd4103f80 con 0x7f8fd410c520 2024-04-25T18:53:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.110+0000 7f8fcbe00640 1 -- 172.21.15.112:0/1829133898 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f8fcc04a060 con 0x7f8fd410c520 2024-04-25T18:53:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.110+0000 7f8fdc218740 1 Processor -- start 2024-04-25T18:53:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.110+0000 7f8fdc218740 1 -- start start 2024-04-25T18:53:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 0x56414432ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432b140 0x5641443354c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564144335a00 0x564144337df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fdc218740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564144290900 con 0x56414432a820 2024-04-25T18:53:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fdc218740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564144236ef0 con 0x564144335a00 2024-04-25T18:53:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fdc218740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56414428c8c0 con 0x56414432b140 2024-04-25T18:53:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fd2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432b140 0x5641443354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fd2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432b140 0x5641443354c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:35494/0 (socket says 172.21.15.112:35494) 2024-04-25T18:53:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fd2800640 1 -- 172.21.15.112:0/2957870553 learned_addr learned my addr 172.21.15.112:0/2957870553 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fc9e00640 1 --2- 172.21.15.112:0/2957870553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 0x56414432ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fc9e00640 1 --2- 172.21.15.112:0/2957870553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564144335a00 0x564144337df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fd2800640 1 -- 172.21.15.112:0/2957870553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564144335a00 msgr2=0x564144337df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fd2800640 1 --2- 172.21.15.112:0/2957870553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564144335a00 0x564144337df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fd2800640 1 -- 172.21.15.112:0/2957870553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 msgr2=0x56414432ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fd2800640 1 --2- 172.21.15.112:0/2957870553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 0x56414432ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.111+0000 7f8fd2800640 1 -- 172.21.15.112:0/2957870553 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564143fa3210 con 0x56414432b140 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.112+0000 7f8fd2800640 1 --2- 172.21.15.112:0/2957870553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432b140 0x5641443354c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4011650 tx=0x7f8fc4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b52a59130b859b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.112+0000 7f8fcbe00640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8fb80a3dc0 0x7f8fb80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.112+0000 7f8fc3400640 1 -- 172.21.15.112:0/2957870553 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc401e070 con 0x56414432b140 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.112+0000 7f8fc3400640 1 -- 172.21.15.112:0/2957870553 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc400fb70 con 0x56414432b140 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.112+0000 7f8fc3400640 1 -- 172.21.15.112:0/2957870553 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4019c60 con 0x56414432b140 2024-04-25T18:53:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.112+0000 7f8fcbe00640 1 -- 172.21.15.112:0/1829133898 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7f8fcc0f12a0 con 0x7f8fd410c520 2024-04-25T18:53:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.113+0000 7f8fc9e00640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8fb80a3dc0 0x7f8fb80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.113+0000 7f8fdc218740 1 -- 172.21.15.112:0/2957870553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432b140 msgr2=0x5641443354c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.113+0000 7f8fdc218740 1 --2- 172.21.15.112:0/2957870553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432b140 0x5641443354c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4011650 tx=0x7f8fc4009f90 comp rx=0 tx=0).stop 2024-04-25T18:53:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.113+0000 7f8fdc218740 1 -- 172.21.15.112:0/2957870553 shutdown_connections 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.113+0000 7f8fdc218740 1 --2- 172.21.15.112:0/2957870553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564144335a00 0x564144337df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.113+0000 7f8fdc218740 1 --2- 172.21.15.112:0/2957870553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432b140 0x5641443354c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.113+0000 7f8fdc218740 1 --2- 172.21.15.112:0/2957870553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 0x56414432ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.113+0000 7f8fdc218740 1 -- 172.21.15.112:0/2957870553 >> 172.21.15.112:0/2957870553 conn(0x564144326d30 msgr2=0x564144327110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.113+0000 7f8fdc218740 1 -- 172.21.15.112:0/2957870553 shutdown_connections 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.113+0000 7f8fdc218740 1 -- 172.21.15.112:0/2957870553 wait complete. 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.113+0000 7f8fc9e00640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8fb80a3dc0 0x7f8fb80a6280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc005f60 tx=0x7f8fbc005f90 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.114+0000 7f8fdc218740 1 -- start start 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.114+0000 7f8fdc218740 1 Processor -- start 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.114+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 0x564144332a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.114+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432b140 0x564144332f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564144335a00 0x564144333490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fdc218740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564144338630 con 0x56414432b140 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fdc218740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56414424a310 con 0x564144335a00 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fdc218740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5641443387b0 con 0x56414432a820 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fdc218740 10 client.0 _ll_register_callbacks cb 0x564144218cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432b140 0x564144332f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fd2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 0x564144332a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432b140 0x564144332f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50426/0 (socket says 172.21.15.112:50426) 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fd2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 0x564144332a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:35502/0 (socket says 172.21.15.112:35502) 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fc9e00640 1 -- 172.21.15.112:0/3320683062 learned_addr learned my addr 172.21.15.112:0/3320683062 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fd3200640 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564144335a00 0x564144333490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fd3200640 1 -- 172.21.15.112:0/3320683062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 msgr2=0x564144332a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fd3200640 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 0x564144332a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fd3200640 1 -- 172.21.15.112:0/3320683062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432b140 msgr2=0x564144332f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fd3200640 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432b140 0x564144332f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fd3200640 1 -- 172.21.15.112:0/3320683062 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414406f620 con 0x564144335a00 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fd2800640 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 0x564144332a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.115+0000 7f8fc9e00640 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432b140 0x564144332f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.116+0000 7f8fd3200640 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564144335a00 0x564144333490 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0ee990 tx=0x7f8fcc0bcea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b5cb1ef17dac484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.116+0000 7f8fb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.116+0000 7f8fb7e00640 1 -- 172.21.15.112:0/3320683062 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0c5bc0 con 0x564144335a00 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.116+0000 7f8fb7e00640 1 -- 172.21.15.112:0/3320683062 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7f8fcc0c1120 con 0x564144335a00 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.116+0000 7f8fdc218740 1 -- 172.21.15.112:0/3320683062 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc400fb70 con 0x564144335a00 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.116+0000 7f8fb7e00640 1 -- 172.21.15.112:0/3320683062 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8fcc0bd270 con 0x564144335a00 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.116+0000 7f8fdc218740 10 client.25784 fetch_fsmap learned FSMap version 36 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.116+0000 7f8fdc218740 1 -- 172.21.15.112:0/3320683062 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564144338630 con 0x564144335a00 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.117+0000 7f8fb7e00640 1 -- 172.21.15.112:0/3320683062 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc400fb70 con 0x564144335a00 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.117+0000 7f8fb7e00640 1 -- 172.21.15.112:0/3320683062 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f8fcc0b8fc0 con 0x564144335a00 2024-04-25T18:53:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.117+0000 7f8fdc218740 10 client.25784 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:53:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.117+0000 7f8fdc218740 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:53:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.117+0000 7f8fdc218740 20 client.25784 populate_metadata read hostname 'smithi112' 2024-04-25T18:53:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.117+0000 7f8fdc218740 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564144345d40 0x564144348150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.117+0000 7f8fdc218740 4 client.25784 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:53:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.117+0000 7f8fdc218740 1 -- 172.21.15.112:0/3320683062 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641442a48b0 con 0x564144345d40 2024-04-25T18:53:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.117+0000 7f8fd2800640 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564144345d40 0x564144348150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.118+0000 7f8fd2800640 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564144345d40 0x564144348150 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a5ba930663edfb server_cookie=7cdd7544eb07fd45 in_seq=0 out_seq=0 2024-04-25T18:53:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.118+0000 7f8fb7e00640 10 client.25784 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.118+0000 7f8fb7e00640 1 -- 172.21.15.112:0/3320683062 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641442a48b0 con 0x564144345d40 2024-04-25T18:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.118+0000 7f8fb7e00640 10 client.25784 handle_command_reply: tid=0 2024-04-25T18:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 1 client.25784 shutdown 2024-04-25T18:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 1 -- 172.21.15.112:0/3320683062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564144335a00 msgr2=0x564144333490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564144335a00 0x564144333490 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0ee990 tx=0x7f8fcc0bcea0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 1 -- 172.21.15.112:0/3320683062 shutdown_connections 2024-04-25T18:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564144335a00 0x564144333490 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564144345d40 0x564144348150 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432b140 0x564144332f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 1 --2- 172.21.15.112:0/3320683062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 0x564144332a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 1 -- 172.21.15.112:0/3320683062 >> 172.21.15.112:0/3320683062 conn(0x564144326d30 msgr2=0x56414432cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fb7e00640 0 client.25784 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 1 -- 172.21.15.112:0/3320683062 shutdown_connections 2024-04-25T18:53:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 1 -- 172.21.15.112:0/3320683062 wait complete. 2024-04-25T18:53:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.119+0000 7f8fdc218740 20 client.25784 trim_cache size 0 max 16384 2024-04-25T18:53:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.142+0000 7f8fdc218740 1 Processor -- start 2024-04-25T18:53:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.142+0000 7f8fdc218740 1 -- start start 2024-04-25T18:53:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 0x5641443414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 0x56414434da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564144335a00 0x56414434dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fdc218740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41ae240 con 0x56414432a820 2024-04-25T18:53:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fdc218740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41abb10 con 0x56414432b140 2024-04-25T18:53:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fdc218740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41a99a0 con 0x564144335a00 2024-04-25T18:53:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fd2800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 0x5641443414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 0x56414434da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fd2800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 0x5641443414f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50432/0 (socket says 172.21.15.112:50432) 2024-04-25T18:53:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fd2800640 1 -- 172.21.15.112:0/3132939967 learned_addr learned my addr 172.21.15.112:0/3132939967 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fd3200640 1 --2- 172.21.15.112:0/3132939967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564144335a00 0x56414434dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fd2800640 1 -- 172.21.15.112:0/3132939967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564144335a00 msgr2=0x56414434dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fd2800640 1 --2- 172.21.15.112:0/3132939967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564144335a00 0x56414434dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fd2800640 1 -- 172.21.15.112:0/3132939967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 msgr2=0x56414434da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fd2800640 1 --2- 172.21.15.112:0/3132939967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 0x56414434da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.143+0000 7f8fd2800640 1 -- 172.21.15.112:0/3132939967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564144337df0 con 0x56414432a820 2024-04-25T18:53:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.144+0000 7f8fd2800640 1 --2- 172.21.15.112:0/3132939967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 0x5641443414f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40115a0 tx=0x7f8fc4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd76494bebb67736 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.144+0000 7f8fc2a00640 1 -- 172.21.15.112:0/3132939967 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc402f070 con 0x56414432a820 2024-04-25T18:53:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.144+0000 7f8fc2a00640 1 -- 172.21.15.112:0/3132939967 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc4029070 con 0x56414432a820 2024-04-25T18:53:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.144+0000 7f8fc2a00640 1 -- 172.21.15.112:0/3132939967 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc401e040 con 0x56414432a820 2024-04-25T18:53:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.145+0000 7f8fdc218740 1 -- 172.21.15.112:0/3132939967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 msgr2=0x5641443414f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.145+0000 7f8fdc218740 1 --2- 172.21.15.112:0/3132939967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 0x5641443414f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40115a0 tx=0x7f8fc4009780 comp rx=0 tx=0).stop 2024-04-25T18:53:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.145+0000 7f8fdc218740 1 -- 172.21.15.112:0/3132939967 shutdown_connections 2024-04-25T18:53:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.145+0000 7f8fdc218740 1 --2- 172.21.15.112:0/3132939967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564144335a00 0x56414434dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.145+0000 7f8fdc218740 1 --2- 172.21.15.112:0/3132939967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 0x56414434da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.145+0000 7f8fdc218740 1 --2- 172.21.15.112:0/3132939967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56414432a820 0x5641443414f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.145+0000 7f8fdc218740 1 -- 172.21.15.112:0/3132939967 >> 172.21.15.112:0/3132939967 conn(0x564144326d30 msgr2=0x564144327110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.145+0000 7f8fdc218740 1 -- 172.21.15.112:0/3132939967 shutdown_connections 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.146+0000 7f8fdc218740 1 -- 172.21.15.112:0/3132939967 wait complete. 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.146+0000 7f8fdc218740 1 -- start start 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.146+0000 7f8fdc218740 1 Processor -- start 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 0x56414433daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 0x56414433afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fdc218740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564144335a00 0x56414433b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fdc218740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41a9ff0 con 0x564144335a00 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fdc218740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41ab440 con 0x56414432b140 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fdc218740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd41aa6a0 con 0x56414432a820 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fdc218740 10 client.0 _ll_register_callbacks cb 0x7f8fcc0bd0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 0x56414433afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fd2800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 0x56414433daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fd3200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564144335a00 0x56414433b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 0x56414433afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48170/0 (socket says 172.21.15.112:48170) 2024-04-25T18:53:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.147+0000 7f8fc9e00640 1 -- 172.21.15.112:0/4018099158 learned_addr learned my addr 172.21.15.112:0/4018099158 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.148+0000 7f8fc9e00640 1 -- 172.21.15.112:0/4018099158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 msgr2=0x56414433daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.148+0000 7f8fc9e00640 1 --2- 172.21.15.112:0/4018099158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 0x56414433daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.148+0000 7f8fc9e00640 1 -- 172.21.15.112:0/4018099158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564144335a00 msgr2=0x56414433b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.148+0000 7f8fc9e00640 1 --2- 172.21.15.112:0/4018099158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564144335a00 0x56414433b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.148+0000 7f8fc9e00640 1 -- 172.21.15.112:0/4018099158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414433ba30 con 0x56414432b140 2024-04-25T18:53:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.148+0000 7f8fc9e00640 1 --2- 172.21.15.112:0/4018099158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 0x56414433afb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x564144332270 tx=0x7f8fbc016040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110101e6ee4414de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.148+0000 7f8fb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.148+0000 7f8fb7e00640 1 -- 172.21.15.112:0/4018099158 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc037070 con 0x56414432b140 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.148+0000 7f8fdc218740 1 -- 172.21.15.112:0/4018099158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc4029070 con 0x56414432b140 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fb7e00640 1 -- 172.21.15.112:0/4018099158 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7f8fbc033430 con 0x56414432b140 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fb7e00640 1 -- 172.21.15.112:0/4018099158 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8fbc005020 con 0x56414432b140 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fdc218740 10 client.25790 fetch_fsmap learned FSMap version 36 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fdc218740 1 -- 172.21.15.112:0/4018099158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5641443ac140 con 0x56414432b140 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fb7e00640 1 -- 172.21.15.112:0/4018099158 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc4029070 con 0x56414432b140 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fb7e00640 1 -- 172.21.15.112:0/4018099158 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f8fbc024070 con 0x56414432b140 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fdc218740 10 client.25790 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fdc218740 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fdc218740 20 client.25790 populate_metadata read hostname 'smithi112' 2024-04-25T18:53:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fdc218740 1 --2- 172.21.15.112:0/4018099158 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5641443ac920 0x5641443aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fdc218740 4 client.25790 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:53:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.149+0000 7f8fdc218740 1 -- 172.21.15.112:0/4018099158 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56414429e320 con 0x5641443ac920 2024-04-25T18:53:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.150+0000 7f8fd2800640 1 --2- 172.21.15.112:0/4018099158 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5641443ac920 0x5641443aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.150+0000 7f8fd2800640 1 --2- 172.21.15.112:0/4018099158 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5641443ac920 0x5641443aed40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfbb22062e83fb3d server_cookie=26fa38cbcf65389a in_seq=0 out_seq=0 2024-04-25T18:53:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.150+0000 7f8fb7e00640 10 client.25790 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.164+0000 7f8fb7e00640 1 -- 172.21.15.112:0/4018099158 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56414429e320 con 0x5641443ac920 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.164+0000 7f8fb7e00640 10 client.25790 handle_command_reply: tid=0 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.164+0000 7f8fdc218740 1 client.25790 shutdown 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.164+0000 7f8fdc218740 1 -- 172.21.15.112:0/4018099158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 msgr2=0x56414433afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.164+0000 7f8fdc218740 1 --2- 172.21.15.112:0/4018099158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 0x56414433afb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x564144332270 tx=0x7f8fbc016040 comp rx=0 tx=0).stop 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.164+0000 7f8fdc218740 1 -- 172.21.15.112:0/4018099158 shutdown_connections 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.164+0000 7f8fdc218740 1 --2- 172.21.15.112:0/4018099158 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5641443ac920 0x5641443aed40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.164+0000 7f8fdc218740 1 --2- 172.21.15.112:0/4018099158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564144335a00 0x56414433b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.164+0000 7f8fdc218740 1 --2- 172.21.15.112:0/4018099158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56414432b140 0x56414433afb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.165+0000 7f8fdc218740 1 --2- 172.21.15.112:0/4018099158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56414432a820 0x56414433daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.165+0000 7f8fdc218740 1 -- 172.21.15.112:0/4018099158 >> 172.21.15.112:0/4018099158 conn(0x564144326d30 msgr2=0x56414432c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.165+0000 7f8fb7e00640 0 client.25790 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.165+0000 7f8fdc218740 1 -- 172.21.15.112:0/4018099158 shutdown_connections 2024-04-25T18:53:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.165+0000 7f8fdc218740 1 -- 172.21.15.112:0/4018099158 wait complete. 2024-04-25T18:53:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.165+0000 7f8fdc218740 20 client.25790 trim_cache size 0 max 16384 2024-04-25T18:53:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:53:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:53:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:53:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:53:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 -- 172.21.15.112:0/1829133898 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8fb80a3dc0 msgr2=0x7f8fb80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8fb80a3dc0 0x7f8fb80a6280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc005f60 tx=0x7f8fbc005f90 comp rx=0 tx=0).stop 2024-04-25T18:53:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 -- 172.21.15.112:0/1829133898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd410c520 msgr2=0x7f8fd419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd410c520 0x7f8fd419a6b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc02af00 tx=0x7f8fcc02fbe0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 -- 172.21.15.112:0/1829133898 shutdown_connections 2024-04-25T18:53:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8fb80a3dc0 0x7f8fb80a6280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fd410c520 0x7f8fd419a6b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fd4103b00 0x7f8fd41a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 --2- 172.21.15.112:0/1829133898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd41031e0 0x7f8fd41a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 -- 172.21.15.112:0/1829133898 >> 172.21.15.112:0/1829133898 conn(0x7f8fd4077fd0 msgr2=0x7f8fd4108390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 -- 172.21.15.112:0/1829133898 shutdown_connections 2024-04-25T18:53:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:33.166+0000 7f8fd9400640 1 -- 172.21.15.112:0/1829133898 wait complete. 2024-04-25T18:53:34.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:34 smithi171 ceph-mon[33266]: pgmap v190: 97 pgs: 97 active+clean; 55 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-04-25T18:53:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:34 smithi161 ceph-mon[32609]: pgmap v190: 97 pgs: 97 active+clean; 55 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-04-25T18:53:34.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:34 smithi112 ceph-mon[27864]: pgmap v190: 97 pgs: 97 active+clean; 55 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-04-25T18:53:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:36 smithi171 ceph-mon[33266]: pgmap v191: 97 pgs: 97 active+clean; 55 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-04-25T18:53:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:36 smithi161 ceph-mon[32609]: pgmap v191: 97 pgs: 97 active+clean; 55 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-04-25T18:53:36.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:36 smithi112 ceph-mon[27864]: pgmap v191: 97 pgs: 97 active+clean; 55 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-04-25T18:53:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:38 smithi171 ceph-mon[33266]: pgmap v192: 97 pgs: 97 active+clean; 47 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.2 MiB/s wr, 371 op/s 2024-04-25T18:53:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:38 smithi161 ceph-mon[32609]: pgmap v192: 97 pgs: 97 active+clean; 47 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.2 MiB/s wr, 371 op/s 2024-04-25T18:53:38.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:38 smithi112 ceph-mon[27864]: pgmap v192: 97 pgs: 97 active+clean; 47 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.2 MiB/s wr, 371 op/s 2024-04-25T18:53:40.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:39 smithi171 ceph-mon[33266]: pgmap v193: 97 pgs: 97 active+clean; 47 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 245 op/s 2024-04-25T18:53:40.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:39 smithi161 ceph-mon[32609]: pgmap v193: 97 pgs: 97 active+clean; 47 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 245 op/s 2024-04-25T18:53:40.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:39 smithi112 ceph-mon[27864]: pgmap v193: 97 pgs: 97 active+clean; 47 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 245 op/s 2024-04-25T18:53:42.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:41 smithi171 ceph-mon[33266]: pgmap v194: 97 pgs: 97 active+clean; 50 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.2 MiB/s wr, 362 op/s 2024-04-25T18:53:42.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:41 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:41 smithi161 ceph-mon[32609]: pgmap v194: 97 pgs: 97 active+clean; 50 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.2 MiB/s wr, 362 op/s 2024-04-25T18:53:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:41 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:41 smithi112 ceph-mon[27864]: pgmap v194: 97 pgs: 97 active+clean; 50 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.2 MiB/s wr, 362 op/s 2024-04-25T18:53:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:41 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:43.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:42 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:43.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:42 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:43.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:42 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:53:43.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:42 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:43.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:42 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:43.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:42 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:53:43.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:42 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:43.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:42 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:43.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:42 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:53:44.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:44 smithi171 ceph-mon[33266]: pgmap v195: 97 pgs: 97 active+clean; 50 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:44.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:44 smithi161 ceph-mon[32609]: pgmap v195: 97 pgs: 97 active+clean; 50 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:44.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:44 smithi112 ceph-mon[27864]: pgmap v195: 97 pgs: 97 active+clean; 50 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.320+0000 7fa91da00640 1 -- 172.21.15.112:0/3313023604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9181092c0 msgr2=0x7fa918110e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.320+0000 7fa91da00640 1 --2- 172.21.15.112:0/3313023604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9181092c0 0x7fa918110e90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa910009900 tx=0x7fa91002a3f0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.321+0000 7fa91da00640 1 -- 172.21.15.112:0/3313023604 shutdown_connections 2024-04-25T18:53:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.321+0000 7fa91da00640 1 --2- 172.21.15.112:0/3313023604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9181092c0 0x7fa918110e90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.321+0000 7fa91da00640 1 --2- 172.21.15.112:0/3313023604 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa918104ba0 0x7fa918108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.321+0000 7fa91da00640 1 --2- 172.21.15.112:0/3313023604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9181041f0 0x7fa9181045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.321+0000 7fa91da00640 1 -- 172.21.15.112:0/3313023604 >> 172.21.15.112:0/3313023604 conn(0x7fa9180ffa30 msgr2=0x7fa918101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.321+0000 7fa91da00640 1 -- 172.21.15.112:0/3313023604 shutdown_connections 2024-04-25T18:53:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.321+0000 7fa91da00640 1 -- 172.21.15.112:0/3313023604 wait complete. 2024-04-25T18:53:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.322+0000 7fa91da00640 1 Processor -- start 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.322+0000 7fa91da00640 1 -- start start 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.322+0000 7fa91da00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9181041f0 0x7fa9181a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.322+0000 7fa91da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9181092c0 0x7fa9181a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.322+0000 7fa91da00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa91819a6c0 0x7fa91819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.322+0000 7fa91da00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa918114280 con 0x7fa91819a6c0 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.322+0000 7fa91da00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa918114100 con 0x7fa9181092c0 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.322+0000 7fa91da00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa918114400 con 0x7fa9181041f0 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.323+0000 7fa90f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9181092c0 0x7fa9181a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.323+0000 7fa90f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9181092c0 0x7fa9181a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:33530/0 (socket says 172.21.15.112:33530) 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.323+0000 7fa917e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9181041f0 0x7fa9181a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.323+0000 7fa90f400640 1 -- 172.21.15.112:0/3688837647 learned_addr learned my addr 172.21.15.112:0/3688837647 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.323+0000 7fa91d000640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa91819a6c0 0x7fa91819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.323+0000 7fa90f400640 1 -- 172.21.15.112:0/3688837647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9181041f0 msgr2=0x7fa9181a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.323+0000 7fa90f400640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9181041f0 0x7fa9181a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.323+0000 7fa90f400640 1 -- 172.21.15.112:0/3688837647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa91819a6c0 msgr2=0x7fa91819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.323+0000 7fa90f400640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa91819a6c0 0x7fa91819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.323+0000 7fa90f400640 1 -- 172.21.15.112:0/3688837647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91819b2c0 con 0x7fa9181092c0 2024-04-25T18:53:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.323+0000 7fa917e00640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9181041f0 0x7fa9181a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.324+0000 7fa90f400640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9181092c0 0x7fa9181a0af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa90000e9f0 tx=0x7fa90000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.324+0000 7fa915600640 1 -- 172.21.15.112:0/3688837647 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90000ccc0 con 0x7fa9181092c0 2024-04-25T18:53:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.324+0000 7fa915600640 1 -- 172.21.15.112:0/3688837647 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9000054d0 con 0x7fa9181092c0 2024-04-25T18:53:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.324+0000 7fa915600640 1 -- 172.21.15.112:0/3688837647 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900010430 con 0x7fa9181092c0 2024-04-25T18:53:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.325+0000 7fa91da00640 1 -- 172.21.15.112:0/3688837647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9181a7350 con 0x7fa9181092c0 2024-04-25T18:53:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.325+0000 7fa91da00640 1 -- 172.21.15.112:0/3688837647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9181a7700 con 0x7fa9181092c0 2024-04-25T18:53:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.329+0000 7fa915600640 1 -- 172.21.15.112:0/3688837647 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fa9000046c0 con 0x7fa9181092c0 2024-04-25T18:53:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.329+0000 7fa921df1740 1 Processor -- start 2024-04-25T18:53:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.329+0000 7fa921df1740 1 -- start start 2024-04-25T18:53:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.329+0000 7fa921df1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 0x5556e774bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.329+0000 7fa921df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774c2d0 0x5556e7756650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.329+0000 7fa921df1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e7756b90 0x5556e7758f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.329+0000 7fa921df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5556e76b1900 con 0x5556e774c2d0 2024-04-25T18:53:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.329+0000 7fa921df1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5556e7657ef0 con 0x5556e7756b90 2024-04-25T18:53:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.329+0000 7fa921df1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5556e76ad8c0 con 0x5556e774b9b0 2024-04-25T18:53:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa90f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e7756b90 0x5556e7758f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa91d000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 0x5556e774bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa917e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774c2d0 0x5556e7756650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa91d000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 0x5556e774bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:36762/0 (socket says 172.21.15.112:36762) 2024-04-25T18:53:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa90f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e7756b90 0x5556e7758f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:33544/0 (socket says 172.21.15.112:33544) 2024-04-25T18:53:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa91d000640 1 -- 172.21.15.112:0/1380623925 learned_addr learned my addr 172.21.15.112:0/1380623925 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa91d000640 1 -- 172.21.15.112:0/1380623925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e7756b90 msgr2=0x5556e7758f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa91d000640 1 --2- 172.21.15.112:0/1380623925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e7756b90 0x5556e7758f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa91d000640 1 -- 172.21.15.112:0/1380623925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774c2d0 msgr2=0x5556e7756650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa91d000640 1 --2- 172.21.15.112:0/1380623925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774c2d0 0x5556e7756650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa91d000640 1 -- 172.21.15.112:0/1380623925 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556e73c4210 con 0x5556e774b9b0 2024-04-25T18:53:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.330+0000 7fa91d000640 1 --2- 172.21.15.112:0/1380623925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 0x5556e774bd90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa91002a9e0 tx=0x7fa910005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af464e53a46a1f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.331+0000 7fa915600640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa8fc0a3d90 0x7fa8fc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.331+0000 7fa90d600640 1 -- 172.21.15.112:0/1380623925 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910040070 con 0x5556e774b9b0 2024-04-25T18:53:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.331+0000 7fa90d600640 1 -- 172.21.15.112:0/1380623925 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9100044c0 con 0x5556e774b9b0 2024-04-25T18:53:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.331+0000 7fa90d600640 1 -- 172.21.15.112:0/1380623925 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910035040 con 0x5556e774b9b0 2024-04-25T18:53:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.331+0000 7fa915600640 1 -- 172.21.15.112:0/3688837647 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7fa9000d0ae0 con 0x7fa9181092c0 2024-04-25T18:53:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.331+0000 7fa917e00640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa8fc0a3d90 0x7fa8fc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.332+0000 7fa921df1740 1 -- 172.21.15.112:0/1380623925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 msgr2=0x5556e774bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.332+0000 7fa917e00640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa8fc0a3d90 0x7fa8fc0a6250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5556e774cf50 tx=0x7fa908009500 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.332+0000 7fa921df1740 1 --2- 172.21.15.112:0/1380623925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 0x5556e774bd90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa91002a9e0 tx=0x7fa910005be0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.332+0000 7fa921df1740 1 -- 172.21.15.112:0/1380623925 shutdown_connections 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.332+0000 7fa921df1740 1 --2- 172.21.15.112:0/1380623925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e7756b90 0x5556e7758f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.332+0000 7fa921df1740 1 --2- 172.21.15.112:0/1380623925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774c2d0 0x5556e7756650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.332+0000 7fa921df1740 1 --2- 172.21.15.112:0/1380623925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 0x5556e774bd90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.332+0000 7fa921df1740 1 -- 172.21.15.112:0/1380623925 >> 172.21.15.112:0/1380623925 conn(0x5556e7747ec0 msgr2=0x5556e77482a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.332+0000 7fa921df1740 1 -- 172.21.15.112:0/1380623925 shutdown_connections 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.332+0000 7fa921df1740 1 -- 172.21.15.112:0/1380623925 wait complete. 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.333+0000 7fa921df1740 1 -- start start 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.333+0000 7fa921df1740 1 Processor -- start 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.333+0000 7fa921df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774b9b0 0x5556e7753a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa921df1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 0x5556e7753fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa921df1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 0x5556e7754510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa921df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5556e7759760 con 0x5556e774b9b0 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa921df1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5556e766b310 con 0x5556e774c2d0 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa921df1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5556e77598e0 con 0x5556e7756b90 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa921df1740 10 client.0 _ll_register_callbacks cb 0x5556e7639cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa917e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774b9b0 0x5556e7753a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa90f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 0x5556e7753fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa91d000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 0x5556e7754510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa917e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774b9b0 0x5556e7753a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57354/0 (socket says 172.21.15.112:57354) 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa917e00640 1 -- 172.21.15.112:0/1936535868 learned_addr learned my addr 172.21.15.112:0/1936535868 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa91d000640 1 -- 172.21.15.112:0/1936535868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 msgr2=0x5556e7753fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa91d000640 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 0x5556e7753fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa91d000640 1 -- 172.21.15.112:0/1936535868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774b9b0 msgr2=0x5556e7753a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa91d000640 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774b9b0 0x5556e7753a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa91d000640 1 -- 172.21.15.112:0/1936535868 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556e7490620 con 0x5556e7756b90 2024-04-25T18:53:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.334+0000 7fa90f400640 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 0x5556e7753fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.335+0000 7fa91d000640 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 0x5556e7754510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa910005ab0 tx=0x7fa910004f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11c0e31f55fb1978 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.335+0000 7fa906000640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:53:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.335+0000 7fa906000640 1 -- 172.21.15.112:0/1936535868 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910040030 con 0x5556e7756b90 2024-04-25T18:53:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.335+0000 7fa906000640 1 -- 172.21.15.112:0/1936535868 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7fa910005000 con 0x5556e7756b90 2024-04-25T18:53:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.335+0000 7fa921df1740 1 -- 172.21.15.112:0/1936535868 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556e76c58b0 con 0x5556e7756b90 2024-04-25T18:53:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.335+0000 7fa906000640 1 -- 172.21.15.112:0/1936535868 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa910056050 con 0x5556e7756b90 2024-04-25T18:53:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.335+0000 7fa921df1740 10 client.25831 fetch_fsmap learned FSMap version 36 2024-04-25T18:53:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.335+0000 7fa921df1740 1 -- 172.21.15.112:0/1936535868 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5556e7759760 con 0x5556e7756b90 2024-04-25T18:53:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.335+0000 7fa906000640 1 -- 172.21.15.112:0/1936535868 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5556e76c58b0 con 0x5556e7756b90 2024-04-25T18:53:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.335+0000 7fa906000640 1 -- 172.21.15.112:0/1936535868 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fa9100314a0 con 0x5556e7756b90 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.336+0000 7fa921df1740 10 client.25831 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.336+0000 7fa921df1740 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.336+0000 7fa921df1740 20 client.25831 populate_metadata read hostname 'smithi112' 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.336+0000 7fa921df1740 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5556e7766ae0 0x5556e7768ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.336+0000 7fa921df1740 4 client.25831 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.336+0000 7fa921df1740 1 -- 172.21.15.112:0/1936535868 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556e76bf320 con 0x5556e7766ae0 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.336+0000 7fa917e00640 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5556e7766ae0 0x5556e7768ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.336+0000 7fa917e00640 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5556e7766ae0 0x5556e7768ed0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703c268f3958bb server_cookie=cfe6a44b1ec1446c in_seq=0 out_seq=0 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.336+0000 7fa906000640 10 client.25831 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.337+0000 7fa906000640 1 -- 172.21.15.112:0/1936535868 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556e76bf320 con 0x5556e7766ae0 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.337+0000 7fa906000640 10 client.25831 handle_command_reply: tid=0 2024-04-25T18:53:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.337+0000 7fa921df1740 1 client.25831 shutdown 2024-04-25T18:53:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa921df1740 1 -- 172.21.15.112:0/1936535868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 msgr2=0x5556e7754510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa921df1740 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 0x5556e7754510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa910005ab0 tx=0x7fa910004f00 comp rx=0 tx=0).stop 2024-04-25T18:53:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa921df1740 1 -- 172.21.15.112:0/1936535868 shutdown_connections 2024-04-25T18:53:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa921df1740 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 0x5556e7754510 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa921df1740 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 0x5556e7753fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa921df1740 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5556e7766ae0 0x5556e7768ed0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa921df1740 1 --2- 172.21.15.112:0/1936535868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774b9b0 0x5556e7753a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa921df1740 1 -- 172.21.15.112:0/1936535868 >> 172.21.15.112:0/1936535868 conn(0x5556e7747ec0 msgr2=0x5556e774dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa906000640 0 client.25831 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa921df1740 1 -- 172.21.15.112:0/1936535868 shutdown_connections 2024-04-25T18:53:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa921df1740 1 -- 172.21.15.112:0/1936535868 wait complete. 2024-04-25T18:53:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.338+0000 7fa921df1740 20 client.25831 trim_cache size 0 max 16384 2024-04-25T18:53:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa921df1740 1 Processor -- start 2024-04-25T18:53:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa921df1740 1 -- start start 2024-04-25T18:53:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa921df1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774b9b0 0x5556e7762620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa921df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774c2d0 0x5556e7754330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa921df1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 0x5556e776e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa921df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181b2540 con 0x5556e774c2d0 2024-04-25T18:53:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa921df1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181afe10 con 0x5556e774b9b0 2024-04-25T18:53:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa921df1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181adc60 con 0x5556e7756b90 2024-04-25T18:53:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa917e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774b9b0 0x5556e7762620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa917e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774b9b0 0x5556e7762620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:33562/0 (socket says 172.21.15.112:33562) 2024-04-25T18:53:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa917e00640 1 -- 172.21.15.112:0/2465952269 learned_addr learned my addr 172.21.15.112:0/2465952269 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.359+0000 7fa91d000640 1 --2- 172.21.15.112:0/2465952269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 0x5556e776e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.360+0000 7fa90f400640 1 --2- 172.21.15.112:0/2465952269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774c2d0 0x5556e7754330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.360+0000 7fa91d000640 1 -- 172.21.15.112:0/2465952269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774b9b0 msgr2=0x5556e7762620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.360+0000 7fa91d000640 1 --2- 172.21.15.112:0/2465952269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774b9b0 0x5556e7762620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.360+0000 7fa91d000640 1 -- 172.21.15.112:0/2465952269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774c2d0 msgr2=0x5556e7754330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.360+0000 7fa91d000640 1 --2- 172.21.15.112:0/2465952269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774c2d0 0x5556e7754330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.360+0000 7fa91d000640 1 -- 172.21.15.112:0/2465952269 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556e774c100 con 0x5556e7756b90 2024-04-25T18:53:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.360+0000 7fa917e00640 1 --2- 172.21.15.112:0/2465952269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774b9b0 0x5556e7762620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.360+0000 7fa91d000640 1 --2- 172.21.15.112:0/2465952269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 0x5556e776e660 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa9100318c0 tx=0x7fa910004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6bdd60a776b0dc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.361+0000 7fa90cc00640 1 -- 172.21.15.112:0/2465952269 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910040070 con 0x5556e7756b90 2024-04-25T18:53:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.361+0000 7fa90cc00640 1 -- 172.21.15.112:0/2465952269 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa91004d070 con 0x5556e7756b90 2024-04-25T18:53:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.361+0000 7fa90cc00640 1 -- 172.21.15.112:0/2465952269 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910035040 con 0x5556e7756b90 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.361+0000 7fa921df1740 1 -- 172.21.15.112:0/2465952269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 msgr2=0x5556e776e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.361+0000 7fa921df1740 1 --2- 172.21.15.112:0/2465952269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 0x5556e776e660 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa9100318c0 tx=0x7fa910004c60 comp rx=0 tx=0).stop 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.362+0000 7fa921df1740 1 -- 172.21.15.112:0/2465952269 shutdown_connections 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.362+0000 7fa921df1740 1 --2- 172.21.15.112:0/2465952269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e7756b90 0x5556e776e660 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.362+0000 7fa921df1740 1 --2- 172.21.15.112:0/2465952269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e774c2d0 0x5556e7754330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.362+0000 7fa921df1740 1 --2- 172.21.15.112:0/2465952269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774b9b0 0x5556e7762620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.362+0000 7fa921df1740 1 -- 172.21.15.112:0/2465952269 >> 172.21.15.112:0/2465952269 conn(0x5556e7747ec0 msgr2=0x5556e7747020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.362+0000 7fa921df1740 1 -- 172.21.15.112:0/2465952269 shutdown_connections 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.362+0000 7fa921df1740 1 -- 172.21.15.112:0/2465952269 wait complete. 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.362+0000 7fa921df1740 1 -- start start 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.362+0000 7fa921df1740 1 Processor -- start 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa921df1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 0x5556e775bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa921df1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 0x5556e775c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa921df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e7756b90 0x5556e77caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa921df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181ae2d0 con 0x5556e7756b90 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa921df1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181af740 con 0x5556e774c2d0 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa921df1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa9181ae9a0 con 0x5556e774b9b0 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa921df1740 10 client.0 _ll_register_callbacks cb 0x7fa910009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa91d000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e7756b90 0x5556e77caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa90f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 0x5556e775c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa917e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 0x5556e775bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa90f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 0x5556e775c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:33564/0 (socket says 172.21.15.112:33564) 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.363+0000 7fa90f400640 1 -- 172.21.15.112:0/1434635869 learned_addr learned my addr 172.21.15.112:0/1434635869 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa917e00640 1 -- 172.21.15.112:0/1434635869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 msgr2=0x5556e775c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa917e00640 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 0x5556e775c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa917e00640 1 -- 172.21.15.112:0/1434635869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e7756b90 msgr2=0x5556e77caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa917e00640 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e7756b90 0x5556e77caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa917e00640 1 -- 172.21.15.112:0/1434635869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556e77cb410 con 0x5556e774b9b0 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa90f400640 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 0x5556e775c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa917e00640 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 0x5556e775bc80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa908009ed0 tx=0x7fa908017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bbc24d96b3ea388 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa906000640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa906000640 1 -- 172.21.15.112:0/1434635869 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90802b380 con 0x5556e774b9b0 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa921df1740 1 -- 172.21.15.112:0/1434635869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa91004d070 con 0x5556e774b9b0 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa906000640 1 -- 172.21.15.112:0/1434635869 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7fa90802b520 con 0x5556e774b9b0 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.364+0000 7fa906000640 1 -- 172.21.15.112:0/1434635869 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa90802b720 con 0x5556e774b9b0 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.365+0000 7fa921df1740 10 client.25837 fetch_fsmap learned FSMap version 36 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.365+0000 7fa921df1740 1 -- 172.21.15.112:0/1434635869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5556e77cc770 con 0x5556e774b9b0 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.365+0000 7fa906000640 1 -- 172.21.15.112:0/1434635869 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa91004d070 con 0x5556e774b9b0 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.365+0000 7fa906000640 1 -- 172.21.15.112:0/1434635869 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fa90803c530 con 0x5556e774b9b0 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.365+0000 7fa921df1740 10 client.25837 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:53:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.365+0000 7fa921df1740 10 client.25837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:53:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.365+0000 7fa921df1740 20 client.25837 populate_metadata read hostname 'smithi112' 2024-04-25T18:53:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.365+0000 7fa921df1740 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5556e77ccd80 0x5556e77cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.365+0000 7fa921df1740 4 client.25837 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:53:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.365+0000 7fa921df1740 1 -- 172.21.15.112:0/1434635869 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556e76b9190 con 0x5556e77ccd80 2024-04-25T18:53:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.365+0000 7fa90f400640 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5556e77ccd80 0x5556e77cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.366+0000 7fa90f400640 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5556e77ccd80 0x5556e77cf1a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=538601c737a6f671 server_cookie=c32e5784223130da in_seq=0 out_seq=0 2024-04-25T18:53:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.366+0000 7fa906000640 10 client.25837 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.384+0000 7fa906000640 1 -- 172.21.15.112:0/1434635869 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5556e76b9190 con 0x5556e77ccd80 2024-04-25T18:53:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.384+0000 7fa906000640 10 client.25837 handle_command_reply: tid=0 2024-04-25T18:53:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.384+0000 7fa921df1740 1 client.25837 shutdown 2024-04-25T18:53:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa921df1740 1 -- 172.21.15.112:0/1434635869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 msgr2=0x5556e775bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa921df1740 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 0x5556e775bc80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa908009ed0 tx=0x7fa908017000 comp rx=0 tx=0).stop 2024-04-25T18:53:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa921df1740 1 -- 172.21.15.112:0/1434635869 shutdown_connections 2024-04-25T18:53:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa921df1740 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5556e77ccd80 0x5556e77cf1a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa921df1740 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5556e7756b90 0x5556e77caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa921df1740 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556e774c2d0 0x5556e775c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa921df1740 1 --2- 172.21.15.112:0/1434635869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556e774b9b0 0x5556e775bc80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa921df1740 1 -- 172.21.15.112:0/1434635869 >> 172.21.15.112:0/1434635869 conn(0x5556e7747ec0 msgr2=0x5556e7753ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa906000640 0 client.25837 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa921df1740 1 -- 172.21.15.112:0/1434635869 shutdown_connections 2024-04-25T18:53:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa921df1740 1 -- 172.21.15.112:0/1434635869 wait complete. 2024-04-25T18:53:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.385+0000 7fa921df1740 20 client.25837 trim_cache size 0 max 16384 2024-04-25T18:53:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:53:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:53:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:53:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:53:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 -- 172.21.15.112:0/3688837647 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa8fc0a3d90 msgr2=0x7fa8fc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa8fc0a3d90 0x7fa8fc0a6250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5556e774cf50 tx=0x7fa908009500 comp rx=0 tx=0).stop 2024-04-25T18:53:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 -- 172.21.15.112:0/3688837647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9181092c0 msgr2=0x7fa9181a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9181092c0 0x7fa9181a0af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa90000e9f0 tx=0x7fa90000eeb0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 -- 172.21.15.112:0/3688837647 shutdown_connections 2024-04-25T18:53:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa8fc0a3d90 0x7fa8fc0a6250 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa91819a6c0 0x7fa91819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9181092c0 0x7fa9181a0af0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 --2- 172.21.15.112:0/3688837647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9181041f0 0x7fa9181a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 -- 172.21.15.112:0/3688837647 >> 172.21.15.112:0/3688837647 conn(0x7fa9180ffa30 msgr2=0x7fa918100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 -- 172.21.15.112:0/3688837647 shutdown_connections 2024-04-25T18:53:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:46.386+0000 7fa91da00640 1 -- 172.21.15.112:0/3688837647 wait complete. 2024-04-25T18:53:46.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:46 smithi171 ceph-mon[33266]: pgmap v196: 97 pgs: 97 active+clean; 50 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:46.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:46 smithi161 ceph-mon[32609]: pgmap v196: 97 pgs: 97 active+clean; 50 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:46.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:46 smithi112 ceph-mon[27864]: pgmap v196: 97 pgs: 97 active+clean; 50 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-04-25T18:53:48.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:48 smithi171 ceph-mon[33266]: pgmap v197: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.3 MiB/s wr, 359 op/s 2024-04-25T18:53:48.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:48 smithi161 ceph-mon[32609]: pgmap v197: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.3 MiB/s wr, 359 op/s 2024-04-25T18:53:48.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:48 smithi112 ceph-mon[27864]: pgmap v197: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.3 MiB/s wr, 359 op/s 2024-04-25T18:53:50.636 INFO:tasks.workunit.client.0.smithi112.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-04-25T18:53:50.636 INFO:tasks.workunit.client.0.smithi112.stdout:All 50000 operations completed A-OK! 2024-04-25T18:53:50.650 INFO:tasks.workunit.client.0.smithi112.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-04-25T18:53:50.658 INFO:tasks.workunit.client.0.smithi112.stdout:Seed set to 1 2024-04-25T18:53:50.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:50 smithi171 ceph-mon[33266]: pgmap v198: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2024-04-25T18:53:50.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:50 smithi161 ceph-mon[32609]: pgmap v198: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2024-04-25T18:53:50.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:50 smithi112 ceph-mon[27864]: pgmap v198: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2024-04-25T18:53:52.007 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T18:53:52.007 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T18:53:52.010 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T18:53:52.010 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T18:53:52.010 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T18:53:52.015 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T18:53:52.019 INFO:tasks.workunit.client.0.smithi112.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T18:53:52.019 INFO:tasks.workunit.client.0.smithi112.stdout:skipping zero size read 2024-04-25T18:53:52.039 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x1c37b90 2024-04-25T18:53:52.049 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x5bd330d 2024-04-25T18:53:52.083 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x637bf99 2024-04-25T18:53:52.575 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x63d9b9c 2024-04-25T18:53:52.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:52 smithi171 ceph-mon[33266]: pgmap v199: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 353 op/s 2024-04-25T18:53:52.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:52 smithi161 ceph-mon[32609]: pgmap v199: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 353 op/s 2024-04-25T18:53:52.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:52 smithi112 ceph-mon[27864]: pgmap v199: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 353 op/s 2024-04-25T18:53:54.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:54 smithi171 ceph-mon[33266]: pgmap v200: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-04-25T18:53:54.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:54 smithi161 ceph-mon[32609]: pgmap v200: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-04-25T18:53:54.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:54 smithi112 ceph-mon[27864]: pgmap v200: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-04-25T18:53:56.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:56 smithi171 ceph-mon[33266]: pgmap v201: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-04-25T18:53:56.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:56 smithi161 ceph-mon[32609]: pgmap v201: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-04-25T18:53:56.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:56 smithi112 ceph-mon[27864]: pgmap v201: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-04-25T18:53:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:58 smithi171 ceph-mon[33266]: pgmap v202: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.8 MiB/s wr, 342 op/s 2024-04-25T18:53:58.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:58 smithi161 ceph-mon[32609]: pgmap v202: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.8 MiB/s wr, 342 op/s 2024-04-25T18:53:58.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:58 smithi112 ceph-mon[27864]: pgmap v202: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.8 MiB/s wr, 342 op/s 2024-04-25T18:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.538+0000 7fb736600640 1 -- 172.21.15.112:0/4041322799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb730103830 msgr2=0x7fb73010bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.538+0000 7fb736600640 1 --2- 172.21.15.112:0/4041322799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb730103830 0x7fb73010bd10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb718009ca0 tx=0x7fb71802a950 comp rx=0 tx=0).stop 2024-04-25T18:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.538+0000 7fb736600640 1 -- 172.21.15.112:0/4041322799 shutdown_connections 2024-04-25T18:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.538+0000 7fb736600640 1 --2- 172.21.15.112:0/4041322799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb73010c330 0x7fb73010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.538+0000 7fb736600640 1 --2- 172.21.15.112:0/4041322799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb730103830 0x7fb73010bd10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.538+0000 7fb736600640 1 --2- 172.21.15.112:0/4041322799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb730102f10 0x7fb7301032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.538+0000 7fb736600640 1 -- 172.21.15.112:0/4041322799 >> 172.21.15.112:0/4041322799 conn(0x7fb7300fc770 msgr2=0x7fb7300feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.538+0000 7fb736600640 1 -- 172.21.15.112:0/4041322799 shutdown_connections 2024-04-25T18:53:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.538+0000 7fb736600640 1 -- 172.21.15.112:0/4041322799 wait complete. 2024-04-25T18:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.539+0000 7fb736600640 1 Processor -- start 2024-04-25T18:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.539+0000 7fb736600640 1 -- start start 2024-04-25T18:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.540+0000 7fb736600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb730102f10 0x7fb730100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.540+0000 7fb736600640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb730103830 0x7fb7301010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.540+0000 7fb736600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb73010c330 0x7fb7301015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.540+0000 7fb736600640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb730112010 con 0x7fb730103830 2024-04-25T18:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.540+0000 7fb736600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb730111e90 con 0x7fb730102f10 2024-04-25T18:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.540+0000 7fb736600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb730112190 con 0x7fb73010c330 2024-04-25T18:53:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.540+0000 7fb735200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb730102f10 0x7fb730100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.540+0000 7fb72fe00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb730103830 0x7fb7301010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.540+0000 7fb735200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb730102f10 0x7fb730100b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:36406/0 (socket says 172.21.15.112:36406) 2024-04-25T18:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.540+0000 7fb735c00640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb73010c330 0x7fb7301015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.540+0000 7fb735200640 1 -- 172.21.15.112:0/1266520222 learned_addr learned my addr 172.21.15.112:0/1266520222 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb72fe00640 1 -- 172.21.15.112:0/1266520222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb73010c330 msgr2=0x7fb7301015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb72fe00640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb73010c330 0x7fb7301015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb72fe00640 1 -- 172.21.15.112:0/1266520222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb730102f10 msgr2=0x7fb730100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb72fe00640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb730102f10 0x7fb730100b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb72fe00640 1 -- 172.21.15.112:0/1266520222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7301a6a70 con 0x7fb730103830 2024-04-25T18:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb735c00640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb73010c330 0x7fb7301015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb72fe00640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb730103830 0x7fb7301010a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb718005e30 tx=0x7fb718003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb72d600640 1 -- 172.21.15.112:0/1266520222 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718034040 con 0x7fb730103830 2024-04-25T18:53:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb736600640 1 -- 172.21.15.112:0/1266520222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7301a6d00 con 0x7fb730103830 2024-04-25T18:53:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb72d600640 1 -- 172.21.15.112:0/1266520222 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb718003e10 con 0x7fb730103830 2024-04-25T18:53:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb72d600640 1 -- 172.21.15.112:0/1266520222 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7180045d0 con 0x7fb730103830 2024-04-25T18:53:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.541+0000 7fb736600640 1 -- 172.21.15.112:0/1266520222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7301a71e0 con 0x7fb730103830 2024-04-25T18:53:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.545+0000 7fb72d600640 1 -- 172.21.15.112:0/1266520222 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fb718037020 con 0x7fb730103830 2024-04-25T18:53:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.546+0000 7fb72d600640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb7100a3dc0 0x7fb7100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.547+0000 7fb72d600640 1 -- 172.21.15.112:0/1266520222 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7fb7180f1240 con 0x7fb730103830 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.547+0000 7fb735200640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb7100a3dc0 0x7fb7100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.547+0000 7fb73a9f0740 1 Processor -- start 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.547+0000 7fb73a9f0740 1 -- start start 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd8583d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 0x557cd858e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 0x557cd8590f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557cd84e9900 con 0x557cd858eb90 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557cd848fef0 con 0x557cd85842d0 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557cd84e58c0 con 0x557cd85839b0 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb735200640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb7100a3dc0 0x7fb7100a6280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb720009c90 tx=0x7fb720005f90 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb735200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd8583d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb735200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd8583d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:51302/0 (socket says 172.21.15.112:51302) 2024-04-25T18:53:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb72fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 0x557cd858e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb735c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 0x557cd8590f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb735200640 1 -- 172.21.15.112:0/786345889 learned_addr learned my addr 172.21.15.112:0/786345889 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb735200640 1 -- 172.21.15.112:0/786345889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 msgr2=0x557cd858e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb735200640 1 --2- 172.21.15.112:0/786345889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 0x557cd858e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb735200640 1 -- 172.21.15.112:0/786345889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 msgr2=0x557cd8590f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb735200640 1 --2- 172.21.15.112:0/786345889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 0x557cd8590f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb735200640 1 -- 172.21.15.112:0/786345889 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd81fc210 con 0x557cd85839b0 2024-04-25T18:53:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.548+0000 7fb72fe00640 1 --2- 172.21.15.112:0/786345889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 0x557cd858e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.549+0000 7fb735200640 1 --2- 172.21.15.112:0/786345889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd8583d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb720009fc0 tx=0x7fb720014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7442b21dbc9a701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.549+0000 7fb726000640 1 -- 172.21.15.112:0/786345889 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7200265f0 con 0x557cd85839b0 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.549+0000 7fb726000640 1 -- 172.21.15.112:0/786345889 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb720026c40 con 0x557cd85839b0 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.549+0000 7fb726000640 1 -- 172.21.15.112:0/786345889 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72000ecc0 con 0x557cd85839b0 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.550+0000 7fb73a9f0740 1 -- 172.21.15.112:0/786345889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 msgr2=0x557cd8583d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.550+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/786345889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd8583d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb720009fc0 tx=0x7fb720014000 comp rx=0 tx=0).stop 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.550+0000 7fb73a9f0740 1 -- 172.21.15.112:0/786345889 shutdown_connections 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.550+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/786345889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 0x557cd8590f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.550+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/786345889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 0x557cd858e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.550+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/786345889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd8583d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.550+0000 7fb73a9f0740 1 -- 172.21.15.112:0/786345889 >> 172.21.15.112:0/786345889 conn(0x557cd857fec0 msgr2=0x557cd85802a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.550+0000 7fb73a9f0740 1 -- 172.21.15.112:0/786345889 shutdown_connections 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.550+0000 7fb73a9f0740 1 -- 172.21.15.112:0/786345889 wait complete. 2024-04-25T18:53:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.551+0000 7fb73a9f0740 1 -- start start 2024-04-25T18:53:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.551+0000 7fb73a9f0740 1 Processor -- start 2024-04-25T18:53:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.551+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 0x557cd858bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.551+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd85842d0 0x557cd858c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.551+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd858eb90 0x557cd858c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.551+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557cd85917c0 con 0x557cd85842d0 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.551+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557cd84a3310 con 0x557cd85839b0 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.551+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557cd8591940 con 0x557cd858eb90 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.551+0000 7fb73a9f0740 10 client.0 _ll_register_callbacks cb 0x557cd8471cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb735200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 0x557cd858bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb735200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 0x557cd858bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:36420/0 (socket says 172.21.15.112:36420) 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb735200640 1 -- 172.21.15.112:0/2129693135 learned_addr learned my addr 172.21.15.112:0/2129693135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb72fe00640 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd85842d0 0x557cd858c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb735c00640 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd858eb90 0x557cd858c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb735200640 1 -- 172.21.15.112:0/2129693135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd858eb90 msgr2=0x557cd858c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb735200640 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd858eb90 0x557cd858c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb735200640 1 -- 172.21.15.112:0/2129693135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd85842d0 msgr2=0x557cd858c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb735200640 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd85842d0 0x557cd858c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb735200640 1 -- 172.21.15.112:0/2129693135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd82c8620 con 0x557cd85839b0 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb72fe00640 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd85842d0 0x557cd858c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb735200640 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 0x557cd858bb60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb72000e790 tx=0x7fb72000e7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d04cb90b78e1846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.552+0000 7fb71ea00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.553+0000 7fb71ea00640 1 -- 172.21.15.112:0/2129693135 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720035060 con 0x557cd85839b0 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.553+0000 7fb71ea00640 1 -- 172.21.15.112:0/2129693135 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7fb72000ec10 con 0x557cd85839b0 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.553+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2129693135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb720026c40 con 0x557cd85839b0 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.553+0000 7fb71ea00640 1 -- 172.21.15.112:0/2129693135 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb720005040 con 0x557cd85839b0 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.553+0000 7fb73a9f0740 10 client.25820 fetch_fsmap learned FSMap version 36 2024-04-25T18:53:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.553+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2129693135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557cd859da50 con 0x557cd85839b0 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.553+0000 7fb71ea00640 1 -- 172.21.15.112:0/2129693135 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb720026c40 con 0x557cd85839b0 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.553+0000 7fb71ea00640 1 -- 172.21.15.112:0/2129693135 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fb7200046c0 con 0x557cd85839b0 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.554+0000 7fb73a9f0740 10 client.25820 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.554+0000 7fb73a9f0740 10 client.25820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.554+0000 7fb73a9f0740 20 client.25820 populate_metadata read hostname 'smithi112' 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.554+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557cd859eda0 0x557cd85a1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.554+0000 7fb73a9f0740 4 client.25820 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.554+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2129693135 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cd84fd8b0 con 0x557cd859eda0 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.554+0000 7fb72fe00640 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557cd859eda0 0x557cd85a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.554+0000 7fb72fe00640 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557cd859eda0 0x557cd85a1190 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4414a8fc40aa7d6 server_cookie=5b8214988425620b in_seq=0 out_seq=0 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.554+0000 7fb71ea00640 10 client.25820 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb71ea00640 1 -- 172.21.15.112:0/2129693135 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557cd84fd8b0 con 0x557cd859eda0 2024-04-25T18:53:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb71ea00640 10 client.25820 handle_command_reply: tid=0 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb73a9f0740 1 client.25820 shutdown 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2129693135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 msgr2=0x557cd858bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 0x557cd858bb60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb72000e790 tx=0x7fb72000e7c0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2129693135 shutdown_connections 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd858eb90 0x557cd858c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557cd859eda0 0x557cd85a1190 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd85842d0 0x557cd858c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/2129693135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 0x557cd858bb60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb71ea00640 0 client.25820 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.555+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2129693135 >> 172.21.15.112:0/2129693135 conn(0x557cd857fec0 msgr2=0x557cd8585bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.556+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2129693135 shutdown_connections 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.556+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2129693135 wait complete. 2024-04-25T18:53:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.556+0000 7fb73a9f0740 20 client.25820 trim_cache size 0 max 16384 2024-04-25T18:53:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.573+0000 7fb73a9f0740 1 Processor -- start 2024-04-25T18:53:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.573+0000 7fb73a9f0740 1 -- start start 2024-04-25T18:53:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.574+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 0x557cd859a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.574+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85842d0 0x557cd857eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.574+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 0x557cd85a6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.574+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7301b2580 con 0x557cd858eb90 2024-04-25T18:53:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.574+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb7301afe50 con 0x557cd85839b0 2024-04-25T18:53:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.574+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb7301adca0 con 0x557cd85842d0 2024-04-25T18:53:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.574+0000 7fb72fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85842d0 0x557cd857eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.574+0000 7fb72fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85842d0 0x557cd857eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:51316/0 (socket says 172.21.15.112:51316) 2024-04-25T18:53:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.574+0000 7fb72fe00640 1 -- 172.21.15.112:0/2537057430 learned_addr learned my addr 172.21.15.112:0/2537057430 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.574+0000 7fb735200640 1 --2- 172.21.15.112:0/2537057430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 0x557cd859a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.575+0000 7fb735200640 1 -- 172.21.15.112:0/2537057430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85842d0 msgr2=0x557cd857eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.575+0000 7fb735200640 1 --2- 172.21.15.112:0/2537057430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85842d0 0x557cd857eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.575+0000 7fb735200640 1 -- 172.21.15.112:0/2537057430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 msgr2=0x557cd85a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.575+0000 7fb735200640 1 --2- 172.21.15.112:0/2537057430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 0x557cd85a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.575+0000 7fb735200640 1 -- 172.21.15.112:0/2537057430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd8580840 con 0x557cd85839b0 2024-04-25T18:53:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.575+0000 7fb735200640 1 --2- 172.21.15.112:0/2537057430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 0x557cd859a680 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb720004a90 tx=0x7fb720009fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25584248756eec10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.575+0000 7fb725600640 1 -- 172.21.15.112:0/2537057430 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720020070 con 0x557cd85839b0 2024-04-25T18:53:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.575+0000 7fb725600640 1 -- 172.21.15.112:0/2537057430 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb720005690 con 0x557cd85839b0 2024-04-25T18:53:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.575+0000 7fb725600640 1 -- 172.21.15.112:0/2537057430 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720035040 con 0x557cd85839b0 2024-04-25T18:53:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.576+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2537057430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 msgr2=0x557cd859a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.576+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/2537057430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 0x557cd859a680 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb720004a90 tx=0x7fb720009fc0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.576+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2537057430 shutdown_connections 2024-04-25T18:53:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.576+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/2537057430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 0x557cd85a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.576+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/2537057430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85842d0 0x557cd857eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.576+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/2537057430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85839b0 0x557cd859a680 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.576+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2537057430 >> 172.21.15.112:0/2537057430 conn(0x557cd857fec0 msgr2=0x557cd857e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.576+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2537057430 shutdown_connections 2024-04-25T18:53:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.577+0000 7fb73a9f0740 1 -- 172.21.15.112:0/2537057430 wait complete. 2024-04-25T18:53:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.577+0000 7fb73a9f0740 1 -- start start 2024-04-25T18:53:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.577+0000 7fb73a9f0740 1 Processor -- start 2024-04-25T18:53:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd85970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 0x557cd85944d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb73a9f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 0x557cd8594bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7301ae310 con 0x557cd858eb90 2024-04-25T18:53:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb7301af780 con 0x557cd85842d0 2024-04-25T18:53:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb73a9f0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb7301ae9e0 con 0x557cd85839b0 2024-04-25T18:53:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb73a9f0740 10 client.0 _ll_register_callbacks cb 0x7fb720009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:53:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb735200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd85970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb72fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 0x557cd85944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb735200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd85970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:51324/0 (socket says 172.21.15.112:51324) 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb735200640 1 -- 172.21.15.112:0/668387784 learned_addr learned my addr 172.21.15.112:0/668387784 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb735c00640 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 0x557cd8594bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb735200640 1 -- 172.21.15.112:0/668387784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 msgr2=0x557cd85944d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb735200640 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 0x557cd85944d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb735200640 1 -- 172.21.15.112:0/668387784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 msgr2=0x557cd8594bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb735200640 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 0x557cd8594bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb735200640 1 -- 172.21.15.112:0/668387784 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd860c8b0 con 0x557cd85839b0 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.578+0000 7fb72fe00640 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 0x557cd85944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.579+0000 7fb735200640 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd85970f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb720016040 tx=0x7fb720011e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=754caee0a5a92ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.579+0000 7fb71ea00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.579+0000 7fb71ea00640 1 -- 172.21.15.112:0/668387784 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720020070 con 0x557cd85839b0 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.579+0000 7fb73a9f0740 1 -- 172.21.15.112:0/668387784 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557cd84f7320 con 0x557cd85839b0 2024-04-25T18:53:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.579+0000 7fb71ea00640 1 -- 172.21.15.112:0/668387784 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8527+0+0 (secure 0 0 0) 0x7fb72000ec10 con 0x557cd85839b0 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.579+0000 7fb71ea00640 1 -- 172.21.15.112:0/668387784 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb72004f050 con 0x557cd85839b0 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.579+0000 7fb73a9f0740 10 client.25855 fetch_fsmap learned FSMap version 36 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.579+0000 7fb73a9f0740 1 -- 172.21.15.112:0/668387784 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557cd860da60 con 0x557cd85839b0 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.580+0000 7fb71ea00640 1 -- 172.21.15.112:0/668387784 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557cd84f7320 con 0x557cd85839b0 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.580+0000 7fb71ea00640 1 -- 172.21.15.112:0/668387784 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fb720026780 con 0x557cd85839b0 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.580+0000 7fb73a9f0740 10 client.25855 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.580+0000 7fb73a9f0740 10 client.25855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.580+0000 7fb73a9f0740 20 client.25855 populate_metadata read hostname 'smithi112' 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.580+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557cd860e010 0x557cd8610400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.580+0000 7fb73a9f0740 4 client.25855 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.580+0000 7fb73a9f0740 1 -- 172.21.15.112:0/668387784 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557cd84f1190 con 0x557cd860e010 2024-04-25T18:53:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.580+0000 7fb72fe00640 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557cd860e010 0x557cd8610400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.581+0000 7fb72fe00640 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557cd860e010 0x557cd8610400 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca413741d623dd62 server_cookie=26b1a3f516406ecf in_seq=0 out_seq=0 2024-04-25T18:53:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.581+0000 7fb71ea00640 10 client.25855 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.595+0000 7fb71ea00640 1 -- 172.21.15.112:0/668387784 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557cd84f1190 con 0x557cd860e010 2024-04-25T18:53:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.595+0000 7fb71ea00640 10 client.25855 handle_command_reply: tid=0 2024-04-25T18:53:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.595+0000 7fb73a9f0740 1 client.25855 shutdown 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.595+0000 7fb73a9f0740 1 -- 172.21.15.112:0/668387784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 msgr2=0x557cd85970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.595+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd85970f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb720016040 tx=0x7fb720011e60 comp rx=0 tx=0).stop 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.596+0000 7fb73a9f0740 1 -- 172.21.15.112:0/668387784 shutdown_connections 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.596+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x557cd860e010 0x557cd8610400 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.596+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557cd858eb90 0x557cd8594bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.596+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cd85842d0 0x557cd85944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.596+0000 7fb73a9f0740 1 --2- 172.21.15.112:0/668387784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557cd85839b0 0x557cd85970f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.596+0000 7fb71ea00640 0 client.25855 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.596+0000 7fb73a9f0740 1 -- 172.21.15.112:0/668387784 >> 172.21.15.112:0/668387784 conn(0x557cd857fec0 msgr2=0x557cd857f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.596+0000 7fb73a9f0740 1 -- 172.21.15.112:0/668387784 shutdown_connections 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.596+0000 7fb73a9f0740 1 -- 172.21.15.112:0/668387784 wait complete. 2024-04-25T18:53:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.596+0000 7fb73a9f0740 20 client.25855 trim_cache size 0 max 16384 2024-04-25T18:53:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:53:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:53:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:53:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:53:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 -- 172.21.15.112:0/1266520222 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb7100a3dc0 msgr2=0x7fb7100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb7100a3dc0 0x7fb7100a6280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb720009c90 tx=0x7fb720005f90 comp rx=0 tx=0).stop 2024-04-25T18:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 -- 172.21.15.112:0/1266520222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb730103830 msgr2=0x7fb7301010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb730103830 0x7fb7301010a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb718005e30 tx=0x7fb718003c20 comp rx=0 tx=0).stop 2024-04-25T18:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 -- 172.21.15.112:0/1266520222 shutdown_connections 2024-04-25T18:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb7100a3dc0 0x7fb7100a6280 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb73010c330 0x7fb7301015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb730103830 0x7fb7301010a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 --2- 172.21.15.112:0/1266520222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb730102f10 0x7fb730100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 -- 172.21.15.112:0/1266520222 >> 172.21.15.112:0/1266520222 conn(0x7fb7300fc770 msgr2=0x7fb7300fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 -- 172.21.15.112:0/1266520222 shutdown_connections 2024-04-25T18:53:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:53:59.597+0000 7fb736600640 1 -- 172.21.15.112:0/1266520222 wait complete. 2024-04-25T18:54:00.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:53:59 smithi171 ceph-mon[33266]: pgmap v203: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.0 MiB/s wr, 218 op/s 2024-04-25T18:54:00.222 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:53:59 smithi161 ceph-mon[32609]: pgmap v203: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.0 MiB/s wr, 218 op/s 2024-04-25T18:54:00.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:53:59 smithi112 ceph-mon[27864]: pgmap v203: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.0 MiB/s wr, 218 op/s 2024-04-25T18:54:02.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:01 smithi171 ceph-mon[33266]: pgmap v204: 97 pgs: 97 active+clean; 80 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.7 MiB/s wr, 425 op/s 2024-04-25T18:54:02.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:01 smithi161 ceph-mon[32609]: pgmap v204: 97 pgs: 97 active+clean; 80 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.7 MiB/s wr, 425 op/s 2024-04-25T18:54:02.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:01 smithi112 ceph-mon[27864]: pgmap v204: 97 pgs: 97 active+clean; 80 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.7 MiB/s wr, 425 op/s 2024-04-25T18:54:03.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:02 smithi171 ceph-mon[33266]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T18:54:03.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:02 smithi161 ceph-mon[32609]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T18:54:03.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:02 smithi112 ceph-mon[27864]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T18:54:04.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:03 smithi171 ceph-mon[33266]: pgmap v206: 97 pgs: 97 active+clean; 80 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.2 MiB/s wr, 374 op/s 2024-04-25T18:54:04.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:03 smithi161 ceph-mon[32609]: pgmap v206: 97 pgs: 97 active+clean; 80 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.2 MiB/s wr, 374 op/s 2024-04-25T18:54:04.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:03 smithi112 ceph-mon[27864]: pgmap v206: 97 pgs: 97 active+clean; 80 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.2 MiB/s wr, 374 op/s 2024-04-25T18:54:06.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:05 smithi171 ceph-mon[33266]: pgmap v207: 97 pgs: 97 active+clean; 80 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.2 MiB/s wr, 374 op/s 2024-04-25T18:54:06.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:05 smithi161 ceph-mon[32609]: pgmap v207: 97 pgs: 97 active+clean; 80 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.2 MiB/s wr, 374 op/s 2024-04-25T18:54:06.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:05 smithi112 ceph-mon[27864]: pgmap v207: 97 pgs: 97 active+clean; 80 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.2 MiB/s wr, 374 op/s 2024-04-25T18:54:08.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:07 smithi171 ceph-mon[33266]: pgmap v208: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.3 MiB/s wr, 462 op/s 2024-04-25T18:54:08.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:07 smithi161 ceph-mon[32609]: pgmap v208: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.3 MiB/s wr, 462 op/s 2024-04-25T18:54:08.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:07 smithi112 ceph-mon[27864]: pgmap v208: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.3 MiB/s wr, 462 op/s 2024-04-25T18:54:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:09 smithi171 ceph-mon[33266]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T18:54:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:09 smithi171 ceph-mon[33266]: pgmap v210: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 652 KiB/s wr, 268 op/s 2024-04-25T18:54:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:09 smithi161 ceph-mon[32609]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T18:54:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:09 smithi161 ceph-mon[32609]: pgmap v210: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 652 KiB/s wr, 268 op/s 2024-04-25T18:54:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:09 smithi112 ceph-mon[27864]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T18:54:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:09 smithi112 ceph-mon[27864]: pgmap v210: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 652 KiB/s wr, 268 op/s 2024-04-25T18:54:12.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:11 smithi171 ceph-mon[33266]: pgmap v211: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 465 op/s 2024-04-25T18:54:12.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:11 smithi161 ceph-mon[32609]: pgmap v211: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 465 op/s 2024-04-25T18:54:12.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:11 smithi112 ceph-mon[27864]: pgmap v211: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 465 op/s 2024-04-25T18:54:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.744+0000 7fc37a000640 1 -- 172.21.15.112:0/1861579417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3741021c0 msgr2=0x7fc3741025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.744+0000 7fc37a000640 1 --2- 172.21.15.112:0/1861579417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3741021c0 0x7fc3741025a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc364009900 tx=0x7fc36402a400 comp rx=0 tx=0).stop 2024-04-25T18:54:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.744+0000 7fc37a000640 1 -- 172.21.15.112:0/1861579417 shutdown_connections 2024-04-25T18:54:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.744+0000 7fc37a000640 1 --2- 172.21.15.112:0/1861579417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc374107290 0x7fc37410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.744+0000 7fc37a000640 1 --2- 172.21.15.112:0/1861579417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc374102b70 0x7fc374106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.744+0000 7fc37a000640 1 --2- 172.21.15.112:0/1861579417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3741021c0 0x7fc3741025a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.744+0000 7fc37a000640 1 -- 172.21.15.112:0/1861579417 >> 172.21.15.112:0/1861579417 conn(0x7fc3740fd9e0 msgr2=0x7fc3740ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.745+0000 7fc37a000640 1 -- 172.21.15.112:0/1861579417 shutdown_connections 2024-04-25T18:54:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.745+0000 7fc37a000640 1 -- 172.21.15.112:0/1861579417 wait complete. 2024-04-25T18:54:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.746+0000 7fc37a000640 1 Processor -- start 2024-04-25T18:54:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.746+0000 7fc37a000640 1 -- start start 2024-04-25T18:54:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.746+0000 7fc37a000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3741021c0 0x7fc3741a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.746+0000 7fc37a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc374107290 0x7fc3741a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.746+0000 7fc37a000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc37419a7e0 0x7fc37419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.746+0000 7fc37a000640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc374112240 con 0x7fc3741021c0 2024-04-25T18:54:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.746+0000 7fc37a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3741120c0 con 0x7fc374107290 2024-04-25T18:54:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.746+0000 7fc37a000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc3741123c0 con 0x7fc37419a7e0 2024-04-25T18:54:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.747+0000 7fc373400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3741021c0 0x7fc3741a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.747+0000 7fc372a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc374107290 0x7fc3741a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.747+0000 7fc373400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3741021c0 0x7fc3741a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56004/0 (socket says 172.21.15.112:56004) 2024-04-25T18:54:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.747+0000 7fc373e00640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc37419a7e0 0x7fc37419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.747+0000 7fc373400640 1 -- 172.21.15.112:0/3474357733 learned_addr learned my addr 172.21.15.112:0/3474357733 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.747+0000 7fc373e00640 1 -- 172.21.15.112:0/3474357733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc374107290 msgr2=0x7fc3741a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.747+0000 7fc373e00640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc374107290 0x7fc3741a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.747+0000 7fc373e00640 1 -- 172.21.15.112:0/3474357733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3741021c0 msgr2=0x7fc3741a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.747+0000 7fc373e00640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3741021c0 0x7fc3741a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.747+0000 7fc373e00640 1 -- 172.21.15.112:0/3474357733 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37419b370 con 0x7fc37419a7e0 2024-04-25T18:54:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.748+0000 7fc373e00640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc37419a7e0 0x7fc37419ac90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00ec50 tx=0x7fc36c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.748+0000 7fc36be00640 1 -- 172.21.15.112:0/3474357733 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c003b70 con 0x7fc37419a7e0 2024-04-25T18:54:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.748+0000 7fc36be00640 1 -- 172.21.15.112:0/3474357733 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc36c003d10 con 0x7fc37419a7e0 2024-04-25T18:54:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.748+0000 7fc36be00640 1 -- 172.21.15.112:0/3474357733 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c010830 con 0x7fc37419a7e0 2024-04-25T18:54:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.748+0000 7fc37a000640 1 -- 172.21.15.112:0/3474357733 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37419b660 con 0x7fc37419a7e0 2024-04-25T18:54:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.748+0000 7fc37a000640 1 -- 172.21.15.112:0/3474357733 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3741a77f0 con 0x7fc37419a7e0 2024-04-25T18:54:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.751+0000 7fc36be00640 1 -- 172.21.15.112:0/3474357733 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fc36c003ec0 con 0x7fc37419a7e0 2024-04-25T18:54:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.753+0000 7fc36be00640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc3580a3e40 0x7fc3580a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.753+0000 7fc36be00640 1 -- 172.21.15.112:0/3474357733 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fc36c0d0cb0 con 0x7fc37419a7e0 2024-04-25T18:54:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.753+0000 7fc37ce50740 1 Processor -- start 2024-04-25T18:54:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.753+0000 7fc373400640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc3580a3e40 0x7fc3580a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.753+0000 7fc37ce50740 1 -- start start 2024-04-25T18:54:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cec820 0x565413cecc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413ced140 0x565413cf74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cf7a00 0x565413cf9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc37ce50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565413c52900 con 0x565413cf7a00 2024-04-25T18:54:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc37ce50740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565413bf8ef0 con 0x565413cec820 2024-04-25T18:54:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc37ce50740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565413c4e8c0 con 0x565413ced140 2024-04-25T18:54:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc373400640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc3580a3e40 0x7fc3580a6300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc3640096c0 tx=0x7fc364005a20 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc373e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cf7a00 0x565413cf9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc373e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cf7a00 0x565413cf9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56008/0 (socket says 172.21.15.112:56008) 2024-04-25T18:54:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc373e00640 1 -- 172.21.15.112:0/3059966667 learned_addr learned my addr 172.21.15.112:0/3059966667 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc373400640 1 --2- 172.21.15.112:0/3059966667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cec820 0x565413cecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.754+0000 7fc372a00640 1 --2- 172.21.15.112:0/3059966667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413ced140 0x565413cf74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.755+0000 7fc373e00640 1 -- 172.21.15.112:0/3059966667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413ced140 msgr2=0x565413cf74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.755+0000 7fc373e00640 1 --2- 172.21.15.112:0/3059966667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413ced140 0x565413cf74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.755+0000 7fc373e00640 1 -- 172.21.15.112:0/3059966667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cec820 msgr2=0x565413cecc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.755+0000 7fc373e00640 1 --2- 172.21.15.112:0/3059966667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cec820 0x565413cecc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.755+0000 7fc373e00640 1 -- 172.21.15.112:0/3059966667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565413965210 con 0x565413cf7a00 2024-04-25T18:54:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.755+0000 7fc373e00640 1 --2- 172.21.15.112:0/3059966667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cf7a00 0x565413cf9df0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc36c099590 tx=0x7fc36c09c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b720cba8b7ce447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.755+0000 7fc35fe00640 1 -- 172.21.15.112:0/3059966667 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0b6070 con 0x565413cf7a00 2024-04-25T18:54:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.755+0000 7fc35fe00640 1 -- 172.21.15.112:0/3059966667 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc36c09d2c0 con 0x565413cf7a00 2024-04-25T18:54:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.755+0000 7fc35fe00640 1 -- 172.21.15.112:0/3059966667 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0ce5f0 con 0x565413cf7a00 2024-04-25T18:54:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.756+0000 7fc37ce50740 1 -- 172.21.15.112:0/3059966667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cf7a00 msgr2=0x565413cf9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.756+0000 7fc37ce50740 1 --2- 172.21.15.112:0/3059966667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cf7a00 0x565413cf9df0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc36c099590 tx=0x7fc36c09c7c0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.756+0000 7fc37ce50740 1 -- 172.21.15.112:0/3059966667 shutdown_connections 2024-04-25T18:54:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.756+0000 7fc37ce50740 1 --2- 172.21.15.112:0/3059966667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cf7a00 0x565413cf9df0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.756+0000 7fc37ce50740 1 --2- 172.21.15.112:0/3059966667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413ced140 0x565413cf74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.756+0000 7fc37ce50740 1 --2- 172.21.15.112:0/3059966667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cec820 0x565413cecc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.756+0000 7fc37ce50740 1 -- 172.21.15.112:0/3059966667 >> 172.21.15.112:0/3059966667 conn(0x565413ce8d30 msgr2=0x565413ce9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.756+0000 7fc37ce50740 1 -- 172.21.15.112:0/3059966667 shutdown_connections 2024-04-25T18:54:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.757+0000 7fc37ce50740 1 -- 172.21.15.112:0/3059966667 wait complete. 2024-04-25T18:54:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.757+0000 7fc37ce50740 1 -- start start 2024-04-25T18:54:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.757+0000 7fc37ce50740 1 Processor -- start 2024-04-25T18:54:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413cf49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413ced140 0x565413cf4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 0x565413cf5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc37ce50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565413cfa630 con 0x565413ced140 2024-04-25T18:54:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc37ce50740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565413c0c310 con 0x565413cf7a00 2024-04-25T18:54:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc37ce50740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565413cfa7b0 con 0x565413cec820 2024-04-25T18:54:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc37ce50740 10 client.0 _ll_register_callbacks cb 0x565413bdacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:54:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc373400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413cf49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc373400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413cf49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:46626/0 (socket says 172.21.15.112:46626) 2024-04-25T18:54:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc373e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 0x565413cf5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc373400640 1 -- 172.21.15.112:0/27785021 learned_addr learned my addr 172.21.15.112:0/27785021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.758+0000 7fc372a00640 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413ced140 0x565413cf4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.759+0000 7fc373400640 1 -- 172.21.15.112:0/27785021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 msgr2=0x565413cf5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.759+0000 7fc373400640 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 0x565413cf5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.759+0000 7fc373400640 1 -- 172.21.15.112:0/27785021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413ced140 msgr2=0x565413cf4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.759+0000 7fc373400640 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413ced140 0x565413cf4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.759+0000 7fc373400640 1 -- 172.21.15.112:0/27785021 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565413a31620 con 0x565413cec820 2024-04-25T18:54:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.759+0000 7fc373e00640 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 0x565413cf5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.759+0000 7fc373400640 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413cf49e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc364003870 tx=0x7fc364009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c2fd1991e39b08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.759+0000 7fc35cc00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:54:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.759+0000 7fc35cc00640 1 -- 172.21.15.112:0/27785021 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36404e070 con 0x565413cec820 2024-04-25T18:54:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.759+0000 7fc35cc00640 1 -- 172.21.15.112:0/27785021 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fc3640318b0 con 0x565413cec820 2024-04-25T18:54:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc37ce50740 1 -- 172.21.15.112:0/27785021 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc36c09d2c0 con 0x565413cec820 2024-04-25T18:54:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc35cc00640 1 -- 172.21.15.112:0/27785021 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc364058050 con 0x565413cec820 2024-04-25T18:54:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc37ce50740 10 client.25867 fetch_fsmap learned FSMap version 36 2024-04-25T18:54:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc37ce50740 1 -- 172.21.15.112:0/27785021 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x565413d068e0 con 0x565413cec820 2024-04-25T18:54:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc35cc00640 1 -- 172.21.15.112:0/27785021 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc36c09d2c0 con 0x565413cec820 2024-04-25T18:54:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc35cc00640 1 -- 172.21.15.112:0/27785021 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fc364053b90 con 0x565413cec820 2024-04-25T18:54:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc37ce50740 10 client.25867 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:54:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc37ce50740 10 client.25867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:54:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc37ce50740 20 client.25867 populate_metadata read hostname 'smithi112' 2024-04-25T18:54:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc37ce50740 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565413d07e60 0x565413d0a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc37ce50740 4 client.25867 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:54:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.760+0000 7fc37ce50740 1 -- 172.21.15.112:0/27785021 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565413c668b0 con 0x565413d07e60 2024-04-25T18:54:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.761+0000 7fc372a00640 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565413d07e60 0x565413d0a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.761+0000 7fc372a00640 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565413d07e60 0x565413d0a250 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d288ebfdec5a99de server_cookie=b8d404b1ebb33c3b in_seq=0 out_seq=0 2024-04-25T18:54:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.761+0000 7fc35cc00640 10 client.25867 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:12.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc35cc00640 1 -- 172.21.15.112:0/27785021 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565413c668b0 con 0x565413d07e60 2024-04-25T18:54:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc35cc00640 10 client.25867 handle_command_reply: tid=0 2024-04-25T18:54:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc37ce50740 1 client.25867 shutdown 2024-04-25T18:54:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc37ce50740 1 -- 172.21.15.112:0/27785021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 msgr2=0x565413cf49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc37ce50740 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413cf49e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc364003870 tx=0x7fc364009260 comp rx=0 tx=0).stop 2024-04-25T18:54:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc37ce50740 1 -- 172.21.15.112:0/27785021 shutdown_connections 2024-04-25T18:54:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc37ce50740 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 0x565413cf5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc37ce50740 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565413d07e60 0x565413d0a250 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc37ce50740 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413ced140 0x565413cf4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc37ce50740 1 --2- 172.21.15.112:0/27785021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413cf49e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc37ce50740 1 -- 172.21.15.112:0/27785021 >> 172.21.15.112:0/27785021 conn(0x565413ce8d30 msgr2=0x565413ceea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc35cc00640 0 client.25867 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc37ce50740 1 -- 172.21.15.112:0/27785021 shutdown_connections 2024-04-25T18:54:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.762+0000 7fc37ce50740 1 -- 172.21.15.112:0/27785021 wait complete. 2024-04-25T18:54:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.763+0000 7fc37ce50740 20 client.25867 trim_cache size 0 max 16384 2024-04-25T18:54:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.783+0000 7fc37ce50740 1 Processor -- start 2024-04-25T18:54:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.783+0000 7fc37ce50740 1 -- start start 2024-04-25T18:54:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.783+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413d034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.783+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413ced140 0x565413cef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.783+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 0x565413d0fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc37ce50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3741b27d0 con 0x565413ced140 2024-04-25T18:54:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc37ce50740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3741b00a0 con 0x565413cf7a00 2024-04-25T18:54:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc37ce50740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc3741adf10 con 0x565413cec820 2024-04-25T18:54:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc373400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413d034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc373e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 0x565413d0fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc373400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413d034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:46640/0 (socket says 172.21.15.112:46640) 2024-04-25T18:54:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc373400640 1 -- 172.21.15.112:0/2531905293 learned_addr learned my addr 172.21.15.112:0/2531905293 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc373400640 1 -- 172.21.15.112:0/2531905293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 msgr2=0x565413d0fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc373400640 1 --2- 172.21.15.112:0/2531905293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 0x565413d0fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc373400640 1 -- 172.21.15.112:0/2531905293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413ced140 msgr2=0x565413cef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc373400640 1 --2- 172.21.15.112:0/2531905293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413ced140 0x565413cef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc373400640 1 -- 172.21.15.112:0/2531905293 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565413cf2f30 con 0x565413cec820 2024-04-25T18:54:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.784+0000 7fc373e00640 1 --2- 172.21.15.112:0/2531905293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 0x565413d0fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.785+0000 7fc373400640 1 --2- 172.21.15.112:0/2531905293 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413d034f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc364006010 tx=0x7fc36402ab80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87591d2039ac21a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.785+0000 7fc35f400640 1 -- 172.21.15.112:0/2531905293 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36404e040 con 0x565413cec820 2024-04-25T18:54:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.785+0000 7fc35f400640 1 -- 172.21.15.112:0/2531905293 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc364031d60 con 0x565413cec820 2024-04-25T18:54:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.785+0000 7fc35f400640 1 -- 172.21.15.112:0/2531905293 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364035040 con 0x565413cec820 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.786+0000 7fc37ce50740 1 -- 172.21.15.112:0/2531905293 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 msgr2=0x565413d034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.786+0000 7fc37ce50740 1 --2- 172.21.15.112:0/2531905293 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413d034f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc364006010 tx=0x7fc36402ab80 comp rx=0 tx=0).stop 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.786+0000 7fc37ce50740 1 -- 172.21.15.112:0/2531905293 shutdown_connections 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.786+0000 7fc37ce50740 1 --2- 172.21.15.112:0/2531905293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413cf7a00 0x565413d0fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.786+0000 7fc37ce50740 1 --2- 172.21.15.112:0/2531905293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413ced140 0x565413cef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.786+0000 7fc37ce50740 1 --2- 172.21.15.112:0/2531905293 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cec820 0x565413d034f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.786+0000 7fc37ce50740 1 -- 172.21.15.112:0/2531905293 >> 172.21.15.112:0/2531905293 conn(0x565413ce8d30 msgr2=0x565413ce7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.786+0000 7fc37ce50740 1 -- 172.21.15.112:0/2531905293 shutdown_connections 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.786+0000 7fc37ce50740 1 -- 172.21.15.112:0/2531905293 wait complete. 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.786+0000 7fc37ce50740 1 -- start start 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.787+0000 7fc37ce50740 1 Processor -- start 2024-04-25T18:54:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.787+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cec820 0x565413cffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.787+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413ced140 0x565413cfd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.787+0000 7fc37ce50740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cf7a00 0x565413cfd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.787+0000 7fc37ce50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3741ae560 con 0x565413cec820 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.787+0000 7fc37ce50740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3741af9d0 con 0x565413ced140 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.787+0000 7fc37ce50740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc3741aec30 con 0x565413cf7a00 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.787+0000 7fc37ce50740 10 client.0 _ll_register_callbacks cb 0x7fc364004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc373e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cf7a00 0x565413cfd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc372a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413ced140 0x565413cfd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc373e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cf7a00 0x565413cfd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:46654/0 (socket says 172.21.15.112:46654) 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc373e00640 1 -- 172.21.15.112:0/848981362 learned_addr learned my addr 172.21.15.112:0/848981362 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc373400640 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cec820 0x565413cffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc372a00640 1 -- 172.21.15.112:0/848981362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cf7a00 msgr2=0x565413cfd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc372a00640 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cf7a00 0x565413cfd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc372a00640 1 -- 172.21.15.112:0/848981362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cec820 msgr2=0x565413cffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc372a00640 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cec820 0x565413cffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc372a00640 1 -- 172.21.15.112:0/848981362 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565413cfdfe0 con 0x565413ced140 2024-04-25T18:54:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc373400640 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cec820 0x565413cffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:54:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc372a00640 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413ced140 0x565413cfd210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x565413ceddc0 tx=0x7fc36000bdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1e2a3724560edf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.788+0000 7fc35cc00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.789+0000 7fc35cc00640 1 -- 172.21.15.112:0/848981362 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360005220 con 0x565413ced140 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.789+0000 7fc37ce50740 1 -- 172.21.15.112:0/848981362 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc364031d60 con 0x565413ced140 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.789+0000 7fc35cc00640 1 -- 172.21.15.112:0/848981362 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fc3600053c0 con 0x565413ced140 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.789+0000 7fc35cc00640 1 -- 172.21.15.112:0/848981362 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc36002f050 con 0x565413ced140 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.789+0000 7fc37ce50740 10 client.25844 fetch_fsmap learned FSMap version 36 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.789+0000 7fc37ce50740 1 -- 172.21.15.112:0/848981362 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x565413d6e500 con 0x565413ced140 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.789+0000 7fc35cc00640 1 -- 172.21.15.112:0/848981362 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc364031d60 con 0x565413ced140 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.789+0000 7fc35cc00640 1 -- 172.21.15.112:0/848981362 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fc3600201f0 con 0x565413ced140 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.790+0000 7fc37ce50740 10 client.25844 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.790+0000 7fc37ce50740 10 client.25844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.790+0000 7fc37ce50740 20 client.25844 populate_metadata read hostname 'smithi112' 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.790+0000 7fc37ce50740 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565413d6eca0 0x565413d710c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.790+0000 7fc37ce50740 4 client.25844 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:54:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.790+0000 7fc37ce50740 1 -- 172.21.15.112:0/848981362 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565413c60320 con 0x565413d6eca0 2024-04-25T18:54:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.790+0000 7fc373400640 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565413d6eca0 0x565413d710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.791+0000 7fc373400640 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565413d6eca0 0x565413d710c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c31b1e77471b53 server_cookie=c8a4827e2e5790b8 in_seq=0 out_seq=0 2024-04-25T18:54:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.791+0000 7fc35cc00640 10 client.25844 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc35cc00640 1 -- 172.21.15.112:0/848981362 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565413c60320 con 0x565413d6eca0 2024-04-25T18:54:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc35cc00640 10 client.25844 handle_command_reply: tid=0 2024-04-25T18:54:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc37ce50740 1 client.25844 shutdown 2024-04-25T18:54:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:54:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:54:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:54:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:54:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc37ce50740 1 -- 172.21.15.112:0/848981362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413ced140 msgr2=0x565413cfd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc37ce50740 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413ced140 0x565413cfd210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x565413ceddc0 tx=0x7fc36000bdd0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc37ce50740 1 -- 172.21.15.112:0/848981362 shutdown_connections 2024-04-25T18:54:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc37ce50740 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565413cf7a00 0x565413cfd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc37ce50740 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565413ced140 0x565413cfd210 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc37ce50740 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x565413d6eca0 0x565413d710c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc37ce50740 1 --2- 172.21.15.112:0/848981362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565413cec820 0x565413cffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc37ce50740 1 -- 172.21.15.112:0/848981362 >> 172.21.15.112:0/848981362 conn(0x565413ce8d30 msgr2=0x565413cf49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc35cc00640 0 client.25844 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc37ce50740 1 -- 172.21.15.112:0/848981362 shutdown_connections 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.807+0000 7fc37ce50740 1 -- 172.21.15.112:0/848981362 wait complete. 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.808+0000 7fc37ce50740 20 client.25844 trim_cache size 0 max 16384 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.808+0000 7fc37a000640 1 -- 172.21.15.112:0/3474357733 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc3580a3e40 msgr2=0x7fc3580a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.808+0000 7fc37a000640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc3580a3e40 0x7fc3580a6300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc3640096c0 tx=0x7fc364005a20 comp rx=0 tx=0).stop 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.809+0000 7fc37a000640 1 -- 172.21.15.112:0/3474357733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc37419a7e0 msgr2=0x7fc37419ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.809+0000 7fc37a000640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc37419a7e0 0x7fc37419ac90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00ec50 tx=0x7fc36c00c6a0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.809+0000 7fc37a000640 1 -- 172.21.15.112:0/3474357733 shutdown_connections 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.809+0000 7fc37a000640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fc3580a3e40 0x7fc3580a6300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.809+0000 7fc37a000640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc37419a7e0 0x7fc37419ac90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.809+0000 7fc37a000640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc374107290 0x7fc3741a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.809+0000 7fc37a000640 1 --2- 172.21.15.112:0/3474357733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3741021c0 0x7fc3741a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.809+0000 7fc37a000640 1 -- 172.21.15.112:0/3474357733 >> 172.21.15.112:0/3474357733 conn(0x7fc3740fd9e0 msgr2=0x7fc3740fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.809+0000 7fc37a000640 1 -- 172.21.15.112:0/3474357733 shutdown_connections 2024-04-25T18:54:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:12.809+0000 7fc37a000640 1 -- 172.21.15.112:0/3474357733 wait complete. 2024-04-25T18:54:14.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:13 smithi171 ceph-mon[33266]: pgmap v212: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-04-25T18:54:14.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:13 smithi161 ceph-mon[32609]: pgmap v212: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-04-25T18:54:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:13 smithi112 ceph-mon[27864]: pgmap v212: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-04-25T18:54:16.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:15 smithi171 ceph-mon[33266]: pgmap v213: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-04-25T18:54:16.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:15 smithi161 ceph-mon[32609]: pgmap v213: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-04-25T18:54:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:15 smithi112 ceph-mon[27864]: pgmap v213: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-04-25T18:54:18.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:17 smithi171 ceph-mon[33266]: pgmap v214: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:54:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:17 smithi161 ceph-mon[32609]: pgmap v214: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:54:18.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:17 smithi112 ceph-mon[27864]: pgmap v214: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:54:19.821 INFO:tasks.workunit.client.0.smithi112.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-04-25T18:54:20.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:19 smithi171 ceph-mon[33266]: pgmap v215: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:54:20.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:19 smithi161 ceph-mon[32609]: pgmap v215: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:54:20.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:19 smithi112 ceph-mon[27864]: pgmap v215: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:54:22.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:21 smithi171 ceph-mon[33266]: pgmap v216: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 507 op/s 2024-04-25T18:54:22.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:21 smithi161 ceph-mon[32609]: pgmap v216: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 507 op/s 2024-04-25T18:54:22.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:21 smithi112 ceph-mon[27864]: pgmap v216: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 507 op/s 2024-04-25T18:54:24.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:23 smithi171 ceph-mon[33266]: pgmap v217: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.1 MiB/s wr, 340 op/s 2024-04-25T18:54:24.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:23 smithi161 ceph-mon[32609]: pgmap v217: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.1 MiB/s wr, 340 op/s 2024-04-25T18:54:24.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:23 smithi112 ceph-mon[27864]: pgmap v217: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.1 MiB/s wr, 340 op/s 2024-04-25T18:54:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.952+0000 7f143c600640 1 -- 172.21.15.112:0/1780312365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1434104d50 msgr2=0x7f1434108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.952+0000 7f143c600640 1 --2- 172.21.15.112:0/1780312365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1434104d50 0x7f1434108d90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f142002a410 comp rx=0 tx=0).stop 2024-04-25T18:54:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.952+0000 7f143c600640 1 -- 172.21.15.112:0/1780312365 shutdown_connections 2024-04-25T18:54:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.952+0000 7f143c600640 1 --2- 172.21.15.112:0/1780312365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1434109470 0x7f1434111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.952+0000 7f143c600640 1 --2- 172.21.15.112:0/1780312365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1434104d50 0x7f1434108d90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.952+0000 7f143c600640 1 --2- 172.21.15.112:0/1780312365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14341043a0 0x7f1434104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.952+0000 7f143c600640 1 -- 172.21.15.112:0/1780312365 >> 172.21.15.112:0/1780312365 conn(0x7f14340ffc40 msgr2=0x7f1434102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.952+0000 7f143c600640 1 -- 172.21.15.112:0/1780312365 shutdown_connections 2024-04-25T18:54:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.952+0000 7f143c600640 1 -- 172.21.15.112:0/1780312365 wait complete. 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.953+0000 7f143c600640 1 Processor -- start 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f143c600640 1 -- start start 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f143c600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14341043a0 0x7f14341a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f143c600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1434109470 0x7f14341a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f143c600640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f143419a720 0x7f143419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f143c600640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14341143e0 con 0x7f143419a720 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f143c600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1434114260 con 0x7f14341043a0 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f143c600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1434114560 con 0x7f1434109470 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f1439200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1434109470 0x7f14341a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f1439200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1434109470 0x7f14341a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:42426/0 (socket says 172.21.15.112:42426) 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f1439200640 1 -- 172.21.15.112:0/1401052590 learned_addr learned my addr 172.21.15.112:0/1401052590 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f1439c00640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14341043a0 0x7f14341a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.954+0000 7f143a600640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f143419a720 0x7f143419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.955+0000 7f1439200640 1 -- 172.21.15.112:0/1401052590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14341043a0 msgr2=0x7f14341a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.955+0000 7f1439200640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14341043a0 0x7f14341a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.955+0000 7f1439200640 1 -- 172.21.15.112:0/1401052590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f143419a720 msgr2=0x7f143419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.955+0000 7f1439200640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f143419a720 0x7f143419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.955+0000 7f1439200640 1 -- 172.21.15.112:0/1401052590 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143419b320 con 0x7f1434109470 2024-04-25T18:54:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.955+0000 7f1439c00640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14341043a0 0x7f14341a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.955+0000 7f1439200640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1434109470 0x7f14341a0b50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1420005ab0 tx=0x7f1420003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.955+0000 7f1432000640 1 -- 172.21.15.112:0/1401052590 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142003f070 con 0x7f1434109470 2024-04-25T18:54:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.955+0000 7f1432000640 1 -- 172.21.15.112:0/1401052590 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1420031b50 con 0x7f1434109470 2024-04-25T18:54:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.955+0000 7f143c600640 1 -- 172.21.15.112:0/1401052590 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143419b5b0 con 0x7f1434109470 2024-04-25T18:54:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.955+0000 7f1432000640 1 -- 172.21.15.112:0/1401052590 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420035030 con 0x7f1434109470 2024-04-25T18:54:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.956+0000 7f143c600640 1 -- 172.21.15.112:0/1401052590 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14341a76f0 con 0x7f1434109470 2024-04-25T18:54:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.960+0000 7f1432000640 1 -- 172.21.15.112:0/1401052590 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f1420042070 con 0x7f1434109470 2024-04-25T18:54:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.961+0000 7f143f3fa740 1 Processor -- start 2024-04-25T18:54:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.961+0000 7f143f3fa740 1 -- start start 2024-04-25T18:54:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409c820 0x5585f409cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409d140 0x5585f40a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f143f3fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5585f4002900 con 0x5585f409d140 2024-04-25T18:54:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f143f3fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5585f3fa8ef0 con 0x5585f409c820 2024-04-25T18:54:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f143f3fa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5585f3ffe8c0 con 0x5585f40a7a00 2024-04-25T18:54:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f1432000640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f141c0a3e40 0x7f141c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f1439200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f1439c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409d140 0x5585f40a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f1439200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:42436/0 (socket says 172.21.15.112:42436) 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f1439c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409d140 0x5585f40a74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53256/0 (socket says 172.21.15.112:53256) 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f1439200640 1 -- 172.21.15.112:0/1352238628 learned_addr learned my addr 172.21.15.112:0/1352238628 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f143a600640 1 --2- 172.21.15.112:0/1352238628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409c820 0x5585f409cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f143a600640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f141c0a3e40 0x7f141c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.962+0000 7f1432000640 1 -- 172.21.15.112:0/1401052590 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f14200f0d00 con 0x7f1434109470 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.963+0000 7f1439200640 1 -- 172.21.15.112:0/1352238628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409c820 msgr2=0x5585f409cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.963+0000 7f1439200640 1 --2- 172.21.15.112:0/1352238628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409c820 0x5585f409cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.963+0000 7f1439200640 1 -- 172.21.15.112:0/1352238628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409d140 msgr2=0x5585f40a74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.963+0000 7f1439200640 1 --2- 172.21.15.112:0/1352238628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409d140 0x5585f40a74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.963+0000 7f1439200640 1 -- 172.21.15.112:0/1352238628 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f3d15210 con 0x5585f40a7a00 2024-04-25T18:54:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.963+0000 7f1439c00640 1 --2- 172.21.15.112:0/1352238628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409d140 0x5585f40a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.963+0000 7f1439200640 1 --2- 172.21.15.112:0/1352238628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40a9df0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f14200c53b0 tx=0x7f142003b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2948f719c58a2784 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.963+0000 7f143a600640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f141c0a3e40 0x7f141c0a6300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f142c005fd0 tx=0x7f142c009500 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.963+0000 7f1426a00640 1 -- 172.21.15.112:0/1352238628 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420049070 con 0x5585f40a7a00 2024-04-25T18:54:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.963+0000 7f1426a00640 1 -- 172.21.15.112:0/1352238628 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14200c58e0 con 0x5585f40a7a00 2024-04-25T18:54:25.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.963+0000 7f1426a00640 1 -- 172.21.15.112:0/1352238628 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14200b9390 con 0x5585f40a7a00 2024-04-25T18:54:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.964+0000 7f143f3fa740 1 -- 172.21.15.112:0/1352238628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 msgr2=0x5585f40a9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.964+0000 7f143f3fa740 1 --2- 172.21.15.112:0/1352238628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40a9df0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f14200c53b0 tx=0x7f142003b6b0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.964+0000 7f143f3fa740 1 -- 172.21.15.112:0/1352238628 shutdown_connections 2024-04-25T18:54:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.964+0000 7f143f3fa740 1 --2- 172.21.15.112:0/1352238628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40a9df0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.964+0000 7f143f3fa740 1 --2- 172.21.15.112:0/1352238628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409d140 0x5585f40a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.964+0000 7f143f3fa740 1 --2- 172.21.15.112:0/1352238628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409c820 0x5585f409cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.964+0000 7f143f3fa740 1 -- 172.21.15.112:0/1352238628 >> 172.21.15.112:0/1352238628 conn(0x5585f4098d30 msgr2=0x5585f4099110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.964+0000 7f143f3fa740 1 -- 172.21.15.112:0/1352238628 shutdown_connections 2024-04-25T18:54:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.964+0000 7f143f3fa740 1 -- 172.21.15.112:0/1352238628 wait complete. 2024-04-25T18:54:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.965+0000 7f143f3fa740 1 -- start start 2024-04-25T18:54:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.965+0000 7f143f3fa740 1 Processor -- start 2024-04-25T18:54:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.965+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 0x5585f40a4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.965+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 0x5585f40a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.965+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f40a7a00 0x5585f40a5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.965+0000 7f143f3fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5585f40aa3f0 con 0x5585f409c820 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.965+0000 7f143f3fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5585f3fbc310 con 0x5585f40a7a00 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.965+0000 7f143f3fa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5585f40aa570 con 0x5585f409d140 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f1439200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 0x5585f40a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f143f3fa740 10 client.0 _ll_register_callbacks cb 0x5585f3f8acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f1439200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 0x5585f40a4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:42444/0 (socket says 172.21.15.112:42444) 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f1439200640 1 -- 172.21.15.112:0/1322925456 learned_addr learned my addr 172.21.15.112:0/1322925456 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f1439c00640 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 0x5585f40a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f143a600640 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f40a7a00 0x5585f40a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f1439200640 1 -- 172.21.15.112:0/1322925456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f40a7a00 msgr2=0x5585f40a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f1439200640 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f40a7a00 0x5585f40a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f1439200640 1 -- 172.21.15.112:0/1322925456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 msgr2=0x5585f40a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f1439200640 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 0x5585f40a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f1439200640 1 -- 172.21.15.112:0/1322925456 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f3de1620 con 0x5585f409d140 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f143a600640 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f40a7a00 0x5585f40a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f1439c00640 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 0x5585f40a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f1439200640 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 0x5585f40a4ec0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f14200c53b0 tx=0x7f1420003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d16c9bd11d2a8c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.966+0000 7f141b400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.967+0000 7f141b400640 1 -- 172.21.15.112:0/1322925456 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420049040 con 0x5585f409d140 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.967+0000 7f141b400640 1 -- 172.21.15.112:0/1322925456 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f14200badc0 con 0x5585f409d140 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.967+0000 7f143f3fa740 1 -- 172.21.15.112:0/1322925456 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14200c58e0 con 0x5585f409d140 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.967+0000 7f141b400640 1 -- 172.21.15.112:0/1322925456 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f14200d6140 con 0x5585f409d140 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.967+0000 7f143f3fa740 10 client.25885 fetch_fsmap learned FSMap version 36 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.967+0000 7f143f3fa740 1 -- 172.21.15.112:0/1322925456 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5585f40aa3f0 con 0x5585f409d140 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.967+0000 7f141b400640 1 -- 172.21.15.112:0/1322925456 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14200c58e0 con 0x5585f409d140 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.967+0000 7f141b400640 1 -- 172.21.15.112:0/1322925456 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f14200bcf80 con 0x5585f409d140 2024-04-25T18:54:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.967+0000 7f143f3fa740 10 client.25885 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:54:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.967+0000 7f143f3fa740 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:54:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.967+0000 7f143f3fa740 20 client.25885 populate_metadata read hostname 'smithi112' 2024-04-25T18:54:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.968+0000 7f143f3fa740 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5585f40b7720 0x5585f40b9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.968+0000 7f143f3fa740 4 client.25885 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:54:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.968+0000 7f143f3fa740 1 -- 172.21.15.112:0/1322925456 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585f40168b0 con 0x5585f40b7720 2024-04-25T18:54:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.968+0000 7f1439c00640 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5585f40b7720 0x5585f40b9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.968+0000 7f1439c00640 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5585f40b7720 0x5585f40b9b10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c208f73cad1ec482 server_cookie=1e62f11169f60461 in_seq=0 out_seq=0 2024-04-25T18:54:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.968+0000 7f141b400640 10 client.25885 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.969+0000 7f141b400640 1 -- 172.21.15.112:0/1322925456 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585f40168b0 con 0x5585f40b7720 2024-04-25T18:54:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.969+0000 7f141b400640 10 client.25885 handle_command_reply: tid=0 2024-04-25T18:54:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.969+0000 7f143f3fa740 1 client.25885 shutdown 2024-04-25T18:54:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.969+0000 7f143f3fa740 1 -- 172.21.15.112:0/1322925456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 msgr2=0x5585f40a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.969+0000 7f143f3fa740 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 0x5585f40a4ec0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f14200c53b0 tx=0x7f1420003f90 comp rx=0 tx=0).stop 2024-04-25T18:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.969+0000 7f143f3fa740 1 -- 172.21.15.112:0/1322925456 shutdown_connections 2024-04-25T18:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.969+0000 7f143f3fa740 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f40a7a00 0x5585f40a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.969+0000 7f143f3fa740 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 0x5585f40a4ec0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.969+0000 7f143f3fa740 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5585f40b7720 0x5585f40b9b10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.970+0000 7f143f3fa740 1 --2- 172.21.15.112:0/1322925456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 0x5585f40a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.970+0000 7f143f3fa740 1 -- 172.21.15.112:0/1322925456 >> 172.21.15.112:0/1322925456 conn(0x5585f4098d30 msgr2=0x5585f409eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.970+0000 7f141b400640 0 client.25885 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.970+0000 7f143f3fa740 1 -- 172.21.15.112:0/1322925456 shutdown_connections 2024-04-25T18:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.970+0000 7f143f3fa740 1 -- 172.21.15.112:0/1322925456 wait complete. 2024-04-25T18:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.970+0000 7f143f3fa740 20 client.25885 trim_cache size 0 max 16384 2024-04-25T18:54:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.986+0000 7f143f3fa740 1 Processor -- start 2024-04-25T18:54:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f143f3fa740 1 -- start start 2024-04-25T18:54:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409c820 0x5585f40b3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 0x5585f409fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f40a7a00 0x5585f409f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f143f3fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14341b2710 con 0x5585f40a7a00 2024-04-25T18:54:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f143f3fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f14341affe0 con 0x5585f409c820 2024-04-25T18:54:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f143f3fa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f14341ade50 con 0x5585f409d140 2024-04-25T18:54:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f143a600640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f40a7a00 0x5585f409f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f143a600640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f40a7a00 0x5585f409f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53278/0 (socket says 172.21.15.112:53278) 2024-04-25T18:54:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f143a600640 1 -- 172.21.15.112:0/3528024964 learned_addr learned my addr 172.21.15.112:0/3528024964 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f1439c00640 1 --2- 172.21.15.112:0/3528024964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409c820 0x5585f40b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.987+0000 7f1439200640 1 --2- 172.21.15.112:0/3528024964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 0x5585f409fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.988+0000 7f1439200640 1 -- 172.21.15.112:0/3528024964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409c820 msgr2=0x5585f40b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.988+0000 7f1439200640 1 --2- 172.21.15.112:0/3528024964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409c820 0x5585f40b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.988+0000 7f1439200640 1 -- 172.21.15.112:0/3528024964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f40a7a00 msgr2=0x5585f409f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.988+0000 7f1439200640 1 --2- 172.21.15.112:0/3528024964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f40a7a00 0x5585f409f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.988+0000 7f1439200640 1 -- 172.21.15.112:0/3528024964 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f4099ec0 con 0x5585f409d140 2024-04-25T18:54:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.988+0000 7f1439c00640 1 --2- 172.21.15.112:0/3528024964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409c820 0x5585f40b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.988+0000 7f1439200640 1 --2- 172.21.15.112:0/3528024964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 0x5585f409fce0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14200bccb0 tx=0x7f14200bb230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48693517c495b02a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.988+0000 7f1426000640 1 -- 172.21.15.112:0/3528024964 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420049070 con 0x5585f409d140 2024-04-25T18:54:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.988+0000 7f1426000640 1 -- 172.21.15.112:0/3528024964 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14200ee020 con 0x5585f409d140 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.988+0000 7f1426000640 1 -- 172.21.15.112:0/3528024964 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14200badc0 con 0x5585f409d140 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.989+0000 7f143f3fa740 1 -- 172.21.15.112:0/3528024964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 msgr2=0x5585f409fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.989+0000 7f143f3fa740 1 --2- 172.21.15.112:0/3528024964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 0x5585f409fce0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14200bccb0 tx=0x7f14200bb230 comp rx=0 tx=0).stop 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.989+0000 7f143f3fa740 1 -- 172.21.15.112:0/3528024964 shutdown_connections 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.989+0000 7f143f3fa740 1 --2- 172.21.15.112:0/3528024964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f40a7a00 0x5585f409f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.989+0000 7f143f3fa740 1 --2- 172.21.15.112:0/3528024964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f409d140 0x5585f409fce0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.989+0000 7f143f3fa740 1 --2- 172.21.15.112:0/3528024964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409c820 0x5585f40b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.989+0000 7f143f3fa740 1 -- 172.21.15.112:0/3528024964 >> 172.21.15.112:0/3528024964 conn(0x5585f4098d30 msgr2=0x5585f40a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.989+0000 7f143f3fa740 1 -- 172.21.15.112:0/3528024964 shutdown_connections 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.989+0000 7f143f3fa740 1 -- 172.21.15.112:0/3528024964 wait complete. 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.990+0000 7f143f3fa740 1 -- start start 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.990+0000 7f143f3fa740 1 Processor -- start 2024-04-25T18:54:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.990+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 0x5585f40af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.990+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409d140 0x5585f40acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.990+0000 7f143f3fa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40ad290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.990+0000 7f143f3fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14341ae4a0 con 0x5585f409c820 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.990+0000 7f143f3fa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f14341af910 con 0x5585f409d140 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.990+0000 7f143f3fa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f14341aeb70 con 0x5585f40a7a00 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.990+0000 7f143f3fa740 10 client.0 _ll_register_callbacks cb 0x7f142c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f1439c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 0x5585f40af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f143a600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f143a600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:42464/0 (socket says 172.21.15.112:42464) 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f143a600640 1 -- 172.21.15.112:0/695101715 learned_addr learned my addr 172.21.15.112:0/695101715 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f1439200640 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409d140 0x5585f40acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f143a600640 1 -- 172.21.15.112:0/695101715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409d140 msgr2=0x5585f40acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f143a600640 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409d140 0x5585f40acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f143a600640 1 -- 172.21.15.112:0/695101715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 msgr2=0x5585f40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f143a600640 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 0x5585f40af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f143a600640 1 -- 172.21.15.112:0/695101715 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f40ad980 con 0x5585f40a7a00 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f1439c00640 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 0x5585f40af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f143a600640 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40ad290 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f142c00fe20 tx=0x7f142c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a2069962368f2a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.991+0000 7f141b400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.992+0000 7f141b400640 1 -- 172.21.15.112:0/695101715 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c004b20 con 0x5585f40a7a00 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.992+0000 7f143f3fa740 1 -- 172.21.15.112:0/695101715 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14200ee020 con 0x5585f40a7a00 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.992+0000 7f141b400640 1 -- 172.21.15.112:0/695101715 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f142c004cc0 con 0x5585f40a7a00 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.992+0000 7f141b400640 1 -- 172.21.15.112:0/695101715 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f142c005020 con 0x5585f40a7a00 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.992+0000 7f143f3fa740 10 client.25891 fetch_fsmap learned FSMap version 36 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.992+0000 7f143f3fa740 1 -- 172.21.15.112:0/695101715 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5585f411d9a0 con 0x5585f40a7a00 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.992+0000 7f141b400640 1 -- 172.21.15.112:0/695101715 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14200ee020 con 0x5585f40a7a00 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.993+0000 7f141b400640 1 -- 172.21.15.112:0/695101715 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f142c01d070 con 0x5585f40a7a00 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.993+0000 7f143f3fa740 10 client.25891 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.993+0000 7f143f3fa740 10 client.25891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:54:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.993+0000 7f143f3fa740 20 client.25891 populate_metadata read hostname 'smithi112' 2024-04-25T18:54:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.993+0000 7f143f3fa740 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5585f411e120 0x5585f4120540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.993+0000 7f143f3fa740 4 client.25891 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:54:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.993+0000 7f143f3fa740 1 -- 172.21.15.112:0/695101715 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585f4010320 con 0x5585f411e120 2024-04-25T18:54:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.993+0000 7f1439c00640 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5585f411e120 0x5585f4120540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.994+0000 7f1439c00640 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5585f411e120 0x5585f4120540 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b8e76e85e11b6f server_cookie=e6aa3f85cbbb6bd4 in_seq=0 out_seq=0 2024-04-25T18:54:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:25.994+0000 7f141b400640 10 client.25891 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f141b400640 1 -- 172.21.15.112:0/695101715 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585f4010320 con 0x5585f411e120 2024-04-25T18:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f141b400640 10 client.25891 handle_command_reply: tid=0 2024-04-25T18:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f143f3fa740 1 client.25891 shutdown 2024-04-25T18:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f143f3fa740 1 -- 172.21.15.112:0/695101715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 msgr2=0x5585f40ad290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f143f3fa740 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40ad290 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f142c00fe20 tx=0x7f142c004a40 comp rx=0 tx=0).stop 2024-04-25T18:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f143f3fa740 1 -- 172.21.15.112:0/695101715 shutdown_connections 2024-04-25T18:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f143f3fa740 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5585f40a7a00 0x5585f40ad290 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f143f3fa740 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f409d140 0x5585f40acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f143f3fa740 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5585f411e120 0x5585f4120540 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f143f3fa740 1 --2- 172.21.15.112:0/695101715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585f409c820 0x5585f40af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f141b400640 0 client.25891 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.006+0000 7f143f3fa740 1 -- 172.21.15.112:0/695101715 >> 172.21.15.112:0/695101715 conn(0x5585f4098d30 msgr2=0x5585f4097600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.007+0000 7f143f3fa740 1 -- 172.21.15.112:0/695101715 shutdown_connections 2024-04-25T18:54:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.007+0000 7f143f3fa740 1 -- 172.21.15.112:0/695101715 wait complete. 2024-04-25T18:54:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.007+0000 7f143f3fa740 20 client.25891 trim_cache size 0 max 16384 2024-04-25T18:54:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:54:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:54:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:54:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.008+0000 7f143c600640 1 -- 172.21.15.112:0/1401052590 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f141c0a3e40 msgr2=0x7f141c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.008+0000 7f143c600640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f141c0a3e40 0x7f141c0a6300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f142c005fd0 tx=0x7f142c009500 comp rx=0 tx=0).stop 2024-04-25T18:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.008+0000 7f143c600640 1 -- 172.21.15.112:0/1401052590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1434109470 msgr2=0x7f14341a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.008+0000 7f143c600640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1434109470 0x7f14341a0b50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1420005ab0 tx=0x7f1420003b90 comp rx=0 tx=0).stop 2024-04-25T18:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.008+0000 7f143c600640 1 -- 172.21.15.112:0/1401052590 shutdown_connections 2024-04-25T18:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.008+0000 7f143c600640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f141c0a3e40 0x7f141c0a6300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.008+0000 7f143c600640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f143419a720 0x7f143419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.008+0000 7f143c600640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1434109470 0x7f14341a0b50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.008+0000 7f143c600640 1 --2- 172.21.15.112:0/1401052590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14341043a0 0x7f14341a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.008+0000 7f143c600640 1 -- 172.21.15.112:0/1401052590 >> 172.21.15.112:0/1401052590 conn(0x7f14340ffc40 msgr2=0x7f1434101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.009+0000 7f143c600640 1 -- 172.21.15.112:0/1401052590 shutdown_connections 2024-04-25T18:54:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:26.009+0000 7f143c600640 1 -- 172.21.15.112:0/1401052590 wait complete. 2024-04-25T18:54:26.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:25 smithi171 ceph-mon[33266]: pgmap v218: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.1 MiB/s wr, 340 op/s 2024-04-25T18:54:26.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:25 smithi161 ceph-mon[32609]: pgmap v218: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.1 MiB/s wr, 340 op/s 2024-04-25T18:54:26.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:25 smithi112 ceph-mon[27864]: pgmap v218: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.1 MiB/s wr, 340 op/s 2024-04-25T18:54:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:27 smithi171 ceph-mon[33266]: pgmap v219: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 529 op/s 2024-04-25T18:54:28.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:27 smithi161 ceph-mon[32609]: pgmap v219: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 529 op/s 2024-04-25T18:54:28.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:27 smithi112 ceph-mon[27864]: pgmap v219: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 529 op/s 2024-04-25T18:54:30.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:29 smithi171 ceph-mon[33266]: pgmap v220: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.1 MiB/s wr, 364 op/s 2024-04-25T18:54:30.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:29 smithi161 ceph-mon[32609]: pgmap v220: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.1 MiB/s wr, 364 op/s 2024-04-25T18:54:30.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:29 smithi112 ceph-mon[27864]: pgmap v220: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.1 MiB/s wr, 364 op/s 2024-04-25T18:54:32.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:31 smithi171 ceph-mon[33266]: pgmap v221: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 548 op/s 2024-04-25T18:54:32.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:31 smithi161 ceph-mon[32609]: pgmap v221: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 548 op/s 2024-04-25T18:54:32.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:31 smithi112 ceph-mon[27864]: pgmap v221: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 548 op/s 2024-04-25T18:54:34.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:33 smithi171 ceph-mon[33266]: pgmap v222: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-04-25T18:54:34.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:33 smithi161 ceph-mon[32609]: pgmap v222: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-04-25T18:54:34.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:33 smithi112 ceph-mon[27864]: pgmap v222: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-04-25T18:54:35.694 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x63e5d2a 2024-04-25T18:54:36.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:35 smithi171 ceph-mon[33266]: pgmap v223: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-04-25T18:54:36.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:35 smithi161 ceph-mon[32609]: pgmap v223: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-04-25T18:54:36.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:35 smithi112 ceph-mon[27864]: pgmap v223: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-04-25T18:54:38.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:37 smithi161 ceph-mon[32609]: pgmap v224: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 544 op/s 2024-04-25T18:54:38.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:37 smithi112 ceph-mon[27864]: pgmap v224: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 544 op/s 2024-04-25T18:54:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:37 smithi171 ceph-mon[33266]: pgmap v224: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 544 op/s 2024-04-25T18:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.170+0000 7f710ba00640 1 -- 172.21.15.112:0/2839060844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71041043a0 msgr2=0x7f7104104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.170+0000 7f710ba00640 1 --2- 172.21.15.112:0/2839060844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71041043a0 0x7f7104104780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009900 tx=0x7f70f402a410 comp rx=0 tx=0).stop 2024-04-25T18:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.170+0000 7f710ba00640 1 -- 172.21.15.112:0/2839060844 shutdown_connections 2024-04-25T18:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.170+0000 7f710ba00640 1 --2- 172.21.15.112:0/2839060844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7104109470 0x7f7104111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.170+0000 7f710ba00640 1 --2- 172.21.15.112:0/2839060844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7104104d50 0x7f7104108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.170+0000 7f710ba00640 1 --2- 172.21.15.112:0/2839060844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71041043a0 0x7f7104104780 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.170+0000 7f710ba00640 1 -- 172.21.15.112:0/2839060844 >> 172.21.15.112:0/2839060844 conn(0x7f71040ffc40 msgr2=0x7f7104102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.170+0000 7f710ba00640 1 -- 172.21.15.112:0/2839060844 shutdown_connections 2024-04-25T18:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.170+0000 7f710ba00640 1 -- 172.21.15.112:0/2839060844 wait complete. 2024-04-25T18:54:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.171+0000 7f710ba00640 1 Processor -- start 2024-04-25T18:54:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.171+0000 7f710ba00640 1 -- start start 2024-04-25T18:54:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.172+0000 7f710ba00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71041043a0 0x7f71041a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.172+0000 7f710ba00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7104109470 0x7f71041a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.172+0000 7f710ba00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f710419a740 0x7f710419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.172+0000 7f710ba00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7104114420 con 0x7f7104109470 2024-04-25T18:54:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.172+0000 7f710ba00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f71041142a0 con 0x7f710419a740 2024-04-25T18:54:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.172+0000 7f710ba00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71041145a0 con 0x7f71041043a0 2024-04-25T18:54:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.172+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7104109470 0x7f71041a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.172+0000 7f7109000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71041043a0 0x7f71041a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.172+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7104109470 0x7f71041a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42754/0 (socket says 172.21.15.112:42754) 2024-04-25T18:54:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.172+0000 7f70fbe00640 1 -- 172.21.15.112:0/918786134 learned_addr learned my addr 172.21.15.112:0/918786134 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.173+0000 7f7109a00640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f710419a740 0x7f710419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.173+0000 7f7109000640 1 -- 172.21.15.112:0/918786134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f710419a740 msgr2=0x7f710419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.173+0000 7f7109000640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f710419a740 0x7f710419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.173+0000 7f7109000640 1 -- 172.21.15.112:0/918786134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7104109470 msgr2=0x7f71041a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.173+0000 7f7109000640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7104109470 0x7f71041a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.173+0000 7f7109000640 1 -- 172.21.15.112:0/918786134 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710419b340 con 0x7f71041043a0 2024-04-25T18:54:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.173+0000 7f70fbe00640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7104109470 0x7f71041a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.173+0000 7f7109000640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71041043a0 0x7f71041a0630 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f70f402aa00 tx=0x7f70f4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.174+0000 7f7102000640 1 -- 172.21.15.112:0/918786134 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f403f070 con 0x7f71041043a0 2024-04-25T18:54:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.174+0000 7f710ba00640 1 -- 172.21.15.112:0/918786134 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710419b5d0 con 0x7f71041043a0 2024-04-25T18:54:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.174+0000 7f7102000640 1 -- 172.21.15.112:0/918786134 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70f4031b50 con 0x7f71041043a0 2024-04-25T18:54:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.174+0000 7f7102000640 1 -- 172.21.15.112:0/918786134 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4035030 con 0x7f71041043a0 2024-04-25T18:54:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.174+0000 7f710ba00640 1 -- 172.21.15.112:0/918786134 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71041a7710 con 0x7f71041043a0 2024-04-25T18:54:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.177+0000 7f7102000640 1 -- 172.21.15.112:0/918786134 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f70f4042070 con 0x7f71041043a0 2024-04-25T18:54:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.177+0000 7f710e783740 1 Processor -- start 2024-04-25T18:54:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f710e783740 1 -- start start 2024-04-25T18:54:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f710e783740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 0x56252ed07c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f710e783740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 0x56252ed124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f710e783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed14df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f710e783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56252ec6d900 con 0x56252ed12a00 2024-04-25T18:54:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f710e783740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56252ec13ef0 con 0x56252ed08140 2024-04-25T18:54:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f710e783740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56252ec698c0 con 0x56252ed07820 2024-04-25T18:54:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f7109a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 0x56252ed07c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f7109a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 0x56252ed07c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:45058/0 (socket says 172.21.15.112:45058) 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f7109a00640 1 -- 172.21.15.112:0/3193408450 learned_addr learned my addr 172.21.15.112:0/3193408450 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f7109000640 1 --2- 172.21.15.112:0/3193408450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 0x56252ed124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.178+0000 7f70fbe00640 1 --2- 172.21.15.112:0/3193408450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed14df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.179+0000 7f7109a00640 1 -- 172.21.15.112:0/3193408450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 msgr2=0x56252ed124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.179+0000 7f7109a00640 1 --2- 172.21.15.112:0/3193408450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 0x56252ed124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.179+0000 7f7109a00640 1 -- 172.21.15.112:0/3193408450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 msgr2=0x56252ed14df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.179+0000 7f7109a00640 1 --2- 172.21.15.112:0/3193408450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed14df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.179+0000 7f7109a00640 1 -- 172.21.15.112:0/3193408450 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252e980210 con 0x56252ed07820 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.179+0000 7f7109000640 1 --2- 172.21.15.112:0/3193408450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 0x56252ed124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.179+0000 7f7109a00640 1 --2- 172.21.15.112:0/3193408450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 0x56252ed07c00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009f70 tx=0x7f70fc010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0a96891a17d608c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.179+0000 7f70fa000640 1 -- 172.21.15.112:0/3193408450 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc016070 con 0x56252ed07820 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.179+0000 7f70fa000640 1 -- 172.21.15.112:0/3193408450 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70fc004d20 con 0x56252ed07820 2024-04-25T18:54:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.179+0000 7f70fa000640 1 -- 172.21.15.112:0/3193408450 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc012790 con 0x56252ed07820 2024-04-25T18:54:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.180+0000 7f7102000640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f70e80a3dc0 0x7f70e80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.180+0000 7f7102000640 1 -- 172.21.15.112:0/918786134 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f70f40f0ab0 con 0x7f71041043a0 2024-04-25T18:54:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.180+0000 7f7109000640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f70e80a3dc0 0x7f70e80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.180+0000 7f710e783740 1 -- 172.21.15.112:0/3193408450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 msgr2=0x56252ed07c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.180+0000 7f710e783740 1 --2- 172.21.15.112:0/3193408450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 0x56252ed07c00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009f70 tx=0x7f70fc010800 comp rx=0 tx=0).stop 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.180+0000 7f710e783740 1 -- 172.21.15.112:0/3193408450 shutdown_connections 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.180+0000 7f710e783740 1 --2- 172.21.15.112:0/3193408450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed14df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.180+0000 7f710e783740 1 --2- 172.21.15.112:0/3193408450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 0x56252ed124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.180+0000 7f710e783740 1 --2- 172.21.15.112:0/3193408450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 0x56252ed07c00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.180+0000 7f710e783740 1 -- 172.21.15.112:0/3193408450 >> 172.21.15.112:0/3193408450 conn(0x56252ed03d30 msgr2=0x56252ed04110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.180+0000 7f710e783740 1 -- 172.21.15.112:0/3193408450 shutdown_connections 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.181+0000 7f710e783740 1 -- 172.21.15.112:0/3193408450 wait complete. 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.181+0000 7f7109000640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f70e80a3dc0 0x7f70e80a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f70f40ee280 tx=0x7f70f4031400 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.181+0000 7f710e783740 1 -- start start 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.181+0000 7f710e783740 1 Processor -- start 2024-04-25T18:54:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f710e783740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed07820 0x56252ed0f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f710e783740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed0ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f710e783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f710e783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56252ed15630 con 0x56252ed12a00 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f710e783740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56252ec27310 con 0x56252ed07820 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f710e783740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56252ed157b0 con 0x56252ed08140 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f710e783740 10 client.0 _ll_register_callbacks cb 0x56252ebf5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed0ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f7109a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed07820 0x56252ed0f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed0ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:45070/0 (socket says 172.21.15.112:45070) 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f70fbe00640 1 -- 172.21.15.112:0/3592539368 learned_addr learned my addr 172.21.15.112:0/3592539368 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f7109a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed07820 0x56252ed0f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:40920/0 (socket says 172.21.15.112:40920) 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.182+0000 7f7109a00640 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f70fbe00640 1 -- 172.21.15.112:0/3592539368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed07820 msgr2=0x56252ed0f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f70fbe00640 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed07820 0x56252ed0f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f70fbe00640 1 -- 172.21.15.112:0/3592539368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 msgr2=0x56252ed10450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f70fbe00640 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed10450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f70fbe00640 1 -- 172.21.15.112:0/3592539368 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252ea4c620 con 0x56252ed08140 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f70fbe00640 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed0ff10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70ec012d50 tx=0x7f70ec00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a92511a6dcc249f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f70f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f70f2a00640 1 -- 172.21.15.112:0/3592539368 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec0120d0 con 0x56252ed08140 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f70f2a00640 1 -- 172.21.15.112:0/3592539368 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f70ec010430 con 0x56252ed08140 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f710e783740 1 -- 172.21.15.112:0/3592539368 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70fc004d20 con 0x56252ed08140 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f70f2a00640 1 -- 172.21.15.112:0/3592539368 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f70ec012270 con 0x56252ed08140 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f710e783740 10 client.25903 fetch_fsmap learned FSMap version 36 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.183+0000 7f710e783740 1 -- 172.21.15.112:0/3592539368 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56252ed21940 con 0x56252ed08140 2024-04-25T18:54:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.184+0000 7f70f2a00640 1 -- 172.21.15.112:0/3592539368 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70fc004d20 con 0x56252ed08140 2024-04-25T18:54:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.184+0000 7f70f2a00640 1 -- 172.21.15.112:0/3592539368 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f70ec024550 con 0x56252ed08140 2024-04-25T18:54:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.184+0000 7f710e783740 10 client.25903 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:54:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.184+0000 7f710e783740 10 client.25903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:54:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.184+0000 7f710e783740 20 client.25903 populate_metadata read hostname 'smithi112' 2024-04-25T18:54:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.184+0000 7f710e783740 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56252ed22ea0 0x56252ed25290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.184+0000 7f710e783740 4 client.25903 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:54:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.184+0000 7f710e783740 1 -- 172.21.15.112:0/3592539368 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56252ec818b0 con 0x56252ed22ea0 2024-04-25T18:54:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.185+0000 7f7109a00640 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56252ed22ea0 0x56252ed25290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.185+0000 7f7109a00640 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56252ed22ea0 0x56252ed25290 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a51370dcabcd09 server_cookie=a3a8ef5a4152f314 in_seq=0 out_seq=0 2024-04-25T18:54:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.185+0000 7f70f2a00640 10 client.25903 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f70f2a00640 1 -- 172.21.15.112:0/3592539368 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56252ec818b0 con 0x56252ed22ea0 2024-04-25T18:54:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f70f2a00640 10 client.25903 handle_command_reply: tid=0 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f710e783740 1 client.25903 shutdown 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f710e783740 1 -- 172.21.15.112:0/3592539368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 msgr2=0x56252ed0ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f710e783740 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed0ff10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70ec012d50 tx=0x7f70ec00f790 comp rx=0 tx=0).stop 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f710e783740 1 -- 172.21.15.112:0/3592539368 shutdown_connections 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f710e783740 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56252ed22ea0 0x56252ed25290 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f710e783740 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed10450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f710e783740 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed0ff10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f710e783740 1 --2- 172.21.15.112:0/3592539368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed07820 0x56252ed0f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f710e783740 1 -- 172.21.15.112:0/3592539368 >> 172.21.15.112:0/3592539368 conn(0x56252ed03d30 msgr2=0x56252ed09a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f70f2a00640 0 client.25903 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f710e783740 1 -- 172.21.15.112:0/3592539368 shutdown_connections 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.186+0000 7f710e783740 1 -- 172.21.15.112:0/3592539368 wait complete. 2024-04-25T18:54:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.187+0000 7f710e783740 20 client.25903 trim_cache size 0 max 16384 2024-04-25T18:54:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.204+0000 7f710e783740 1 Processor -- start 2024-04-25T18:54:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.204+0000 7f710e783740 1 -- start start 2024-04-25T18:54:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.205+0000 7f710e783740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed07820 0x56252ed1e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.205+0000 7f710e783740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed10270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.205+0000 7f710e783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed2aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.205+0000 7f710e783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71041b26c0 con 0x56252ed12a00 2024-04-25T18:54:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.205+0000 7f710e783740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f71041aff90 con 0x56252ed07820 2024-04-25T18:54:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.205+0000 7f710e783740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71041ade00 con 0x56252ed08140 2024-04-25T18:54:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.205+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed10270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.205+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed10270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:45080/0 (socket says 172.21.15.112:45080) 2024-04-25T18:54:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.205+0000 7f70fbe00640 1 -- 172.21.15.112:0/2838761238 learned_addr learned my addr 172.21.15.112:0/2838761238 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.205+0000 7f7109a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed07820 0x56252ed1e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.205+0000 7f7109a00640 1 --2- 172.21.15.112:0/2838761238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed2aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.206+0000 7f70fbe00640 1 -- 172.21.15.112:0/2838761238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed07820 msgr2=0x56252ed1e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.206+0000 7f70fbe00640 1 --2- 172.21.15.112:0/2838761238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed07820 0x56252ed1e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.206+0000 7f70fbe00640 1 -- 172.21.15.112:0/2838761238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 msgr2=0x56252ed2aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.206+0000 7f70fbe00640 1 --2- 172.21.15.112:0/2838761238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed2aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.206+0000 7f70fbe00640 1 -- 172.21.15.112:0/2838761238 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252ed15120 con 0x56252ed08140 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.206+0000 7f7109a00640 1 --2- 172.21.15.112:0/2838761238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed2aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.206+0000 7f70fbe00640 1 --2- 172.21.15.112:0/2838761238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed10270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70ec024b80 tx=0x7f70ec009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=390c964c6e8d556e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.206+0000 7f70f9600640 1 -- 172.21.15.112:0/2838761238 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec024c00 con 0x56252ed08140 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.206+0000 7f70f9600640 1 -- 172.21.15.112:0/2838761238 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70ec024da0 con 0x56252ed08140 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.206+0000 7f70f9600640 1 -- 172.21.15.112:0/2838761238 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec004680 con 0x56252ed08140 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.207+0000 7f710e783740 1 -- 172.21.15.112:0/2838761238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 msgr2=0x56252ed10270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.207+0000 7f710e783740 1 --2- 172.21.15.112:0/2838761238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed10270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70ec024b80 tx=0x7f70ec009530 comp rx=0 tx=0).stop 2024-04-25T18:54:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.207+0000 7f710e783740 1 -- 172.21.15.112:0/2838761238 shutdown_connections 2024-04-25T18:54:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.207+0000 7f710e783740 1 --2- 172.21.15.112:0/2838761238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed2aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.207+0000 7f710e783740 1 --2- 172.21.15.112:0/2838761238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed08140 0x56252ed10270 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.207+0000 7f710e783740 1 --2- 172.21.15.112:0/2838761238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed07820 0x56252ed1e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.207+0000 7f710e783740 1 -- 172.21.15.112:0/2838761238 >> 172.21.15.112:0/2838761238 conn(0x56252ed03d30 msgr2=0x56252ed02e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.207+0000 7f710e783740 1 -- 172.21.15.112:0/2838761238 shutdown_connections 2024-04-25T18:54:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.207+0000 7f710e783740 1 -- 172.21.15.112:0/2838761238 wait complete. 2024-04-25T18:54:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.208+0000 7f710e783740 1 -- start start 2024-04-25T18:54:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.208+0000 7f710e783740 1 Processor -- start 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.208+0000 7f710e783740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 0x56252ed18470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.208+0000 7f710e783740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 0x56252ed189b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.208+0000 7f710e783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed18ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.208+0000 7f710e783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71041ae450 con 0x56252ed12a00 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.208+0000 7f710e783740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f71041af8c0 con 0x56252ed08140 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.208+0000 7f710e783740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71041aeb20 con 0x56252ed07820 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f710e783740 10 client.0 _ll_register_callbacks cb 0x7f70ec012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 0x56252ed189b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 0x56252ed189b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:40944/0 (socket says 172.21.15.112:40944) 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f70fbe00640 1 -- 172.21.15.112:0/1255256047 learned_addr learned my addr 172.21.15.112:0/1255256047 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f7109a00640 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed18ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f7109a00640 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 0x56252ed18470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f70fbe00640 1 -- 172.21.15.112:0/1255256047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 msgr2=0x56252ed18470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f70fbe00640 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 0x56252ed18470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f70fbe00640 1 -- 172.21.15.112:0/1255256047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 msgr2=0x56252ed18ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f70fbe00640 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed18ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f70fbe00640 1 -- 172.21.15.112:0/1255256047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252ed88450 con 0x56252ed08140 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f7109a00640 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed18ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f70fbe00640 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 0x56252ed189b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0095e0 tx=0x7f70ec00fd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6223a7fe9d7d523 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.209+0000 7f70f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.210+0000 7f70f2a00640 1 -- 172.21.15.112:0/1255256047 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec024bb0 con 0x56252ed08140 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.210+0000 7f70f2a00640 1 -- 172.21.15.112:0/1255256047 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f70ec024640 con 0x56252ed08140 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.210+0000 7f710e783740 1 -- 172.21.15.112:0/1255256047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70ec024da0 con 0x56252ed08140 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.210+0000 7f70f2a00640 1 -- 172.21.15.112:0/1255256047 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f70ec010430 con 0x56252ed08140 2024-04-25T18:54:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.210+0000 7f710e783740 10 client.25880 fetch_fsmap learned FSMap version 36 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.210+0000 7f710e783740 1 -- 172.21.15.112:0/1255256047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56252ed89740 con 0x56252ed08140 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.210+0000 7f70f2a00640 1 -- 172.21.15.112:0/1255256047 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70ec024da0 con 0x56252ed08140 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.211+0000 7f70f2a00640 1 -- 172.21.15.112:0/1255256047 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f70ec02d550 con 0x56252ed08140 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.211+0000 7f710e783740 10 client.25880 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.211+0000 7f710e783740 10 client.25880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.211+0000 7f710e783740 20 client.25880 populate_metadata read hostname 'smithi112' 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.211+0000 7f710e783740 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56252ed89df0 0x56252ed8c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.211+0000 7f710e783740 4 client.25880 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.211+0000 7f710e783740 1 -- 172.21.15.112:0/1255256047 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56252ec7b320 con 0x56252ed89df0 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.211+0000 7f7109a00640 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56252ed89df0 0x56252ed8c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.211+0000 7f7109a00640 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56252ed89df0 0x56252ed8c1e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4013210f4f7383a8 server_cookie=92e19ab7676501c0 in_seq=0 out_seq=0 2024-04-25T18:54:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.211+0000 7f70f2a00640 10 client.25880 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.232+0000 7f70f2a00640 1 -- 172.21.15.112:0/1255256047 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56252ec7b320 con 0x56252ed89df0 2024-04-25T18:54:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.232+0000 7f70f2a00640 10 client.25880 handle_command_reply: tid=0 2024-04-25T18:54:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.232+0000 7f710e783740 1 client.25880 shutdown 2024-04-25T18:54:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.233+0000 7f710e783740 1 -- 172.21.15.112:0/1255256047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 msgr2=0x56252ed189b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.233+0000 7f710e783740 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 0x56252ed189b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0095e0 tx=0x7f70ec00fd10 comp rx=0 tx=0).stop 2024-04-25T18:54:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.234+0000 7f710e783740 1 -- 172.21.15.112:0/1255256047 shutdown_connections 2024-04-25T18:54:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.234+0000 7f710e783740 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x56252ed89df0 0x56252ed8c1e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.234+0000 7f710e783740 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56252ed12a00 0x56252ed18ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.234+0000 7f710e783740 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56252ed08140 0x56252ed189b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.234+0000 7f710e783740 1 --2- 172.21.15.112:0/1255256047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56252ed07820 0x56252ed18470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.234+0000 7f710e783740 1 -- 172.21.15.112:0/1255256047 >> 172.21.15.112:0/1255256047 conn(0x56252ed03d30 msgr2=0x56252ed091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.234+0000 7f70f2a00640 0 client.25880 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.235+0000 7f710e783740 1 -- 172.21.15.112:0/1255256047 shutdown_connections 2024-04-25T18:54:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.235+0000 7f710e783740 1 -- 172.21.15.112:0/1255256047 wait complete. 2024-04-25T18:54:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.236+0000 7f710e783740 20 client.25880 trim_cache size 0 max 16384 2024-04-25T18:54:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:54:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:54:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:54:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:54:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.236+0000 7f710ba00640 1 -- 172.21.15.112:0/918786134 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f70e80a3dc0 msgr2=0x7f70e80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.236+0000 7f710ba00640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f70e80a3dc0 0x7f70e80a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f70f40ee280 tx=0x7f70f4031400 comp rx=0 tx=0).stop 2024-04-25T18:54:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.236+0000 7f710ba00640 1 -- 172.21.15.112:0/918786134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71041043a0 msgr2=0x7f71041a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.236+0000 7f710ba00640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71041043a0 0x7f71041a0630 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f70f402aa00 tx=0x7f70f4003b90 comp rx=0 tx=0).stop 2024-04-25T18:54:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.237+0000 7f710ba00640 1 -- 172.21.15.112:0/918786134 shutdown_connections 2024-04-25T18:54:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.237+0000 7f710ba00640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f70e80a3dc0 0x7f70e80a6280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.237+0000 7f710ba00640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f710419a740 0x7f710419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.237+0000 7f710ba00640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7104109470 0x7f71041a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.237+0000 7f710ba00640 1 --2- 172.21.15.112:0/918786134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71041043a0 0x7f71041a0630 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.237+0000 7f710ba00640 1 -- 172.21.15.112:0/918786134 >> 172.21.15.112:0/918786134 conn(0x7f71040ffc40 msgr2=0x7f7104101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.237+0000 7f710ba00640 1 -- 172.21.15.112:0/918786134 shutdown_connections 2024-04-25T18:54:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:39.237+0000 7f710ba00640 1 -- 172.21.15.112:0/918786134 wait complete. 2024-04-25T18:54:40.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:39 smithi161 ceph-mon[32609]: pgmap v225: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 355 op/s 2024-04-25T18:54:40.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:39 smithi112 ceph-mon[27864]: pgmap v225: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 355 op/s 2024-04-25T18:54:40.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:39 smithi171 ceph-mon[33266]: pgmap v225: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 355 op/s 2024-04-25T18:54:41.888 INFO:tasks.workunit.client.0.smithi112.stdout:truncating to largest ever: 0x63e9b52 2024-04-25T18:54:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:41 smithi161 ceph-mon[32609]: pgmap v226: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 535 op/s 2024-04-25T18:54:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:41 smithi112 ceph-mon[27864]: pgmap v226: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 535 op/s 2024-04-25T18:54:42.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:41 smithi171 ceph-mon[33266]: pgmap v226: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 535 op/s 2024-04-25T18:54:43.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:42 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:43.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:42 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:43.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:42 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:43 smithi161 ceph-mon[32609]: pgmap v227: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.0 MiB/s wr, 351 op/s 2024-04-25T18:54:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:43 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:43 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:43 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:54:44.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:43 smithi112 ceph-mon[27864]: pgmap v227: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.0 MiB/s wr, 351 op/s 2024-04-25T18:54:44.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:43 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:44.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:43 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:44.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:43 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:54:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:43 smithi171 ceph-mon[33266]: pgmap v227: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.0 MiB/s wr, 351 op/s 2024-04-25T18:54:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:43 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:43 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:43 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:54:46.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:45 smithi161 ceph-mon[32609]: pgmap v228: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.0 MiB/s wr, 351 op/s 2024-04-25T18:54:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:45 smithi112 ceph-mon[27864]: pgmap v228: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.0 MiB/s wr, 351 op/s 2024-04-25T18:54:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:45 smithi171 ceph-mon[33266]: pgmap v228: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.0 MiB/s wr, 351 op/s 2024-04-25T18:54:46.801 INFO:tasks.workunit.client.0.smithi112.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-04-25T18:54:48.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:47 smithi161 ceph-mon[32609]: pgmap v229: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 518 op/s 2024-04-25T18:54:48.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:47 smithi112 ceph-mon[27864]: pgmap v229: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 518 op/s 2024-04-25T18:54:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:47 smithi171 ceph-mon[33266]: pgmap v229: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 518 op/s 2024-04-25T18:54:50.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:49 smithi161 ceph-mon[32609]: pgmap v230: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1018 KiB/s wr, 346 op/s 2024-04-25T18:54:50.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:49 smithi112 ceph-mon[27864]: pgmap v230: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1018 KiB/s wr, 346 op/s 2024-04-25T18:54:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:49 smithi171 ceph-mon[33266]: pgmap v230: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1018 KiB/s wr, 346 op/s 2024-04-25T18:54:51.669 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x63ed6b9 2024-04-25T18:54:52.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:51 smithi161 ceph-mon[32609]: pgmap v231: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-04-25T18:54:52.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:51 smithi112 ceph-mon[27864]: pgmap v231: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-04-25T18:54:52.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:51 smithi171 ceph-mon[33266]: pgmap v231: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-04-25T18:54:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.397+0000 7fb4aec00640 1 -- 172.21.15.112:0/261396387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a81031e0 msgr2=0x7fb4a81035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.397+0000 7fb4aec00640 1 --2- 172.21.15.112:0/261396387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a81031e0 0x7fb4a81035c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb498009ca0 tx=0x7fb49802a9e0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.397+0000 7fb4aec00640 1 -- 172.21.15.112:0/261396387 shutdown_connections 2024-04-25T18:54:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.397+0000 7fb4aec00640 1 --2- 172.21.15.112:0/261396387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a810c520 0x7fb4a810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.397+0000 7fb4aec00640 1 --2- 172.21.15.112:0/261396387 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a8103b00 0x7fb4a810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.397+0000 7fb4aec00640 1 --2- 172.21.15.112:0/261396387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a81031e0 0x7fb4a81035c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.397+0000 7fb4aec00640 1 -- 172.21.15.112:0/261396387 >> 172.21.15.112:0/261396387 conn(0x7fb4a8077fd0 msgr2=0x7fb4a80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.397+0000 7fb4aec00640 1 -- 172.21.15.112:0/261396387 shutdown_connections 2024-04-25T18:54:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.397+0000 7fb4aec00640 1 -- 172.21.15.112:0/261396387 wait complete. 2024-04-25T18:54:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.398+0000 7fb4aec00640 1 Processor -- start 2024-04-25T18:54:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.398+0000 7fb4aec00640 1 -- start start 2024-04-25T18:54:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.398+0000 7fb4aec00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a81031e0 0x7fb4a8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4aec00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a8103b00 0x7fb4a8198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4aec00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a810c520 0x7fb4a819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4aec00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8112200 con 0x7fb4a81031e0 2024-04-25T18:54:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4aec00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8112080 con 0x7fb4a810c520 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4aec00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8112380 con 0x7fb4a8103b00 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4acc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a810c520 0x7fb4a819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4acc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a810c520 0x7fb4a819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:55794/0 (socket says 172.21.15.112:55794) 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4acc00640 1 -- 172.21.15.112:0/1377134212 learned_addr learned my addr 172.21.15.112:0/1377134212 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb49f400640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a8103b00 0x7fb4a8198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a81031e0 0x7fb4a8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4acc00640 1 -- 172.21.15.112:0/1377134212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a8103b00 msgr2=0x7fb4a8198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4acc00640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a8103b00 0x7fb4a8198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4acc00640 1 -- 172.21.15.112:0/1377134212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a81031e0 msgr2=0x7fb4a8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4acc00640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a81031e0 0x7fb4a8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.399+0000 7fb4acc00640 1 -- 172.21.15.112:0/1377134212 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a819d6f0 con 0x7fb4a810c520 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.400+0000 7fb4acc00640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a810c520 0x7fb4a819cfc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000e9b0 tx=0x7fb4a000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.400+0000 7fb4a5600640 1 -- 172.21.15.112:0/1377134212 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a000cd80 con 0x7fb4a810c520 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.400+0000 7fb4a5600640 1 -- 172.21.15.112:0/1377134212 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a00035f0 con 0x7fb4a810c520 2024-04-25T18:54:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.400+0000 7fb4a5600640 1 -- 172.21.15.112:0/1377134212 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0010690 con 0x7fb4a810c520 2024-04-25T18:54:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.400+0000 7fb4aec00640 1 -- 172.21.15.112:0/1377134212 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a819d9e0 con 0x7fb4a810c520 2024-04-25T18:54:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.400+0000 7fb4aec00640 1 -- 172.21.15.112:0/1377134212 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a81a5270 con 0x7fb4a810c520 2024-04-25T18:54:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.403+0000 7fb4a5600640 1 -- 172.21.15.112:0/1377134212 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fb4a0004020 con 0x7fb4a810c520 2024-04-25T18:54:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.404+0000 7fb4b19ce740 1 Processor -- start 2024-04-25T18:54:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.404+0000 7fb4b19ce740 1 -- start start 2024-04-25T18:54:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.404+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349b740 0x56419349bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.404+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56419349c060 0x5641934aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.404+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641934aae10 0x5641934ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.404+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564193401900 con 0x56419349b740 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.404+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5641933a7ef0 con 0x5641934aae10 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.404+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5641933fd8c0 con 0x56419349c060 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.404+0000 7fb49f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349b740 0x56419349bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.404+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56419349c060 0x5641934aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.405+0000 7fb49f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349b740 0x56419349bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39178/0 (socket says 172.21.15.112:39178) 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.405+0000 7fb49f400640 1 -- 172.21.15.112:0/1784964275 learned_addr learned my addr 172.21.15.112:0/1784964275 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.405+0000 7fb49f400640 1 --2- 172.21.15.112:0/1784964275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641934aae10 0x5641934ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.405+0000 7fb4a7e00640 1 -- 172.21.15.112:0/1784964275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641934aae10 msgr2=0x5641934ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.405+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/1784964275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641934aae10 0x5641934ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.405+0000 7fb4a7e00640 1 -- 172.21.15.112:0/1784964275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349b740 msgr2=0x56419349bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.405+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/1784964275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349b740 0x56419349bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.405+0000 7fb4a7e00640 1 -- 172.21.15.112:0/1784964275 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564193114210 con 0x56419349c060 2024-04-25T18:54:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.405+0000 7fb4a5600640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb48c0a3e40 0x7fb48c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.405+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/1784964275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56419349c060 0x5641934aa8d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb498005e30 tx=0x7fb498009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0f9d3bebf95590f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.406+0000 7fb4a5600640 1 -- 172.21.15.112:0/1377134212 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fb4a00d0aa0 con 0x7fb4a810c520 2024-04-25T18:54:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.406+0000 7fb49d200640 1 -- 172.21.15.112:0/1784964275 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498049070 con 0x56419349c060 2024-04-25T18:54:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.406+0000 7fb49d200640 1 -- 172.21.15.112:0/1784964275 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49802e840 con 0x56419349c060 2024-04-25T18:54:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.406+0000 7fb49d200640 1 -- 172.21.15.112:0/1784964275 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498033040 con 0x56419349c060 2024-04-25T18:54:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.406+0000 7fb49f400640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb48c0a3e40 0x7fb48c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.407+0000 7fb4b19ce740 1 -- 172.21.15.112:0/1784964275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56419349c060 msgr2=0x5641934aa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.407+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/1784964275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56419349c060 0x5641934aa8d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb498005e30 tx=0x7fb498009a20 comp rx=0 tx=0).stop 2024-04-25T18:54:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.407+0000 7fb49f400640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb48c0a3e40 0x7fb48c0a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb490009fd0 tx=0x7fb490015040 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.407+0000 7fb4b19ce740 1 -- 172.21.15.112:0/1784964275 shutdown_connections 2024-04-25T18:54:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.407+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/1784964275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5641934aae10 0x5641934ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.407+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/1784964275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56419349c060 0x5641934aa8d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.407+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/1784964275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349b740 0x56419349bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.407+0000 7fb4b19ce740 1 -- 172.21.15.112:0/1784964275 >> 172.21.15.112:0/1784964275 conn(0x564193497d30 msgr2=0x564193498110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.407+0000 7fb4b19ce740 1 -- 172.21.15.112:0/1784964275 shutdown_connections 2024-04-25T18:54:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.407+0000 7fb4b19ce740 1 -- 172.21.15.112:0/1784964275 wait complete. 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.407+0000 7fb4b19ce740 1 -- start start 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.408+0000 7fb4b19ce740 1 Processor -- start 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.408+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349b740 0x5641934a7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.408+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934a8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.408+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5641934aae10 0x5641934a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.408+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5641934ad9e0 con 0x56419349b740 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.408+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5641933bb310 con 0x56419349c060 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.408+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5641934adb60 con 0x5641934aae10 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.408+0000 7fb4b19ce740 10 client.0 _ll_register_callbacks cb 0x564193389cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.408+0000 7fb49f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.409+0000 7fb49f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934a8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:55810/0 (socket says 172.21.15.112:55810) 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.409+0000 7fb49f400640 1 -- 172.21.15.112:0/3450551504 learned_addr learned my addr 172.21.15.112:0/3450551504 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.409+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349b740 0x5641934a7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.409+0000 7fb4acc00640 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5641934aae10 0x5641934a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.409+0000 7fb49f400640 1 -- 172.21.15.112:0/3450551504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5641934aae10 msgr2=0x5641934a88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.409+0000 7fb49f400640 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5641934aae10 0x5641934a88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.409+0000 7fb49f400640 1 -- 172.21.15.112:0/3450551504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349b740 msgr2=0x5641934a7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.409+0000 7fb49f400640 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349b740 0x5641934a7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.409+0000 7fb49f400640 1 -- 172.21.15.112:0/3450551504 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641931e0620 con 0x56419349c060 2024-04-25T18:54:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.410+0000 7fb49f400640 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934a8380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb490004570 tx=0x7fb490005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62a7b7265735106c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.410+0000 7fb495600640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:54:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.410+0000 7fb495600640 1 -- 172.21.15.112:0/3450551504 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490005220 con 0x56419349c060 2024-04-25T18:54:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.410+0000 7fb495600640 1 -- 172.21.15.112:0/3450551504 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fb4900053c0 con 0x56419349c060 2024-04-25T18:54:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.410+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3450551504 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49802e840 con 0x56419349c060 2024-04-25T18:54:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.410+0000 7fb495600640 1 -- 172.21.15.112:0/3450551504 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb4900055c0 con 0x56419349c060 2024-04-25T18:54:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.411+0000 7fb4b19ce740 10 client.25892 fetch_fsmap learned FSMap version 36 2024-04-25T18:54:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.411+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3450551504 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5641934b9d30 con 0x56419349c060 2024-04-25T18:54:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.411+0000 7fb495600640 1 -- 172.21.15.112:0/3450551504 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49802e840 con 0x56419349c060 2024-04-25T18:54:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.411+0000 7fb495600640 1 -- 172.21.15.112:0/3450551504 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fb49001f030 con 0x56419349c060 2024-04-25T18:54:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.412+0000 7fb4b19ce740 10 client.25892 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:54:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.412+0000 7fb4b19ce740 10 client.25892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:54:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.412+0000 7fb4b19ce740 20 client.25892 populate_metadata read hostname 'smithi112' 2024-04-25T18:54:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.412+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5641934bb1a0 0x5641934bd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.412+0000 7fb4b19ce740 4 client.25892 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:54:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.412+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3450551504 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641934158b0 con 0x5641934bb1a0 2024-04-25T18:54:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.415+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5641934bb1a0 0x5641934bd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.417+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5641934bb1a0 0x5641934bd610 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f483c9fe9e796c03 server_cookie=40be3bc08f921027 in_seq=0 out_seq=0 2024-04-25T18:54:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.417+0000 7fb495600640 10 client.25892 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.418+0000 7fb495600640 1 -- 172.21.15.112:0/3450551504 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641934158b0 con 0x5641934bb1a0 2024-04-25T18:54:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.418+0000 7fb495600640 10 client.25892 handle_command_reply: tid=0 2024-04-25T18:54:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.418+0000 7fb4b19ce740 1 client.25892 shutdown 2024-04-25T18:54:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.418+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3450551504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 msgr2=0x5641934a8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.418+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934a8380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb490004570 tx=0x7fb490005f90 comp rx=0 tx=0).stop 2024-04-25T18:54:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.419+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3450551504 shutdown_connections 2024-04-25T18:54:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.419+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5641934aae10 0x5641934a88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.419+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934a8380 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.419+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5641934bb1a0 0x5641934bd610 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.419+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/3450551504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349b740 0x5641934a7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.419+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3450551504 >> 172.21.15.112:0/3450551504 conn(0x564193497d30 msgr2=0x56419349dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.419+0000 7fb495600640 0 client.25892 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.419+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3450551504 shutdown_connections 2024-04-25T18:54:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.419+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3450551504 wait complete. 2024-04-25T18:54:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.419+0000 7fb4b19ce740 20 client.25892 trim_cache size 0 max 16384 2024-04-25T18:54:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.436+0000 7fb4b19ce740 1 Processor -- start 2024-04-25T18:54:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.436+0000 7fb4b19ce740 1 -- start start 2024-04-25T18:54:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.436+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349b740 0x5641934b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.436+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349c060 0x5641934c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.436+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5641934aae10 0x5641934c3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.436+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81ae2c0 con 0x56419349c060 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.436+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81abb90 con 0x56419349b740 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.437+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81a9a00 con 0x5641934aae10 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.437+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349b740 0x5641934b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.437+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349b740 0x5641934b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:55818/0 (socket says 172.21.15.112:55818) 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.437+0000 7fb49f400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349c060 0x5641934c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.437+0000 7fb4a7e00640 1 -- 172.21.15.112:0/3953069289 learned_addr learned my addr 172.21.15.112:0/3953069289 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.437+0000 7fb4acc00640 1 --2- 172.21.15.112:0/3953069289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5641934aae10 0x5641934c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.437+0000 7fb4a7e00640 1 -- 172.21.15.112:0/3953069289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5641934aae10 msgr2=0x5641934c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.437+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/3953069289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5641934aae10 0x5641934c3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.437+0000 7fb4a7e00640 1 -- 172.21.15.112:0/3953069289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349c060 msgr2=0x5641934c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.437+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/3953069289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349c060 0x5641934c2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.437+0000 7fb4a7e00640 1 -- 172.21.15.112:0/3953069289 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641934a6390 con 0x56419349b740 2024-04-25T18:54:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.438+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/3953069289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349b740 0x5641934b68a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb49802ea60 tx=0x7fb498000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a29f2768732e500f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.438+0000 7fb497e00640 1 -- 172.21.15.112:0/3953069289 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498049040 con 0x56419349b740 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.438+0000 7fb497e00640 1 -- 172.21.15.112:0/3953069289 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49802e840 con 0x56419349b740 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.438+0000 7fb497e00640 1 -- 172.21.15.112:0/3953069289 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498033040 con 0x56419349b740 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.439+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3953069289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349b740 msgr2=0x5641934b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.439+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/3953069289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349b740 0x5641934b68a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb49802ea60 tx=0x7fb498000f40 comp rx=0 tx=0).stop 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.439+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3953069289 shutdown_connections 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.439+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/3953069289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5641934aae10 0x5641934c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.439+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/3953069289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56419349c060 0x5641934c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.439+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/3953069289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349b740 0x5641934b68a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.439+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3953069289 >> 172.21.15.112:0/3953069289 conn(0x564193497d30 msgr2=0x564193498110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.439+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3953069289 shutdown_connections 2024-04-25T18:54:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.439+0000 7fb4b19ce740 1 -- 172.21.15.112:0/3953069289 wait complete. 2024-04-25T18:54:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.440+0000 7fb4b19ce740 1 -- start start 2024-04-25T18:54:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.440+0000 7fb4b19ce740 1 Processor -- start 2024-04-25T18:54:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.440+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56419349b740 0x5641934b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.440+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.440+0000 7fb4b19ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641934aae10 0x5641934b1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.440+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81aa050 con 0x5641934aae10 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.440+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81ab4c0 con 0x56419349c060 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.440+0000 7fb4b19ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81aa720 con 0x56419349b740 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.440+0000 7fb4b19ce740 10 client.0 _ll_register_callbacks cb 0x7fb490004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb49f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb49f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:55834/0 (socket says 172.21.15.112:55834) 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb49f400640 1 -- 172.21.15.112:0/2567663323 learned_addr learned my addr 172.21.15.112:0/2567663323 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56419349b740 0x5641934b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb4acc00640 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641934aae10 0x5641934b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb49f400640 1 -- 172.21.15.112:0/2567663323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56419349b740 msgr2=0x5641934b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb49f400640 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56419349b740 0x5641934b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb49f400640 1 -- 172.21.15.112:0/2567663323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641934aae10 msgr2=0x5641934b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:54:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb49f400640 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641934aae10 0x5641934b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb49f400640 1 -- 172.21.15.112:0/2567663323 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641935209a0 con 0x56419349c060 2024-04-25T18:54:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb49f400640 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934b0f00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb490009c60 tx=0x7fb49002b6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c256a97285d695a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.441+0000 7fb495600640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:54:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.442+0000 7fb495600640 1 -- 172.21.15.112:0/2567663323 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490037070 con 0x56419349c060 2024-04-25T18:54:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.442+0000 7fb495600640 1 -- 172.21.15.112:0/2567663323 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fb490004f70 con 0x56419349c060 2024-04-25T18:54:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.442+0000 7fb4b19ce740 1 -- 172.21.15.112:0/2567663323 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49802e840 con 0x56419349c060 2024-04-25T18:54:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.442+0000 7fb495600640 1 -- 172.21.15.112:0/2567663323 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb490005170 con 0x56419349c060 2024-04-25T18:54:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.442+0000 7fb4b19ce740 10 client.25898 fetch_fsmap learned FSMap version 36 2024-04-25T18:54:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.442+0000 7fb4b19ce740 1 -- 172.21.15.112:0/2567663323 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564193521c70 con 0x56419349c060 2024-04-25T18:54:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.442+0000 7fb495600640 1 -- 172.21.15.112:0/2567663323 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49802e840 con 0x56419349c060 2024-04-25T18:54:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.443+0000 7fb495600640 1 -- 172.21.15.112:0/2567663323 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fb49001f040 con 0x56419349c060 2024-04-25T18:54:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.443+0000 7fb4b19ce740 10 client.25898 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:54:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.443+0000 7fb4b19ce740 10 client.25898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:54:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.443+0000 7fb4b19ce740 20 client.25898 populate_metadata read hostname 'smithi112' 2024-04-25T18:54:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.443+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564193522380 0x5641935247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.443+0000 7fb4b19ce740 4 client.25898 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:54:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.443+0000 7fb4b19ce740 1 -- 172.21.15.112:0/2567663323 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56419340f320 con 0x564193522380 2024-04-25T18:54:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.443+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564193522380 0x5641935247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.444+0000 7fb4a7e00640 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564193522380 0x5641935247a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f40c4a6355a237bb server_cookie=ca5c6a114900bbf9 in_seq=0 out_seq=0 2024-04-25T18:54:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.444+0000 7fb495600640 10 client.25898 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb495600640 1 -- 172.21.15.112:0/2567663323 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56419340f320 con 0x564193522380 2024-04-25T18:54:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb495600640 10 client.25898 handle_command_reply: tid=0 2024-04-25T18:54:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 1 client.25898 shutdown 2024-04-25T18:54:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 1 -- 172.21.15.112:0/2567663323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 msgr2=0x5641934b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934b0f00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb490009c60 tx=0x7fb49002b6a0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 1 -- 172.21.15.112:0/2567663323 shutdown_connections 2024-04-25T18:54:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x564193522380 0x5641935247a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641934aae10 0x5641934b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56419349c060 0x5641934b0f00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb495600640 0 client.25898 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:54:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 1 --2- 172.21.15.112:0/2567663323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56419349b740 0x5641934b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 1 -- 172.21.15.112:0/2567663323 >> 172.21.15.112:0/2567663323 conn(0x564193497d30 msgr2=0x5641934a7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 1 -- 172.21.15.112:0/2567663323 shutdown_connections 2024-04-25T18:54:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 1 -- 172.21.15.112:0/2567663323 wait complete. 2024-04-25T18:54:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.462+0000 7fb4b19ce740 20 client.25898 trim_cache size 0 max 16384 2024-04-25T18:54:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:54:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:54:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:54:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:54:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 -- 172.21.15.112:0/1377134212 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb48c0a3e40 msgr2=0x7fb48c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb48c0a3e40 0x7fb48c0a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb490009fd0 tx=0x7fb490015040 comp rx=0 tx=0).stop 2024-04-25T18:54:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 -- 172.21.15.112:0/1377134212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a810c520 msgr2=0x7fb4a819cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a810c520 0x7fb4a819cfc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000e9b0 tx=0x7fb4a000ee70 comp rx=0 tx=0).stop 2024-04-25T18:54:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 -- 172.21.15.112:0/1377134212 shutdown_connections 2024-04-25T18:54:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb48c0a3e40 0x7fb48c0a6300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a810c520 0x7fb4a819cfc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4a8103b00 0x7fb4a8198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 --2- 172.21.15.112:0/1377134212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4a81031e0 0x7fb4a8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 -- 172.21.15.112:0/1377134212 >> 172.21.15.112:0/1377134212 conn(0x7fb4a8077fd0 msgr2=0x7fb4a80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 -- 172.21.15.112:0/1377134212 shutdown_connections 2024-04-25T18:54:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:54:52.463+0000 7fb4aec00640 1 -- 172.21.15.112:0/1377134212 wait complete. 2024-04-25T18:54:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:53 smithi171 ceph-mon[33266]: pgmap v232: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 958 KiB/s wr, 334 op/s 2024-04-25T18:54:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:53 smithi161 ceph-mon[32609]: pgmap v232: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 958 KiB/s wr, 334 op/s 2024-04-25T18:54:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:53 smithi112 ceph-mon[27864]: pgmap v232: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 958 KiB/s wr, 334 op/s 2024-04-25T18:54:56.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:55 smithi171 ceph-mon[33266]: pgmap v233: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 958 KiB/s wr, 334 op/s 2024-04-25T18:54:56.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:55 smithi161 ceph-mon[32609]: pgmap v233: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 958 KiB/s wr, 334 op/s 2024-04-25T18:54:56.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:55 smithi112 ceph-mon[27864]: pgmap v233: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 958 KiB/s wr, 334 op/s 2024-04-25T18:54:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:57 smithi112 ceph-mon[27864]: pgmap v234: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.5 MiB/s wr, 507 op/s 2024-04-25T18:54:58.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:57 smithi171 ceph-mon[33266]: pgmap v234: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.5 MiB/s wr, 507 op/s 2024-04-25T18:54:58.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:57 smithi161 ceph-mon[32609]: pgmap v234: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.5 MiB/s wr, 507 op/s 2024-04-25T18:55:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:54:59 smithi171 ceph-mon[33266]: pgmap v235: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.1 MiB/s wr, 340 op/s 2024-04-25T18:55:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:54:59 smithi161 ceph-mon[32609]: pgmap v235: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.1 MiB/s wr, 340 op/s 2024-04-25T18:55:00.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:54:59 smithi112 ceph-mon[27864]: pgmap v235: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.1 MiB/s wr, 340 op/s 2024-04-25T18:55:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:01 smithi171 ceph-mon[33266]: pgmap v236: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.6 MiB/s wr, 507 op/s 2024-04-25T18:55:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:01 smithi161 ceph-mon[32609]: pgmap v236: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.6 MiB/s wr, 507 op/s 2024-04-25T18:55:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:01 smithi112 ceph-mon[27864]: pgmap v236: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.6 MiB/s wr, 507 op/s 2024-04-25T18:55:03.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:03 smithi171 ceph-mon[33266]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T18:55:03.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:03 smithi161 ceph-mon[32609]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T18:55:03.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:03 smithi112 ceph-mon[27864]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T18:55:04.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:04 smithi171 ceph-mon[33266]: pgmap v238: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.4 MiB/s wr, 407 op/s 2024-04-25T18:55:04.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:04 smithi161 ceph-mon[32609]: pgmap v238: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.4 MiB/s wr, 407 op/s 2024-04-25T18:55:04.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:04 smithi112 ceph-mon[27864]: pgmap v238: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.4 MiB/s wr, 407 op/s 2024-04-25T18:55:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.626+0000 7f4ed0600640 1 -- 172.21.15.112:0/3889201791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec8109470 msgr2=0x7f4ec8111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.626+0000 7f4ed0600640 1 --2- 172.21.15.112:0/3889201791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec8109470 0x7f4ec8111040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009be0 tx=0x7f4ec002a6d0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.626+0000 7f4ed0600640 1 -- 172.21.15.112:0/3889201791 shutdown_connections 2024-04-25T18:55:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.626+0000 7f4ed0600640 1 --2- 172.21.15.112:0/3889201791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec8109470 0x7f4ec8111040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.626+0000 7f4ed0600640 1 --2- 172.21.15.112:0/3889201791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec8104d50 0x7f4ec8108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.626+0000 7f4ed0600640 1 --2- 172.21.15.112:0/3889201791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec81043a0 0x7f4ec8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.626+0000 7f4ed0600640 1 -- 172.21.15.112:0/3889201791 >> 172.21.15.112:0/3889201791 conn(0x7f4ec80ffc40 msgr2=0x7f4ec8102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.627+0000 7f4ed0600640 1 -- 172.21.15.112:0/3889201791 shutdown_connections 2024-04-25T18:55:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.627+0000 7f4ed0600640 1 -- 172.21.15.112:0/3889201791 wait complete. 2024-04-25T18:55:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ed0600640 1 Processor -- start 2024-04-25T18:55:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ed0600640 1 -- start start 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ed0600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec81043a0 0x7f4ec81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ed0600640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8109470 0x7f4ec81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ed0600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec819a720 0x7f4ec819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ed0600640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81143e0 con 0x7f4ec8109470 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ed0600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec8114260 con 0x7f4ec81043a0 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ed0600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec8114560 con 0x7f4ec819a720 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ecdc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec81043a0 0x7f4ec81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ecd200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8109470 0x7f4ec81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ecdc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec81043a0 0x7f4ec81a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:39632/0 (socket says 172.21.15.112:39632) 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ecdc00640 1 -- 172.21.15.112:0/746538318 learned_addr learned my addr 172.21.15.112:0/746538318 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.628+0000 7f4ece600640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec819a720 0x7f4ec819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.629+0000 7f4ecdc00640 1 -- 172.21.15.112:0/746538318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec819a720 msgr2=0x7f4ec819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.629+0000 7f4ecdc00640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec819a720 0x7f4ec819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.629+0000 7f4ecdc00640 1 -- 172.21.15.112:0/746538318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8109470 msgr2=0x7f4ec81a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.629+0000 7f4ecdc00640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8109470 0x7f4ec81a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.629+0000 7f4ecdc00640 1 -- 172.21.15.112:0/746538318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec819b320 con 0x7f4ec81043a0 2024-04-25T18:55:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.629+0000 7f4ecd200640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8109470 0x7f4ec81a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.629+0000 7f4ecdc00640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec81043a0 0x7f4ec81a0660 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00e9c0 tx=0x7f4ebc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.629+0000 7f4ec6000640 1 -- 172.21.15.112:0/746538318 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00cde0 con 0x7f4ec81043a0 2024-04-25T18:55:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.630+0000 7f4ed0600640 1 -- 172.21.15.112:0/746538318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec81a7400 con 0x7f4ec81043a0 2024-04-25T18:55:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.630+0000 7f4ed0600640 1 -- 172.21.15.112:0/746538318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec81a7860 con 0x7f4ec81043a0 2024-04-25T18:55:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.630+0000 7f4ec6000640 1 -- 172.21.15.112:0/746538318 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ebc0035f0 con 0x7f4ec81043a0 2024-04-25T18:55:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.630+0000 7f4ec6000640 1 -- 172.21.15.112:0/746538318 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc010690 con 0x7f4ec81043a0 2024-04-25T18:55:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.633+0000 7f4ed338f740 1 Processor -- start 2024-04-25T18:55:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.633+0000 7f4ed338f740 1 -- start start 2024-04-25T18:55:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.633+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc56210c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.633+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc56211140 0x55bc5621b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.633+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 0x55bc5621ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.633+0000 7f4ed338f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc56176900 con 0x55bc56210820 2024-04-25T18:55:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.633+0000 7f4ed338f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bc5611cef0 con 0x55bc56211140 2024-04-25T18:55:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.633+0000 7f4ed338f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bc561728c0 con 0x55bc5621ba00 2024-04-25T18:55:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.634+0000 7f4ece600640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc56210c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.634+0000 7f4ecd200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 0x55bc5621ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.634+0000 7f4ece600640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc56210c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59490/0 (socket says 172.21.15.112:59490) 2024-04-25T18:55:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.634+0000 7f4ece600640 1 -- 172.21.15.112:0/1053694631 learned_addr learned my addr 172.21.15.112:0/1053694631 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.634+0000 7f4ecd200640 1 -- 172.21.15.112:0/1053694631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc56211140 msgr2=0x55bc5621b4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T18:55:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.634+0000 7f4ecd200640 1 --2- 172.21.15.112:0/1053694631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc56211140 0x55bc5621b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.634+0000 7f4ecd200640 1 -- 172.21.15.112:0/1053694631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 msgr2=0x55bc56210c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.634+0000 7f4ecd200640 1 --2- 172.21.15.112:0/1053694631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc56210c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.634+0000 7f4ecd200640 1 -- 172.21.15.112:0/1053694631 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc55e89210 con 0x55bc5621ba00 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.634+0000 7f4ec6000640 1 -- 172.21.15.112:0/746538318 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f4ebc003aa0 con 0x7f4ec81043a0 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.634+0000 7f4ecd200640 1 --2- 172.21.15.112:0/1053694631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 0x55bc5621ddf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4ec806b6c0 tx=0x7f4eb40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec7855b9eeaaba76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.635+0000 7f4ebaa00640 1 -- 172.21.15.112:0/1053694631 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4004230 con 0x55bc5621ba00 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.635+0000 7f4ebaa00640 1 -- 172.21.15.112:0/1053694631 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb40043d0 con 0x55bc5621ba00 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.635+0000 7f4ebaa00640 1 -- 172.21.15.112:0/1053694631 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40035f0 con 0x55bc5621ba00 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ed338f740 1 -- 172.21.15.112:0/1053694631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 msgr2=0x55bc5621ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ed338f740 1 --2- 172.21.15.112:0/1053694631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 0x55bc5621ddf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4ec806b6c0 tx=0x7f4eb40118d0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ec6000640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4eb00a3dc0 0x7f4eb00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ec6000640 1 -- 172.21.15.112:0/746538318 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8615+0+0 (secure 0 0 0) 0x7f4ebc0d0bf0 con 0x7f4ec81043a0 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ed338f740 1 -- 172.21.15.112:0/1053694631 shutdown_connections 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ed338f740 1 --2- 172.21.15.112:0/1053694631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 0x55bc5621ddf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ed338f740 1 --2- 172.21.15.112:0/1053694631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc56211140 0x55bc5621b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ed338f740 1 --2- 172.21.15.112:0/1053694631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc56210c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ed338f740 1 -- 172.21.15.112:0/1053694631 >> 172.21.15.112:0/1053694631 conn(0x55bc5620cd30 msgr2=0x55bc5620d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ece600640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4eb00a3dc0 0x7f4eb00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ed338f740 1 -- 172.21.15.112:0/1053694631 shutdown_connections 2024-04-25T18:55:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.636+0000 7f4ed338f740 1 -- 172.21.15.112:0/1053694631 wait complete. 2024-04-25T18:55:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.637+0000 7f4ed338f740 1 -- start start 2024-04-25T18:55:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.637+0000 7f4ece600640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4eb00a3dc0 0x7f4eb00a6280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00099a0 tx=0x7f4ec0005fd0 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.637+0000 7f4ed338f740 1 Processor -- start 2024-04-25T18:55:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56210820 0x55bc56218960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56211140 0x55bc56218ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 0x55bc562193e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ed338f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc5621e630 con 0x55bc56211140 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ed338f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bc56130310 con 0x55bc5621ba00 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ed338f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bc5621e7b0 con 0x55bc56210820 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ed338f740 10 client.0 _ll_register_callbacks cb 0x55bc560fecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ece600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 0x55bc562193e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ecdc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56210820 0x55bc56218960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ecd200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56211140 0x55bc56218ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ece600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 0x55bc562193e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:39642/0 (socket says 172.21.15.112:39642) 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ece600640 1 -- 172.21.15.112:0/2998587954 learned_addr learned my addr 172.21.15.112:0/2998587954 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ecdc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56210820 0x55bc56218960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:45940/0 (socket says 172.21.15.112:45940) 2024-04-25T18:55:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ecdc00640 1 -- 172.21.15.112:0/2998587954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 msgr2=0x55bc562193e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ecdc00640 1 --2- 172.21.15.112:0/2998587954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 0x55bc562193e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ecdc00640 1 -- 172.21.15.112:0/2998587954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56211140 msgr2=0x55bc56218ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.638+0000 7f4ecdc00640 1 --2- 172.21.15.112:0/2998587954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56211140 0x55bc56218ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.639+0000 7f4ecdc00640 1 -- 172.21.15.112:0/2998587954 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc55f55620 con 0x55bc56210820 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.639+0000 7f4ecdc00640 1 --2- 172.21.15.112:0/2998587954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56210820 0x55bc56218960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0ce280 tx=0x7f4ebc09cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b084345aae8081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.639+0000 7f4eaf400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.639+0000 7f4eaf400640 1 -- 172.21.15.112:0/2998587954 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0bd070 con 0x55bc56210820 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.639+0000 7f4ed338f740 1 -- 172.21.15.112:0/2998587954 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb40043d0 con 0x55bc56210820 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.639+0000 7f4eaf400640 1 -- 172.21.15.112:0/2998587954 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8615+0+0 (secure 0 0 0) 0x7f4ebc0037a0 con 0x55bc56210820 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.639+0000 7f4eaf400640 1 -- 172.21.15.112:0/2998587954 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4ebc099140 con 0x55bc56210820 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.639+0000 7f4ed338f740 10 client.25939 fetch_fsmap learned FSMap version 36 2024-04-25T18:55:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.639+0000 7f4ed338f740 1 -- 172.21.15.112:0/2998587954 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bc5622a920 con 0x55bc56210820 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.640+0000 7f4eaf400640 1 -- 172.21.15.112:0/2998587954 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb40043d0 con 0x55bc56210820 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.640+0000 7f4eaf400640 1 -- 172.21.15.112:0/2998587954 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f4ebc09b990 con 0x55bc56210820 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.640+0000 7f4ed338f740 10 client.25939 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.640+0000 7f4ed338f740 10 client.25939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.640+0000 7f4ed338f740 20 client.25939 populate_metadata read hostname 'smithi112' 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.640+0000 7f4ed338f740 1 --2- 172.21.15.112:0/2998587954 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55bc5622bde0 0x55bc5622e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.640+0000 7f4ed338f740 4 client.25939 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.640+0000 7f4ed338f740 1 -- 172.21.15.112:0/2998587954 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc5618a8b0 con 0x55bc5622bde0 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.640+0000 7f4ecd200640 1 --2- 172.21.15.112:0/2998587954 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55bc5622bde0 0x55bc5622e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.641+0000 7f4ecd200640 1 --2- 172.21.15.112:0/2998587954 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55bc5622bde0 0x55bc5622e1d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4c0d2bf6bbc1a9 server_cookie=8e3e65960c077ccf in_seq=0 out_seq=0 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.641+0000 7f4eaf400640 10 client.25939 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4eaf400640 1 -- 172.21.15.112:0/2998587954 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc5618a8b0 con 0x55bc5622bde0 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4eaf400640 10 client.25939 handle_command_reply: tid=0 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 1 client.25939 shutdown 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 1 -- 172.21.15.112:0/2998587954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56210820 msgr2=0x55bc56218960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 1 --2- 172.21.15.112:0/2998587954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56210820 0x55bc56218960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0ce280 tx=0x7f4ebc09cad0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 1 -- 172.21.15.112:0/2998587954 shutdown_connections 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 1 --2- 172.21.15.112:0/2998587954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 0x55bc562193e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 1 --2- 172.21.15.112:0/2998587954 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55bc5622bde0 0x55bc5622e1d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 1 --2- 172.21.15.112:0/2998587954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56211140 0x55bc56218ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 1 --2- 172.21.15.112:0/2998587954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56210820 0x55bc56218960 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 1 -- 172.21.15.112:0/2998587954 >> 172.21.15.112:0/2998587954 conn(0x55bc5620cd30 msgr2=0x55bc56212ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4eaf400640 0 client.25939 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 1 -- 172.21.15.112:0/2998587954 shutdown_connections 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 1 -- 172.21.15.112:0/2998587954 wait complete. 2024-04-25T18:55:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.642+0000 7f4ed338f740 20 client.25939 trim_cache size 0 max 16384 2024-04-25T18:55:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.659+0000 7f4ed338f740 1 Processor -- start 2024-04-25T18:55:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.659+0000 7f4ed338f740 1 -- start start 2024-04-25T18:55:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.659+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc56227520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.659+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56211140 0x55bc562131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.659+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 0x55bc56233a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.659+0000 7f4ed338f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81b2710 con 0x55bc56210820 2024-04-25T18:55:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.659+0000 7f4ed338f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81affe0 con 0x55bc5621ba00 2024-04-25T18:55:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.659+0000 7f4ed338f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81ade50 con 0x55bc56211140 2024-04-25T18:55:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.660+0000 7f4ece600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 0x55bc56233a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.660+0000 7f4ecdc00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc56227520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.660+0000 7f4ecd200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56211140 0x55bc562131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.660+0000 7f4ece600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 0x55bc56233a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:39652/0 (socket says 172.21.15.112:39652) 2024-04-25T18:55:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.660+0000 7f4ece600640 1 -- 172.21.15.112:0/2473898729 learned_addr learned my addr 172.21.15.112:0/2473898729 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.660+0000 7f4ecd200640 1 --2- 172.21.15.112:0/2473898729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56211140 0x55bc562131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:45952/0 (socket says 172.21.15.112:45952) 2024-04-25T18:55:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.660+0000 7f4ecd200640 1 -- 172.21.15.112:0/2473898729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 msgr2=0x55bc56233a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.660+0000 7f4ecd200640 1 --2- 172.21.15.112:0/2473898729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 0x55bc56233a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.660+0000 7f4ecd200640 1 -- 172.21.15.112:0/2473898729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 msgr2=0x55bc56227520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.660+0000 7f4ecd200640 1 --2- 172.21.15.112:0/2473898729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc56227520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.660+0000 7f4ecd200640 1 -- 172.21.15.112:0/2473898729 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc56217930 con 0x55bc56211140 2024-04-25T18:55:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.661+0000 7f4ecd200640 1 --2- 172.21.15.112:0/2473898729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56211140 0x55bc562131e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40096e0 tx=0x7f4eb4004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35e788568fb26718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.661+0000 7f4eba000640 1 -- 172.21.15.112:0/2473898729 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb401e040 con 0x55bc56211140 2024-04-25T18:55:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.661+0000 7f4eba000640 1 -- 172.21.15.112:0/2473898729 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb401f040 con 0x55bc56211140 2024-04-25T18:55:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.661+0000 7f4eba000640 1 -- 172.21.15.112:0/2473898729 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40267e0 con 0x55bc56211140 2024-04-25T18:55:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.662+0000 7f4ed338f740 1 -- 172.21.15.112:0/2473898729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56211140 msgr2=0x55bc562131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.662+0000 7f4ed338f740 1 --2- 172.21.15.112:0/2473898729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56211140 0x55bc562131e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40096e0 tx=0x7f4eb4004580 comp rx=0 tx=0).stop 2024-04-25T18:55:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.662+0000 7f4ed338f740 1 -- 172.21.15.112:0/2473898729 shutdown_connections 2024-04-25T18:55:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.662+0000 7f4ed338f740 1 --2- 172.21.15.112:0/2473898729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc5621ba00 0x55bc56233a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.662+0000 7f4ed338f740 1 --2- 172.21.15.112:0/2473898729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc56211140 0x55bc562131e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.662+0000 7f4ed338f740 1 --2- 172.21.15.112:0/2473898729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc56227520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.662+0000 7f4ed338f740 1 -- 172.21.15.112:0/2473898729 >> 172.21.15.112:0/2473898729 conn(0x55bc5620cd30 msgr2=0x55bc5620be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.662+0000 7f4ed338f740 1 -- 172.21.15.112:0/2473898729 shutdown_connections 2024-04-25T18:55:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.662+0000 7f4ed338f740 1 -- 172.21.15.112:0/2473898729 wait complete. 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.662+0000 7f4ed338f740 1 -- start start 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ed338f740 1 Processor -- start 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc562217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc56211140 0x55bc56221cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ed338f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 0x55bc562912b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ed338f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81ae4a0 con 0x55bc56210820 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ed338f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81af910 con 0x55bc56211140 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ed338f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81aeb70 con 0x55bc5621ba00 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ed338f740 10 client.0 _ll_register_callbacks cb 0x7f4ebc005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ecd200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc56211140 0x55bc56221cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ece600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 0x55bc562912b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ecdc00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc562217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ecd200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc56211140 0x55bc56221cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:39664/0 (socket says 172.21.15.112:39664) 2024-04-25T18:55:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ecd200640 1 -- 172.21.15.112:0/1626581481 learned_addr learned my addr 172.21.15.112:0/1626581481 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.663+0000 7f4ece600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 0x55bc562912b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:45954/0 (socket says 172.21.15.112:45954) 2024-04-25T18:55:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.664+0000 7f4ece600640 1 -- 172.21.15.112:0/1626581481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc56211140 msgr2=0x55bc56221cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.664+0000 7f4ece600640 1 --2- 172.21.15.112:0/1626581481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc56211140 0x55bc56221cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.664+0000 7f4ece600640 1 -- 172.21.15.112:0/1626581481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 msgr2=0x55bc562217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.664+0000 7f4ece600640 1 --2- 172.21.15.112:0/1626581481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc562217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.664+0000 7f4ece600640 1 -- 172.21.15.112:0/1626581481 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc562225f0 con 0x55bc5621ba00 2024-04-25T18:55:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.664+0000 7f4ece600640 1 --2- 172.21.15.112:0/1626581481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 0x55bc562912b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55bc56212060 tx=0x7f4ec0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30c67145a55403eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.664+0000 7f4eaf400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:55:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.664+0000 7f4eaf400640 1 -- 172.21.15.112:0/1626581481 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0036040 con 0x55bc5621ba00 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4ed338f740 1 -- 172.21.15.112:0/1626581481 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb401f040 con 0x55bc5621ba00 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4eaf400640 1 -- 172.21.15.112:0/1626581481 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8615+0+0 (secure 0 0 0) 0x7f4ec0031680 con 0x55bc5621ba00 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4eaf400640 1 -- 172.21.15.112:0/1626581481 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4ec005d050 con 0x55bc5621ba00 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4ed338f740 10 client.25945 fetch_fsmap learned FSMap version 36 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4ed338f740 1 -- 172.21.15.112:0/1626581481 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bc56292950 con 0x55bc5621ba00 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4eaf400640 1 -- 172.21.15.112:0/1626581481 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb401f040 con 0x55bc5621ba00 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4eaf400640 1 -- 172.21.15.112:0/1626581481 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f4ec0004460 con 0x55bc5621ba00 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4ed338f740 10 client.25945 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4ed338f740 10 client.25945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4ed338f740 20 client.25945 populate_metadata read hostname 'smithi112' 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4ed338f740 1 --2- 172.21.15.112:0/1626581481 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55bc56293050 0x55bc56295470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4ed338f740 4 client.25945 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:55:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.665+0000 7f4ed338f740 1 -- 172.21.15.112:0/1626581481 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc56184320 con 0x55bc56293050 2024-04-25T18:55:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.666+0000 7f4ecdc00640 1 --2- 172.21.15.112:0/1626581481 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55bc56293050 0x55bc56295470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.666+0000 7f4eaf400640 10 client.25945 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.666+0000 7f4ecdc00640 1 --2- 172.21.15.112:0/1626581481 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55bc56293050 0x55bc56295470 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5856daf3fb2787be server_cookie=9b5caa96d416ff8e in_seq=0 out_seq=0 2024-04-25T18:55:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.680+0000 7f4eaf400640 1 -- 172.21.15.112:0/1626581481 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc56184320 con 0x55bc56293050 2024-04-25T18:55:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.680+0000 7f4eaf400640 10 client.25945 handle_command_reply: tid=0 2024-04-25T18:55:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.680+0000 7f4ed338f740 1 client.25945 shutdown 2024-04-25T18:55:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.680+0000 7f4ed338f740 1 -- 172.21.15.112:0/1626581481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 msgr2=0x55bc562912b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.680+0000 7f4ed338f740 1 --2- 172.21.15.112:0/1626581481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 0x55bc562912b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55bc56212060 tx=0x7f4ec0009500 comp rx=0 tx=0).stop 2024-04-25T18:55:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.680+0000 7f4ed338f740 1 -- 172.21.15.112:0/1626581481 shutdown_connections 2024-04-25T18:55:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.680+0000 7f4ed338f740 1 --2- 172.21.15.112:0/1626581481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bc5621ba00 0x55bc562912b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.680+0000 7f4ed338f740 1 --2- 172.21.15.112:0/1626581481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bc56211140 0x55bc56221cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.680+0000 7f4ed338f740 1 --2- 172.21.15.112:0/1626581481 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55bc56293050 0x55bc56295470 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.681+0000 7f4ed338f740 1 --2- 172.21.15.112:0/1626581481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc56210820 0x55bc562217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.681+0000 7f4ed338f740 1 -- 172.21.15.112:0/1626581481 >> 172.21.15.112:0/1626581481 conn(0x55bc5620cd30 msgr2=0x55bc56218900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.681+0000 7f4eaf400640 0 client.25945 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.681+0000 7f4ed338f740 1 -- 172.21.15.112:0/1626581481 shutdown_connections 2024-04-25T18:55:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.681+0000 7f4ed338f740 1 -- 172.21.15.112:0/1626581481 wait complete. 2024-04-25T18:55:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.681+0000 7f4ed338f740 20 client.25945 trim_cache size 0 max 16384 2024-04-25T18:55:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:55:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:55:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:55:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:55:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.682+0000 7f4ed0600640 1 -- 172.21.15.112:0/746538318 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4eb00a3dc0 msgr2=0x7f4eb00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.682+0000 7f4ed0600640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4eb00a3dc0 0x7f4eb00a6280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00099a0 tx=0x7f4ec0005fd0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.682+0000 7f4ed0600640 1 -- 172.21.15.112:0/746538318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec81043a0 msgr2=0x7f4ec81a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.682+0000 7f4ed0600640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec81043a0 0x7f4ec81a0660 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00e9c0 tx=0x7f4ebc00ee80 comp rx=0 tx=0).stop 2024-04-25T18:55:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.682+0000 7f4ed0600640 1 -- 172.21.15.112:0/746538318 shutdown_connections 2024-04-25T18:55:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.682+0000 7f4ed0600640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4eb00a3dc0 0x7f4eb00a6280 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.682+0000 7f4ed0600640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec819a720 0x7f4ec819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.682+0000 7f4ed0600640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec8109470 0x7f4ec81a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.682+0000 7f4ed0600640 1 --2- 172.21.15.112:0/746538318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec81043a0 0x7f4ec81a0660 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.682+0000 7f4ed0600640 1 -- 172.21.15.112:0/746538318 >> 172.21.15.112:0/746538318 conn(0x7f4ec80ffc40 msgr2=0x7f4ec8100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.683+0000 7f4ed0600640 1 -- 172.21.15.112:0/746538318 shutdown_connections 2024-04-25T18:55:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:05.683+0000 7f4ed0600640 1 -- 172.21.15.112:0/746538318 wait complete. 2024-04-25T18:55:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:06 smithi171 ceph-mon[33266]: pgmap v239: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.4 MiB/s wr, 407 op/s 2024-04-25T18:55:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:06 smithi161 ceph-mon[32609]: pgmap v239: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.4 MiB/s wr, 407 op/s 2024-04-25T18:55:06.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:06 smithi112 ceph-mon[27864]: pgmap v239: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.4 MiB/s wr, 407 op/s 2024-04-25T18:55:08.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:08 smithi171 ceph-mon[33266]: pgmap v240: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-04-25T18:55:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:08 smithi161 ceph-mon[32609]: pgmap v240: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-04-25T18:55:08.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:08 smithi112 ceph-mon[27864]: pgmap v240: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-04-25T18:55:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:09 smithi171 ceph-mon[33266]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T18:55:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:09 smithi171 ceph-mon[33266]: pgmap v242: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 762 KiB/s wr, 288 op/s 2024-04-25T18:55:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:09 smithi161 ceph-mon[32609]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T18:55:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:09 smithi161 ceph-mon[32609]: pgmap v242: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 762 KiB/s wr, 288 op/s 2024-04-25T18:55:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:09 smithi112 ceph-mon[27864]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T18:55:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:09 smithi112 ceph-mon[27864]: pgmap v242: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 762 KiB/s wr, 288 op/s 2024-04-25T18:55:12.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:11 smithi171 ceph-mon[33266]: pgmap v243: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 504 op/s 2024-04-25T18:55:12.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:11 smithi161 ceph-mon[32609]: pgmap v243: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 504 op/s 2024-04-25T18:55:12.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:11 smithi112 ceph-mon[27864]: pgmap v243: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 504 op/s 2024-04-25T18:55:14.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:13 smithi171 ceph-mon[33266]: pgmap v244: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-04-25T18:55:14.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:13 smithi161 ceph-mon[32609]: pgmap v244: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-04-25T18:55:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:13 smithi112 ceph-mon[27864]: pgmap v244: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-04-25T18:55:16.186 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:15 smithi171 ceph-mon[33266]: pgmap v245: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-04-25T18:55:16.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:15 smithi161 ceph-mon[32609]: pgmap v245: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-04-25T18:55:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:15 smithi112 ceph-mon[27864]: pgmap v245: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-04-25T18:55:16.846 INFO:tasks.workunit.client.0.smithi112.stdout:copying to largest ever: 0x63ef9ee 2024-04-25T18:55:18.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:17 smithi171 ceph-mon[33266]: pgmap v246: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.2 MiB/s wr, 402 op/s 2024-04-25T18:55:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:17 smithi161 ceph-mon[32609]: pgmap v246: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.2 MiB/s wr, 402 op/s 2024-04-25T18:55:18.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:17 smithi112 ceph-mon[27864]: pgmap v246: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.2 MiB/s wr, 402 op/s 2024-04-25T18:55:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.847+0000 7feb19800640 1 -- 172.21.15.112:0/102453834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb1410e8f0 msgr2=0x7feb14110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.847+0000 7feb19800640 1 --2- 172.21.15.112:0/102453834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb1410e8f0 0x7feb14110ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009ca0 tx=0x7feb0c02a8c0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.847+0000 7feb19800640 1 -- 172.21.15.112:0/102453834 shutdown_connections 2024-04-25T18:55:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.847+0000 7feb19800640 1 --2- 172.21.15.112:0/102453834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb1410e8f0 0x7feb14110ce0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.847+0000 7feb19800640 1 --2- 172.21.15.112:0/102453834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb141016f0 0x7feb1410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.847+0000 7feb19800640 1 --2- 172.21.15.112:0/102453834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb14100d40 0x7feb14101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.847+0000 7feb19800640 1 -- 172.21.15.112:0/102453834 >> 172.21.15.112:0/102453834 conn(0x7feb14078060 msgr2=0x7feb14078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.847+0000 7feb19800640 1 -- 172.21.15.112:0/102453834 shutdown_connections 2024-04-25T18:55:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.847+0000 7feb19800640 1 -- 172.21.15.112:0/102453834 wait complete. 2024-04-25T18:55:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.848+0000 7feb19800640 1 Processor -- start 2024-04-25T18:55:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.848+0000 7feb19800640 1 -- start start 2024-04-25T18:55:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.848+0000 7feb19800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb14100d40 0x7feb141a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.848+0000 7feb19800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb141016f0 0x7feb141a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.848+0000 7feb19800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb1410e8f0 0x7feb1419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.848+0000 7feb19800640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb14114440 con 0x7feb1410e8f0 2024-04-25T18:55:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.848+0000 7feb19800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feb141142c0 con 0x7feb141016f0 2024-04-25T18:55:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.848+0000 7feb19800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7feb141145c0 con 0x7feb14100d40 2024-04-25T18:55:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb12000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb141016f0 0x7feb141a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb12a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb14100d40 0x7feb141a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb13400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb1410e8f0 0x7feb1419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb12000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb141016f0 0x7feb141a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:57298/0 (socket says 172.21.15.112:57298) 2024-04-25T18:55:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb12a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb14100d40 0x7feb141a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:44024/0 (socket says 172.21.15.112:44024) 2024-04-25T18:55:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb12000640 1 -- 172.21.15.112:0/3064066748 learned_addr learned my addr 172.21.15.112:0/3064066748 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb12000640 1 -- 172.21.15.112:0/3064066748 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb14100d40 msgr2=0x7feb141a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb12000640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb14100d40 0x7feb141a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb12000640 1 -- 172.21.15.112:0/3064066748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb1410e8f0 msgr2=0x7feb1419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb12000640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb1410e8f0 0x7feb1419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb12000640 1 -- 172.21.15.112:0/3064066748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1419b040 con 0x7feb141016f0 2024-04-25T18:55:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.849+0000 7feb12000640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb141016f0 0x7feb141a0bb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feb0000e9f0 tx=0x7feb0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.850+0000 7feb0b400640 1 -- 172.21.15.112:0/3064066748 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0000cd80 con 0x7feb141016f0 2024-04-25T18:55:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.850+0000 7feb0b400640 1 -- 172.21.15.112:0/3064066748 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb000108e0 con 0x7feb141016f0 2024-04-25T18:55:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.850+0000 7feb0b400640 1 -- 172.21.15.112:0/3064066748 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00018980 con 0x7feb141016f0 2024-04-25T18:55:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.850+0000 7feb19800640 1 -- 172.21.15.112:0/3064066748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1419b330 con 0x7feb141016f0 2024-04-25T18:55:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.850+0000 7feb19800640 1 -- 172.21.15.112:0/3064066748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb141a75f0 con 0x7feb141016f0 2024-04-25T18:55:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.854+0000 7feb1c636740 1 Processor -- start 2024-04-25T18:55:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.854+0000 7feb1c636740 1 -- start start 2024-04-25T18:55:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.854+0000 7feb1c636740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 0x5604f12eec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.854+0000 7feb1c636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 0x5604f12f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.854+0000 7feb1c636740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12f9a00 0x5604f12fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.854+0000 7feb0b400640 1 -- 172.21.15.112:0/3064066748 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7feb00010430 con 0x7feb141016f0 2024-04-25T18:55:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.854+0000 7feb1c636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5604f1254900 con 0x5604f12ef140 2024-04-25T18:55:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.854+0000 7feb1c636740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5604f11faef0 con 0x5604f12ee820 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.854+0000 7feb1c636740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5604f12508c0 con 0x5604f12f9a00 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb13400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 0x5604f12eec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb13400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 0x5604f12eec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:57304/0 (socket says 172.21.15.112:57304) 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb13400640 1 -- 172.21.15.112:0/3952166022 learned_addr learned my addr 172.21.15.112:0/3952166022 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb12a00640 1 --2- 172.21.15.112:0/3952166022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 0x5604f12f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb12000640 1 --2- 172.21.15.112:0/3952166022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12f9a00 0x5604f12fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb13400640 1 -- 172.21.15.112:0/3952166022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12f9a00 msgr2=0x5604f12fbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb13400640 1 --2- 172.21.15.112:0/3952166022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12f9a00 0x5604f12fbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb13400640 1 -- 172.21.15.112:0/3952166022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 msgr2=0x5604f12f94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb13400640 1 --2- 172.21.15.112:0/3952166022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 0x5604f12f94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb13400640 1 -- 172.21.15.112:0/3952166022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f0f67210 con 0x5604f12ee820 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb12a00640 1 --2- 172.21.15.112:0/3952166022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 0x5604f12f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:55:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.855+0000 7feb13400640 1 --2- 172.21.15.112:0/3952166022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 0x5604f12eec00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feb14101d20 tx=0x7feb0c005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=776ed5793d48a460 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.856+0000 7feaffe00640 1 -- 172.21.15.112:0/3952166022 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c031cf0 con 0x5604f12ee820 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.856+0000 7feaffe00640 1 -- 172.21.15.112:0/3952166022 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0c036030 con 0x5604f12ee820 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.856+0000 7feaffe00640 1 -- 172.21.15.112:0/3952166022 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0042a0 con 0x5604f12ee820 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.856+0000 7feb0b400640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7feaf80a3e40 0x7feaf80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.856+0000 7feb0b400640 1 -- 172.21.15.112:0/3064066748 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7feb000d0a60 con 0x7feb141016f0 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.857+0000 7feb1c636740 1 -- 172.21.15.112:0/3952166022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 msgr2=0x5604f12eec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.857+0000 7feb1c636740 1 --2- 172.21.15.112:0/3952166022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 0x5604f12eec00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feb14101d20 tx=0x7feb0c005f80 comp rx=0 tx=0).stop 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.857+0000 7feb12a00640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7feaf80a3e40 0x7feaf80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.857+0000 7feb1c636740 1 -- 172.21.15.112:0/3952166022 shutdown_connections 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.857+0000 7feb1c636740 1 --2- 172.21.15.112:0/3952166022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12f9a00 0x5604f12fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.857+0000 7feb1c636740 1 --2- 172.21.15.112:0/3952166022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 0x5604f12f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.857+0000 7feb1c636740 1 --2- 172.21.15.112:0/3952166022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 0x5604f12eec00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.857+0000 7feb1c636740 1 -- 172.21.15.112:0/3952166022 >> 172.21.15.112:0/3952166022 conn(0x5604f12ead30 msgr2=0x5604f12eb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.857+0000 7feb1c636740 1 -- 172.21.15.112:0/3952166022 shutdown_connections 2024-04-25T18:55:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.857+0000 7feb1c636740 1 -- 172.21.15.112:0/3952166022 wait complete. 2024-04-25T18:55:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.858+0000 7feb12a00640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7feaf80a3e40 0x7feaf80a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5604f12efdc0 tx=0x7feb04005f90 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.858+0000 7feb1c636740 1 -- start start 2024-04-25T18:55:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.858+0000 7feb1c636740 1 Processor -- start 2024-04-25T18:55:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.858+0000 7feb1c636740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 0x5604f12f6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.858+0000 7feb1c636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 0x5604f12f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb1c636740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12f9a00 0x5604f12f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb1c636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5604f12fc450 con 0x5604f12ef140 2024-04-25T18:55:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb1c636740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5604f120e310 con 0x5604f12ee820 2024-04-25T18:55:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb1c636740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5604f12fc5d0 con 0x5604f12f9a00 2024-04-25T18:55:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb1c636740 10 client.0 _ll_register_callbacks cb 0x5604f11dccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:55:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb12000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 0x5604f12f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb12a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 0x5604f12f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb12000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 0x5604f12f6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44238/0 (socket says 172.21.15.112:44238) 2024-04-25T18:55:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb12000640 1 -- 172.21.15.112:0/16127666 learned_addr learned my addr 172.21.15.112:0/16127666 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb13400640 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12f9a00 0x5604f12f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb12000640 1 -- 172.21.15.112:0/16127666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12f9a00 msgr2=0x5604f12f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb12000640 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12f9a00 0x5604f12f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb12000640 1 -- 172.21.15.112:0/16127666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 msgr2=0x5604f12f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb12000640 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 0x5604f12f6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb12000640 1 -- 172.21.15.112:0/16127666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f1033620 con 0x5604f12ef140 2024-04-25T18:55:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.859+0000 7feb12a00640 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 0x5604f12f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.860+0000 7feb12000640 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 0x5604f12f6eb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feb0000e9c0 tx=0x7feb000d40d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4c7891edd16836 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.860+0000 7feafcc00640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T18:55:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.860+0000 7feafcc00640 1 -- 172.21.15.112:0/16127666 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000ce5a0 con 0x5604f12ef140 2024-04-25T18:55:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.860+0000 7feafcc00640 1 -- 172.21.15.112:0/16127666 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7feb000bbb30 con 0x5604f12ef140 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.860+0000 7feb1c636740 1 -- 172.21.15.112:0/16127666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0c036030 con 0x5604f12ef140 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.860+0000 7feafcc00640 1 -- 172.21.15.112:0/16127666 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7feb000a0d20 con 0x5604f12ef140 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.860+0000 7feb1c636740 10 client.15783 fetch_fsmap learned FSMap version 36 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.860+0000 7feb1c636740 1 -- 172.21.15.112:0/16127666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5604f13086f0 con 0x5604f12ef140 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.861+0000 7feafcc00640 1 -- 172.21.15.112:0/16127666 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0c036030 con 0x5604f12ef140 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.861+0000 7feafcc00640 1 -- 172.21.15.112:0/16127666 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7feb00018cb0 con 0x5604f12ef140 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.861+0000 7feb1c636740 10 client.15783 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.861+0000 7feb1c636740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.861+0000 7feb1c636740 20 client.15783 populate_metadata read hostname 'smithi112' 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.861+0000 7feb1c636740 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5604f1309bb0 0x5604f130bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.861+0000 7feb1c636740 4 client.15783 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.861+0000 7feb1c636740 1 -- 172.21.15.112:0/16127666 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604f12688b0 con 0x5604f1309bb0 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.861+0000 7feb12a00640 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5604f1309bb0 0x5604f130bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.862+0000 7feb12a00640 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5604f1309bb0 0x5604f130bfa0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cbbffc4e23b1341 server_cookie=8e025b2ec1598426 in_seq=0 out_seq=0 2024-04-25T18:55:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.862+0000 7feafcc00640 10 client.15783 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.862+0000 7feafcc00640 1 -- 172.21.15.112:0/16127666 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604f12688b0 con 0x5604f1309bb0 2024-04-25T18:55:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.862+0000 7feafcc00640 10 client.15783 handle_command_reply: tid=0 2024-04-25T18:55:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 1 client.15783 shutdown 2024-04-25T18:55:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 1 -- 172.21.15.112:0/16127666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 msgr2=0x5604f12f6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 0x5604f12f6eb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feb0000e9c0 tx=0x7feb000d40d0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 1 -- 172.21.15.112:0/16127666 shutdown_connections 2024-04-25T18:55:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12f9a00 0x5604f12f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5604f1309bb0 0x5604f130bfa0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ef140 0x5604f12f6eb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 1 --2- 172.21.15.112:0/16127666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ee820 0x5604f12f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 1 -- 172.21.15.112:0/16127666 >> 172.21.15.112:0/16127666 conn(0x5604f12ead30 msgr2=0x5604f12f0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feafcc00640 0 client.15783 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 1 -- 172.21.15.112:0/16127666 shutdown_connections 2024-04-25T18:55:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 1 -- 172.21.15.112:0/16127666 wait complete. 2024-04-25T18:55:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.863+0000 7feb1c636740 20 client.15783 trim_cache size 0 max 16384 2024-04-25T18:55:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.883+0000 7feb1c636740 1 Processor -- start 2024-04-25T18:55:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.886+0000 7feb1c636740 1 -- start start 2024-04-25T18:55:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.887+0000 7feb1c636740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ee820 0x5604f1305310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.887+0000 7feb1c636740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ef140 0x5604f12f1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.887+0000 7feb1c636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12f9a00 0x5604f1311890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.887+0000 7feb1c636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb141b2700 con 0x5604f12f9a00 2024-04-25T18:55:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.887+0000 7feb1c636740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feb141affd0 con 0x5604f12ef140 2024-04-25T18:55:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.887+0000 7feb1c636740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7feb141ade40 con 0x5604f12ee820 2024-04-25T18:55:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.887+0000 7feb12a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ee820 0x5604f1305310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.887+0000 7feb12a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ee820 0x5604f1305310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:44058/0 (socket says 172.21.15.112:44058) 2024-04-25T18:55:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.887+0000 7feb12a00640 1 -- 172.21.15.112:0/3764481379 learned_addr learned my addr 172.21.15.112:0/3764481379 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.888+0000 7feb12000640 1 --2- 172.21.15.112:0/3764481379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ef140 0x5604f12f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.888+0000 7feb12000640 1 -- 172.21.15.112:0/3764481379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ee820 msgr2=0x5604f1305310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.888+0000 7feb12000640 1 --2- 172.21.15.112:0/3764481379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ee820 0x5604f1305310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.888+0000 7feb12000640 1 -- 172.21.15.112:0/3764481379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12f9a00 msgr2=0x5604f1311890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.888+0000 7feb12000640 1 --2- 172.21.15.112:0/3764481379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12f9a00 0x5604f1311890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.888+0000 7feb12000640 1 -- 172.21.15.112:0/3764481379 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f12f4ec0 con 0x5604f12ef140 2024-04-25T18:55:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.888+0000 7feb12000640 1 --2- 172.21.15.112:0/3764481379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ef140 0x5604f12f1230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feb0009c720 tx=0x7feb00000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe29fa8cf396a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.889+0000 7feaff400640 1 -- 172.21.15.112:0/3764481379 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000ce5a0 con 0x5604f12ef140 2024-04-25T18:55:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.889+0000 7feaff400640 1 -- 172.21.15.112:0/3764481379 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb000d0e50 con 0x5604f12ef140 2024-04-25T18:55:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.889+0000 7feaff400640 1 -- 172.21.15.112:0/3764481379 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0009c160 con 0x5604f12ef140 2024-04-25T18:55:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.889+0000 7feb1c636740 1 -- 172.21.15.112:0/3764481379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ef140 msgr2=0x5604f12f1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.889+0000 7feb1c636740 1 --2- 172.21.15.112:0/3764481379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ef140 0x5604f12f1230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feb0009c720 tx=0x7feb00000bc0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.890+0000 7feb1c636740 1 -- 172.21.15.112:0/3764481379 shutdown_connections 2024-04-25T18:55:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.890+0000 7feb1c636740 1 --2- 172.21.15.112:0/3764481379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12f9a00 0x5604f1311890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.890+0000 7feb1c636740 1 --2- 172.21.15.112:0/3764481379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12ef140 0x5604f12f1230 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.890+0000 7feb1c636740 1 --2- 172.21.15.112:0/3764481379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ee820 0x5604f1305310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.890+0000 7feb1c636740 1 -- 172.21.15.112:0/3764481379 >> 172.21.15.112:0/3764481379 conn(0x5604f12ead30 msgr2=0x5604f12f71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.890+0000 7feb1c636740 1 -- 172.21.15.112:0/3764481379 shutdown_connections 2024-04-25T18:55:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.890+0000 7feb1c636740 1 -- 172.21.15.112:0/3764481379 wait complete. 2024-04-25T18:55:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.890+0000 7feb1c636740 1 -- start start 2024-04-25T18:55:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.890+0000 7feb1c636740 1 Processor -- start 2024-04-25T18:55:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb1c636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ee820 0x5604f1301a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb1c636740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ef140 0x5604f12fed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb1c636740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12f9a00 0x5604f12ff2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb1c636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb141ae490 con 0x5604f12ee820 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb1c636740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feb141af900 con 0x5604f12f9a00 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb1c636740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7feb141aeb60 con 0x5604f12ef140 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb1c636740 10 client.0 _ll_register_callbacks cb 0x7feb0009c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb12000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ef140 0x5604f12fed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb12000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ef140 0x5604f12fed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:44062/0 (socket says 172.21.15.112:44062) 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb12a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ee820 0x5604f1301a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb12000640 1 -- 172.21.15.112:0/401177509 learned_addr learned my addr 172.21.15.112:0/401177509 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.891+0000 7feb13400640 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12f9a00 0x5604f12ff2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.892+0000 7feb12000640 1 -- 172.21.15.112:0/401177509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12f9a00 msgr2=0x5604f12ff2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.892+0000 7feb12000640 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12f9a00 0x5604f12ff2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.892+0000 7feb12000640 1 -- 172.21.15.112:0/401177509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ee820 msgr2=0x5604f1301a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.892+0000 7feb12000640 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ee820 0x5604f1301a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.892+0000 7feb12000640 1 -- 172.21.15.112:0/401177509 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f12ff820 con 0x5604f12ef140 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.892+0000 7feb12000640 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ef140 0x5604f12fed70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feb000a50b0 tx=0x7feb000d4040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93699314a2ca642d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.892+0000 7feafcc00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.892+0000 7feafcc00640 1 -- 172.21.15.112:0/401177509 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb000a55e0 con 0x5604f12ef140 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.892+0000 7feafcc00640 1 -- 172.21.15.112:0/401177509 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7feb0009be70 con 0x5604f12ef140 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.892+0000 7feb1c636740 1 -- 172.21.15.112:0/401177509 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb000d0e50 con 0x5604f12ef140 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.893+0000 7feafcc00640 1 -- 172.21.15.112:0/401177509 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7feb0009c070 con 0x5604f12ef140 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.893+0000 7feb1c636740 10 client.25963 fetch_fsmap learned FSMap version 36 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.893+0000 7feb1c636740 1 -- 172.21.15.112:0/401177509 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5604f13786b0 con 0x5604f12ef140 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.893+0000 7feafcc00640 1 -- 172.21.15.112:0/401177509 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb000d0e50 con 0x5604f12ef140 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.893+0000 7feafcc00640 1 -- 172.21.15.112:0/401177509 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7feb0009ba00 con 0x5604f12ef140 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.893+0000 7feb1c636740 10 client.25963 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:55:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.893+0000 7feb1c636740 10 client.25963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:55:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.893+0000 7feb1c636740 20 client.25963 populate_metadata read hostname 'smithi112' 2024-04-25T18:55:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.893+0000 7feb1c636740 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5604f1378bb0 0x5604f137afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.893+0000 7feb1c636740 4 client.25963 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:55:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.893+0000 7feb1c636740 1 -- 172.21.15.112:0/401177509 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604f1262320 con 0x5604f1378bb0 2024-04-25T18:55:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.894+0000 7feb12a00640 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5604f1378bb0 0x5604f137afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.894+0000 7feb12a00640 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5604f1378bb0 0x5604f137afd0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e268560ed649836 server_cookie=d15070e42b08b519 in_seq=0 out_seq=0 2024-04-25T18:55:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.894+0000 7feafcc00640 10 client.25963 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feafcc00640 1 -- 172.21.15.112:0/401177509 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604f1262320 con 0x5604f1378bb0 2024-04-25T18:55:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feafcc00640 10 client.25963 handle_command_reply: tid=0 2024-04-25T18:55:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feb1c636740 1 client.25963 shutdown 2024-04-25T18:55:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feb1c636740 1 -- 172.21.15.112:0/401177509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ef140 msgr2=0x5604f12fed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feb1c636740 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ef140 0x5604f12fed70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feb000a50b0 tx=0x7feb000d4040 comp rx=0 tx=0).stop 2024-04-25T18:55:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feb1c636740 1 -- 172.21.15.112:0/401177509 shutdown_connections 2024-04-25T18:55:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feb1c636740 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604f12f9a00 0x5604f12ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feb1c636740 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604f12ef140 0x5604f12fed70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feb1c636740 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x5604f1378bb0 0x5604f137afd0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feb1c636740 1 --2- 172.21.15.112:0/401177509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604f12ee820 0x5604f1301a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feb1c636740 1 -- 172.21.15.112:0/401177509 >> 172.21.15.112:0/401177509 conn(0x5604f12ead30 msgr2=0x5604f12e9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feafcc00640 0 client.25963 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feb1c636740 1 -- 172.21.15.112:0/401177509 shutdown_connections 2024-04-25T18:55:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.910+0000 7feb1c636740 1 -- 172.21.15.112:0/401177509 wait complete. 2024-04-25T18:55:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.911+0000 7feb1c636740 20 client.25963 trim_cache size 0 max 16384 2024-04-25T18:55:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:55:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:55:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:55:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:55:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.911+0000 7feb19800640 1 -- 172.21.15.112:0/3064066748 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7feaf80a3e40 msgr2=0x7feaf80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.911+0000 7feb19800640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7feaf80a3e40 0x7feaf80a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5604f12efdc0 tx=0x7feb04005f90 comp rx=0 tx=0).stop 2024-04-25T18:55:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.911+0000 7feb19800640 1 -- 172.21.15.112:0/3064066748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb141016f0 msgr2=0x7feb141a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.911+0000 7feb19800640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb141016f0 0x7feb141a0bb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feb0000e9f0 tx=0x7feb0000eeb0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.912+0000 7feb19800640 1 -- 172.21.15.112:0/3064066748 shutdown_connections 2024-04-25T18:55:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.912+0000 7feb19800640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7feaf80a3e40 0x7feaf80a6300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.912+0000 7feb19800640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb1410e8f0 0x7feb1419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.912+0000 7feb19800640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb141016f0 0x7feb141a0bb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.912+0000 7feb19800640 1 --2- 172.21.15.112:0/3064066748 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb14100d40 0x7feb141a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.912+0000 7feb19800640 1 -- 172.21.15.112:0/3064066748 >> 172.21.15.112:0/3064066748 conn(0x7feb14078060 msgr2=0x7feb1410fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.912+0000 7feb19800640 1 -- 172.21.15.112:0/3064066748 shutdown_connections 2024-04-25T18:55:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:18.912+0000 7feb19800640 1 -- 172.21.15.112:0/3064066748 wait complete. 2024-04-25T18:55:20.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:19 smithi171 ceph-mon[33266]: pgmap v247: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.2 MiB/s wr, 401 op/s 2024-04-25T18:55:20.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:19 smithi161 ceph-mon[32609]: pgmap v247: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.2 MiB/s wr, 401 op/s 2024-04-25T18:55:20.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:19 smithi112 ceph-mon[27864]: pgmap v247: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.2 MiB/s wr, 401 op/s 2024-04-25T18:55:22.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:21 smithi171 ceph-mon[33266]: pgmap v248: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-04-25T18:55:22.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:21 smithi161 ceph-mon[32609]: pgmap v248: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-04-25T18:55:22.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:21 smithi112 ceph-mon[27864]: pgmap v248: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-04-25T18:55:24.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:23 smithi171 ceph-mon[33266]: pgmap v249: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1016 KiB/s wr, 335 op/s 2024-04-25T18:55:24.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:23 smithi161 ceph-mon[32609]: pgmap v249: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1016 KiB/s wr, 335 op/s 2024-04-25T18:55:24.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:23 smithi112 ceph-mon[27864]: pgmap v249: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1016 KiB/s wr, 335 op/s 2024-04-25T18:55:26.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:25 smithi171 ceph-mon[33266]: pgmap v250: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1016 KiB/s wr, 335 op/s 2024-04-25T18:55:26.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:25 smithi161 ceph-mon[32609]: pgmap v250: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1016 KiB/s wr, 335 op/s 2024-04-25T18:55:26.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:25 smithi112 ceph-mon[27864]: pgmap v250: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1016 KiB/s wr, 335 op/s 2024-04-25T18:55:28.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:27 smithi171 ceph-mon[33266]: pgmap v251: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.5 MiB/s wr, 515 op/s 2024-04-25T18:55:28.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:27 smithi161 ceph-mon[32609]: pgmap v251: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.5 MiB/s wr, 515 op/s 2024-04-25T18:55:28.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:27 smithi112 ceph-mon[27864]: pgmap v251: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.5 MiB/s wr, 515 op/s 2024-04-25T18:55:30.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:29 smithi161 ceph-mon[32609]: pgmap v252: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.1 MiB/s wr, 358 op/s 2024-04-25T18:55:30.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:29 smithi112 ceph-mon[27864]: pgmap v252: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.1 MiB/s wr, 358 op/s 2024-04-25T18:55:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:29 smithi171 ceph-mon[33266]: pgmap v252: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.1 MiB/s wr, 358 op/s 2024-04-25T18:55:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.061+0000 7f4b79600640 1 -- 172.21.15.112:0/637952215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b74109470 msgr2=0x7f4b74111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.061+0000 7f4b79600640 1 --2- 172.21.15.112:0/637952215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b74109470 0x7f4b74111040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009ca0 tx=0x7f4b6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.061+0000 7f4b79600640 1 -- 172.21.15.112:0/637952215 shutdown_connections 2024-04-25T18:55:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.061+0000 7f4b79600640 1 --2- 172.21.15.112:0/637952215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b74109470 0x7f4b74111040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.061+0000 7f4b79600640 1 --2- 172.21.15.112:0/637952215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b74104d50 0x7f4b74108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.061+0000 7f4b79600640 1 --2- 172.21.15.112:0/637952215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b741043a0 0x7f4b74104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.061+0000 7f4b79600640 1 -- 172.21.15.112:0/637952215 >> 172.21.15.112:0/637952215 conn(0x7f4b740ffc40 msgr2=0x7f4b74102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.061+0000 7f4b79600640 1 -- 172.21.15.112:0/637952215 shutdown_connections 2024-04-25T18:55:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.061+0000 7f4b79600640 1 -- 172.21.15.112:0/637952215 wait complete. 2024-04-25T18:55:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.062+0000 7f4b79600640 1 Processor -- start 2024-04-25T18:55:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.063+0000 7f4b79600640 1 -- start start 2024-04-25T18:55:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.063+0000 7f4b79600640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741043a0 0x7f4b741a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.063+0000 7f4b79600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b74109470 0x7f4b741a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.063+0000 7f4b79600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7419a690 0x7f4b7419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.065+0000 7f4b79600640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b74114410 con 0x7f4b741043a0 2024-04-25T18:55:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.064+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741043a0 0x7f4b741a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.065+0000 7f4b79600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4b74114290 con 0x7f4b7419a690 2024-04-25T18:55:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.065+0000 7f4b79600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4b74114590 con 0x7f4b74109470 2024-04-25T18:55:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.065+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741043a0 0x7f4b741a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33806/0 (socket says 172.21.15.112:33806) 2024-04-25T18:55:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.065+0000 7f4b72a00640 1 -- 172.21.15.112:0/964004266 learned_addr learned my addr 172.21.15.112:0/964004266 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.064+0000 7f4b73400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7419a690 0x7f4b7419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.065+0000 7f4b73400640 1 -- 172.21.15.112:0/964004266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b74109470 msgr2=0x7f4b741a0ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T18:55:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.065+0000 7f4b73400640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b74109470 0x7f4b741a0ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.065+0000 7f4b73400640 1 -- 172.21.15.112:0/964004266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741043a0 msgr2=0x7f4b741a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.065+0000 7f4b73400640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741043a0 0x7f4b741a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.065+0000 7f4b73400640 1 -- 172.21.15.112:0/964004266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7419b290 con 0x7f4b7419a690 2024-04-25T18:55:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.066+0000 7f4b73400640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7419a690 0x7f4b7419ab10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c005e30 tx=0x7f4b6c0093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.067+0000 7f4b6b400640 1 -- 172.21.15.112:0/964004266 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c003b70 con 0x7f4b7419a690 2024-04-25T18:55:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.067+0000 7f4b6b400640 1 -- 172.21.15.112:0/964004266 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b6c003d10 con 0x7f4b7419a690 2024-04-25T18:55:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.067+0000 7f4b6b400640 1 -- 172.21.15.112:0/964004266 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c02fa30 con 0x7f4b7419a690 2024-04-25T18:55:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.068+0000 7f4b79600640 1 -- 172.21.15.112:0/964004266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7419b520 con 0x7f4b7419a690 2024-04-25T18:55:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.068+0000 7f4b79600640 1 -- 172.21.15.112:0/964004266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b740766c0 con 0x7f4b7419a690 2024-04-25T18:55:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.074+0000 7f4b7c435740 1 Processor -- start 2024-04-25T18:55:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.076+0000 7f4b6b400640 1 -- 172.21.15.112:0/964004266 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f4b6c02fbd0 con 0x7f4b7419a690 2024-04-25T18:55:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.076+0000 7f4b7c435740 1 -- start start 2024-04-25T18:55:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.076+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 0x558dc7d33c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.076+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 0x558dc7d3e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.076+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d3ea00 0x558dc7d40df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.076+0000 7f4b7c435740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558dc7c99900 con 0x558dc7d34140 2024-04-25T18:55:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.076+0000 7f4b7c435740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558dc7c3fef0 con 0x558dc7d3ea00 2024-04-25T18:55:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.076+0000 7f4b7c435740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558dc7c958c0 con 0x558dc7d33820 2024-04-25T18:55:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.077+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 0x558dc7d3e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.077+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 0x558dc7d3e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33814/0 (socket says 172.21.15.112:33814) 2024-04-25T18:55:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.077+0000 7f4b6be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 0x558dc7d33c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.077+0000 7f4b72a00640 1 -- 172.21.15.112:0/2145320424 learned_addr learned my addr 172.21.15.112:0/2145320424 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.077+0000 7f4b6be00640 1 --2- 172.21.15.112:0/2145320424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d3ea00 0x558dc7d40df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.077+0000 7f4b6b400640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4b580a3f20 0x7f4b580a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.078+0000 7f4b72a00640 1 -- 172.21.15.112:0/2145320424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 msgr2=0x558dc7d33c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.078+0000 7f4b72a00640 1 --2- 172.21.15.112:0/2145320424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 0x558dc7d33c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.078+0000 7f4b72a00640 1 -- 172.21.15.112:0/2145320424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d3ea00 msgr2=0x558dc7d40df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.078+0000 7f4b72a00640 1 --2- 172.21.15.112:0/2145320424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d3ea00 0x558dc7d40df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.078+0000 7f4b72a00640 1 -- 172.21.15.112:0/2145320424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc79ac210 con 0x558dc7d34140 2024-04-25T18:55:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.078+0000 7f4b73400640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4b580a3f20 0x7f4b580a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.078+0000 7f4b6b400640 1 -- 172.21.15.112:0/964004266 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f4b6c0fa680 con 0x7f4b7419a690 2024-04-25T18:55:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.078+0000 7f4b72a00640 1 --2- 172.21.15.112:0/2145320424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 0x558dc7d3e4c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009d70 tx=0x7f4b640117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26877519741490b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.079+0000 7f4b63400640 1 -- 172.21.15.112:0/2145320424 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64009e60 con 0x558dc7d34140 2024-04-25T18:55:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.079+0000 7f4b63400640 1 -- 172.21.15.112:0/2145320424 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b64012b00 con 0x558dc7d34140 2024-04-25T18:55:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.079+0000 7f4b63400640 1 -- 172.21.15.112:0/2145320424 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400fb90 con 0x558dc7d34140 2024-04-25T18:55:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.080+0000 7f4b7c435740 1 -- 172.21.15.112:0/2145320424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 msgr2=0x558dc7d3e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.080+0000 7f4b73400640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4b580a3f20 0x7f4b580a63e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009ca0 tx=0x7f4b6c02a8c0 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.080+0000 7f4b7c435740 1 --2- 172.21.15.112:0/2145320424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 0x558dc7d3e4c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009d70 tx=0x7f4b640117b0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.080+0000 7f4b7c435740 1 -- 172.21.15.112:0/2145320424 shutdown_connections 2024-04-25T18:55:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.080+0000 7f4b7c435740 1 --2- 172.21.15.112:0/2145320424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d3ea00 0x558dc7d40df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.080+0000 7f4b7c435740 1 --2- 172.21.15.112:0/2145320424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 0x558dc7d3e4c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.080+0000 7f4b7c435740 1 --2- 172.21.15.112:0/2145320424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 0x558dc7d33c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.080+0000 7f4b7c435740 1 -- 172.21.15.112:0/2145320424 >> 172.21.15.112:0/2145320424 conn(0x558dc7d2fd30 msgr2=0x558dc7d30110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.080+0000 7f4b7c435740 1 -- 172.21.15.112:0/2145320424 shutdown_connections 2024-04-25T18:55:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.080+0000 7f4b7c435740 1 -- 172.21.15.112:0/2145320424 wait complete. 2024-04-25T18:55:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.081+0000 7f4b7c435740 1 -- start start 2024-04-25T18:55:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.081+0000 7f4b7c435740 1 Processor -- start 2024-04-25T18:55:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d33820 0x558dc7d3b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d3bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d3ea00 0x558dc7d3c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b7c435740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558dc7d41630 con 0x558dc7d33820 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b7c435740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558dc7c53310 con 0x558dc7d34140 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b7c435740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558dc7d417b0 con 0x558dc7d3ea00 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b7c435740 10 client.0 _ll_register_callbacks cb 0x558dc7c21cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d3bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d3bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48118/0 (socket says 172.21.15.112:48118) 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b72a00640 1 -- 172.21.15.112:0/1930304423 learned_addr learned my addr 172.21.15.112:0/1930304423 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d33820 0x558dc7d3b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b72a00640 1 -- 172.21.15.112:0/1930304423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d3ea00 msgr2=0x558dc7d3c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.082+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d3ea00 0x558dc7d3c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b72a00640 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d3ea00 0x558dc7d3c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b72a00640 1 -- 172.21.15.112:0/1930304423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d33820 msgr2=0x558dc7d3b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b72a00640 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d33820 0x558dc7d3b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b72a00640 1 -- 172.21.15.112:0/1930304423 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc7a78620 con 0x558dc7d34140 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d3ea00 0x558dc7d3c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b72a00640 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d3bea0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4b64000c00 tx=0x7f4b64005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e09abf7f1cbd103c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b57e00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b57e00640 1 -- 172.21.15.112:0/1930304423 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64028020 con 0x558dc7d34140 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b57e00640 1 -- 172.21.15.112:0/1930304423 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f4b64012650 con 0x558dc7d34140 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b7c435740 1 -- 172.21.15.112:0/1930304423 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b64012b00 con 0x558dc7d34140 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b57e00640 1 -- 172.21.15.112:0/1930304423 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4b64033050 con 0x558dc7d34140 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b7c435740 10 client.25943 fetch_fsmap learned FSMap version 36 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.083+0000 7f4b7c435740 1 -- 172.21.15.112:0/1930304423 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558dc7d4d8f0 con 0x558dc7d34140 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.084+0000 7f4b57e00640 1 -- 172.21.15.112:0/1930304423 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b64012b00 con 0x558dc7d34140 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.084+0000 7f4b57e00640 1 -- 172.21.15.112:0/1930304423 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f4b640035f0 con 0x558dc7d34140 2024-04-25T18:55:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.084+0000 7f4b7c435740 10 client.25943 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:55:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.084+0000 7f4b7c435740 10 client.25943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:55:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.084+0000 7f4b7c435740 20 client.25943 populate_metadata read hostname 'smithi112' 2024-04-25T18:55:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.084+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558dc7d4ed00 0x558dc7d510f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.084+0000 7f4b7c435740 4 client.25943 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:55:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.084+0000 7f4b7c435740 1 -- 172.21.15.112:0/1930304423 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dc7cad8b0 con 0x558dc7d4ed00 2024-04-25T18:55:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.084+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558dc7d4ed00 0x558dc7d510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.085+0000 7f4b57e00640 10 client.25943 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.085+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558dc7d4ed00 0x558dc7d510f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec747961f3d78b3 server_cookie=6dc3f1d5ff6622b2 in_seq=0 out_seq=0 2024-04-25T18:55:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b57e00640 1 -- 172.21.15.112:0/1930304423 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558dc7cad8b0 con 0x558dc7d4ed00 2024-04-25T18:55:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b57e00640 10 client.25943 handle_command_reply: tid=0 2024-04-25T18:55:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b7c435740 1 client.25943 shutdown 2024-04-25T18:55:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b7c435740 1 -- 172.21.15.112:0/1930304423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 msgr2=0x558dc7d3bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d3bea0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4b64000c00 tx=0x7f4b64005780 comp rx=0 tx=0).stop 2024-04-25T18:55:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b7c435740 1 -- 172.21.15.112:0/1930304423 shutdown_connections 2024-04-25T18:55:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d3ea00 0x558dc7d3c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d3bea0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558dc7d4ed00 0x558dc7d510f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1930304423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d33820 0x558dc7d3b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b57e00640 0 client.25943 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b7c435740 1 -- 172.21.15.112:0/1930304423 >> 172.21.15.112:0/1930304423 conn(0x558dc7d2fd30 msgr2=0x558dc7d35a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.086+0000 7f4b7c435740 1 -- 172.21.15.112:0/1930304423 shutdown_connections 2024-04-25T18:55:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.087+0000 7f4b7c435740 1 -- 172.21.15.112:0/1930304423 wait complete. 2024-04-25T18:55:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.087+0000 7f4b7c435740 20 client.25943 trim_cache size 0 max 16384 2024-04-25T18:55:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.109+0000 7f4b7c435740 1 Processor -- start 2024-04-25T18:55:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.109+0000 7f4b7c435740 1 -- start start 2024-04-25T18:55:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.109+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d33820 0x558dc7d4a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.109+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 0x558dc7d3c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.109+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d3ea00 0x558dc7d567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b7c435740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b500110d0 con 0x558dc7d34140 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b7c435740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5000e9a0 con 0x558dc7d33820 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b7c435740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5000c810 con 0x558dc7d3ea00 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b6be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d33820 0x558dc7d4a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b6be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d33820 0x558dc7d4a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48122/0 (socket says 172.21.15.112:48122) 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b6be00640 1 -- 172.21.15.112:0/1027633090 learned_addr learned my addr 172.21.15.112:0/1027633090 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1027633090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d3ea00 0x558dc7d567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b72a00640 1 --2- 172.21.15.112:0/1027633090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 0x558dc7d3c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b6be00640 1 -- 172.21.15.112:0/1027633090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d3ea00 msgr2=0x558dc7d567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1027633090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d3ea00 0x558dc7d567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b6be00640 1 -- 172.21.15.112:0/1027633090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 msgr2=0x558dc7d3c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1027633090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 0x558dc7d3c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b6be00640 1 -- 172.21.15.112:0/1027633090 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc7d3a910 con 0x558dc7d33820 2024-04-25T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.110+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1027633090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d33820 0x558dc7d4a550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c004700 tx=0x7f4b5c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbf2e5a3bfbc6d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.111+0000 7f4b62a00640 1 -- 172.21.15.112:0/1027633090 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c024030 con 0x558dc7d33820 2024-04-25T18:55:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.111+0000 7f4b62a00640 1 -- 172.21.15.112:0/1027633090 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b5c01e930 con 0x558dc7d33820 2024-04-25T18:55:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.111+0000 7f4b62a00640 1 -- 172.21.15.112:0/1027633090 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c005360 con 0x558dc7d33820 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.112+0000 7f4b7c435740 1 -- 172.21.15.112:0/1027633090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d33820 msgr2=0x558dc7d4a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.112+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1027633090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d33820 0x558dc7d4a550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c004700 tx=0x7f4b5c005f90 comp rx=0 tx=0).stop 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.112+0000 7f4b7c435740 1 -- 172.21.15.112:0/1027633090 shutdown_connections 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.112+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1027633090 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d3ea00 0x558dc7d567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.112+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1027633090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d34140 0x558dc7d3c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.112+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1027633090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d33820 0x558dc7d4a550 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.112+0000 7f4b7c435740 1 -- 172.21.15.112:0/1027633090 >> 172.21.15.112:0/1027633090 conn(0x558dc7d2fd30 msgr2=0x558dc7d30110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.112+0000 7f4b7c435740 1 -- 172.21.15.112:0/1027633090 shutdown_connections 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.112+0000 7f4b7c435740 1 -- 172.21.15.112:0/1027633090 wait complete. 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.112+0000 7f4b7c435740 1 -- start start 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b7c435740 1 Processor -- start 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 0x558dc7d446b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d44bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b7c435740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d3ea00 0x558dc7d45130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b7c435740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5000ce60 con 0x558dc7d3ea00 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b7c435740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5000e2d0 con 0x558dc7d34140 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b7c435740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5000d530 con 0x558dc7d33820 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b7c435740 10 client.0 _ll_register_callbacks cb 0x7f4b64009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d44bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b6be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 0x558dc7d446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d44bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:48134/0 (socket says 172.21.15.112:48134) 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b6be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 0x558dc7d446b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:59634/0 (socket says 172.21.15.112:59634) 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b72a00640 1 -- 172.21.15.112:0/1719944603 learned_addr learned my addr 172.21.15.112:0/1719944603 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.113+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d3ea00 0x558dc7d45130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.114+0000 7f4b72a00640 1 -- 172.21.15.112:0/1719944603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 msgr2=0x558dc7d446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.114+0000 7f4b72a00640 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 0x558dc7d446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.114+0000 7f4b72a00640 1 -- 172.21.15.112:0/1719944603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d3ea00 msgr2=0x558dc7d45130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.114+0000 7f4b72a00640 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d3ea00 0x558dc7d45130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.114+0000 7f4b72a00640 1 -- 172.21.15.112:0/1719944603 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc7d33f70 con 0x558dc7d34140 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.114+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 0x558dc7d446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.114+0000 7f4b72a00640 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d44bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4b64006010 tx=0x7f4b640092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53db58b7fbcf29cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.114+0000 7f4b57e00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.114+0000 7f4b57e00640 1 -- 172.21.15.112:0/1719944603 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64028040 con 0x558dc7d34140 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b57e00640 1 -- 172.21.15.112:0/1719944603 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f4b64003c30 con 0x558dc7d34140 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b7c435740 1 -- 172.21.15.112:0/1719944603 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b5c01e930 con 0x558dc7d34140 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b57e00640 1 -- 172.21.15.112:0/1719944603 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4b64047050 con 0x558dc7d34140 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b7c435740 10 client.25949 fetch_fsmap learned FSMap version 36 2024-04-25T18:55:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b7c435740 1 -- 172.21.15.112:0/1719944603 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558dc7db5370 con 0x558dc7d34140 2024-04-25T18:55:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b57e00640 1 -- 172.21.15.112:0/1719944603 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b5c01e930 con 0x558dc7d34140 2024-04-25T18:55:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b57e00640 1 -- 172.21.15.112:0/1719944603 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f4b640124e0 con 0x558dc7d34140 2024-04-25T18:55:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b7c435740 10 client.25949 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:55:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b7c435740 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:55:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b7c435740 20 client.25949 populate_metadata read hostname 'smithi112' 2024-04-25T18:55:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558dc7db5a60 0x558dc7db7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b7c435740 4 client.25949 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:55:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.115+0000 7f4b7c435740 1 -- 172.21.15.112:0/1719944603 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558dc7ca7320 con 0x558dc7db5a60 2024-04-25T18:55:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.116+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558dc7db5a60 0x558dc7db7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.116+0000 7f4b6be00640 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558dc7db5a60 0x558dc7db7e80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc664d68cf44e3d server_cookie=d64dce45c30ede8c in_seq=0 out_seq=0 2024-04-25T18:55:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.116+0000 7f4b57e00640 10 client.25949 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.133+0000 7f4b57e00640 1 -- 172.21.15.112:0/1719944603 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558dc7ca7320 con 0x558dc7db5a60 2024-04-25T18:55:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.133+0000 7f4b57e00640 10 client.25949 handle_command_reply: tid=0 2024-04-25T18:55:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.133+0000 7f4b7c435740 1 client.25949 shutdown 2024-04-25T18:55:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.133+0000 7f4b7c435740 1 -- 172.21.15.112:0/1719944603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 msgr2=0x558dc7d44bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.133+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d44bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4b64006010 tx=0x7f4b640092d0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.133+0000 7f4b7c435740 1 -- 172.21.15.112:0/1719944603 shutdown_connections 2024-04-25T18:55:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.134+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x558dc7db5a60 0x558dc7db7e80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.134+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558dc7d3ea00 0x558dc7d45130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.134+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc7d34140 0x558dc7d44bf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.134+0000 7f4b57e00640 0 client.25949 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.134+0000 7f4b7c435740 1 --2- 172.21.15.112:0/1719944603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558dc7d33820 0x558dc7d446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.134+0000 7f4b7c435740 1 -- 172.21.15.112:0/1719944603 >> 172.21.15.112:0/1719944603 conn(0x558dc7d2fd30 msgr2=0x558dc7d3c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.134+0000 7f4b7c435740 1 -- 172.21.15.112:0/1719944603 shutdown_connections 2024-04-25T18:55:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.134+0000 7f4b7c435740 1 -- 172.21.15.112:0/1719944603 wait complete. 2024-04-25T18:55:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.134+0000 7f4b7c435740 20 client.25949 trim_cache size 0 max 16384 2024-04-25T18:55:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:55:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:55:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:55:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:55:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.135+0000 7f4b79600640 1 -- 172.21.15.112:0/964004266 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4b580a3f20 msgr2=0x7f4b580a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.135+0000 7f4b79600640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4b580a3f20 0x7f4b580a63e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009ca0 tx=0x7f4b6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.135+0000 7f4b79600640 1 -- 172.21.15.112:0/964004266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7419a690 msgr2=0x7f4b7419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.135+0000 7f4b79600640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7419a690 0x7f4b7419ab10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c005e30 tx=0x7f4b6c0093d0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.135+0000 7f4b79600640 1 -- 172.21.15.112:0/964004266 shutdown_connections 2024-04-25T18:55:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.135+0000 7f4b79600640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f4b580a3f20 0x7f4b580a63e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.135+0000 7f4b79600640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7419a690 0x7f4b7419ab10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.135+0000 7f4b79600640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b74109470 0x7f4b741a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.135+0000 7f4b79600640 1 --2- 172.21.15.112:0/964004266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b741043a0 0x7f4b741a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.135+0000 7f4b79600640 1 -- 172.21.15.112:0/964004266 >> 172.21.15.112:0/964004266 conn(0x7f4b740ffc40 msgr2=0x7f4b74101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.135+0000 7f4b79600640 1 -- 172.21.15.112:0/964004266 shutdown_connections 2024-04-25T18:55:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:32.136+0000 7f4b79600640 1 -- 172.21.15.112:0/964004266 wait complete. 2024-04-25T18:55:32.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:31 smithi161 ceph-mon[32609]: pgmap v253: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.6 MiB/s wr, 517 op/s 2024-04-25T18:55:32.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:31 smithi112 ceph-mon[27864]: pgmap v253: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.6 MiB/s wr, 517 op/s 2024-04-25T18:55:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:31 smithi171 ceph-mon[33266]: pgmap v253: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.6 MiB/s wr, 517 op/s 2024-04-25T18:55:34.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:33 smithi161 ceph-mon[32609]: pgmap v254: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.0 MiB/s wr, 339 op/s 2024-04-25T18:55:34.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:33 smithi112 ceph-mon[27864]: pgmap v254: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.0 MiB/s wr, 339 op/s 2024-04-25T18:55:34.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:33 smithi171 ceph-mon[33266]: pgmap v254: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.0 MiB/s wr, 339 op/s 2024-04-25T18:55:36.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:35 smithi161 ceph-mon[32609]: pgmap v255: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.0 MiB/s wr, 339 op/s 2024-04-25T18:55:36.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:35 smithi112 ceph-mon[27864]: pgmap v255: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.0 MiB/s wr, 339 op/s 2024-04-25T18:55:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:35 smithi171 ceph-mon[33266]: pgmap v255: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.0 MiB/s wr, 339 op/s 2024-04-25T18:55:38.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:37 smithi161 ceph-mon[32609]: pgmap v256: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-04-25T18:55:38.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:37 smithi112 ceph-mon[27864]: pgmap v256: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-04-25T18:55:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:37 smithi171 ceph-mon[33266]: pgmap v256: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.5 MiB/s wr, 514 op/s 2024-04-25T18:55:40.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:39 smithi171 ceph-mon[33266]: pgmap v257: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.0 MiB/s wr, 335 op/s 2024-04-25T18:55:40.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:39 smithi161 ceph-mon[32609]: pgmap v257: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.0 MiB/s wr, 335 op/s 2024-04-25T18:55:40.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:39 smithi112 ceph-mon[27864]: pgmap v257: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.0 MiB/s wr, 335 op/s 2024-04-25T18:55:42.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:41 smithi171 ceph-mon[33266]: pgmap v258: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 506 op/s 2024-04-25T18:55:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:41 smithi161 ceph-mon[32609]: pgmap v258: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 506 op/s 2024-04-25T18:55:42.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:41 smithi112 ceph-mon[27864]: pgmap v258: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 506 op/s 2024-04-25T18:55:44.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:43 smithi171 ceph-mon[33266]: pgmap v259: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1011 KiB/s wr, 346 op/s 2024-04-25T18:55:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:43 smithi161 ceph-mon[32609]: pgmap v259: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1011 KiB/s wr, 346 op/s 2024-04-25T18:55:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:43 smithi112 ceph-mon[27864]: pgmap v259: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1011 KiB/s wr, 346 op/s 2024-04-25T18:55:45.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:44 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:45.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:44 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:45.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:44 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.296+0000 7f6c7ba00640 1 -- 172.21.15.112:0/2379346294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c74104d60 msgr2=0x7f6c74108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.296+0000 7f6c7ba00640 1 --2- 172.21.15.112:0/2379346294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c74104d60 0x7f6c74108da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009900 tx=0x7f6c6002a410 comp rx=0 tx=0).stop 2024-04-25T18:55:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.296+0000 7f6c7ba00640 1 -- 172.21.15.112:0/2379346294 shutdown_connections 2024-04-25T18:55:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.296+0000 7f6c7ba00640 1 --2- 172.21.15.112:0/2379346294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c74109480 0x7f6c74111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.296+0000 7f6c7ba00640 1 --2- 172.21.15.112:0/2379346294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c74104d60 0x7f6c74108da0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.296+0000 7f6c7ba00640 1 --2- 172.21.15.112:0/2379346294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c741043b0 0x7f6c74104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.296+0000 7f6c7ba00640 1 -- 172.21.15.112:0/2379346294 >> 172.21.15.112:0/2379346294 conn(0x7f6c740ffc50 msgr2=0x7f6c74102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.297+0000 7f6c7ba00640 1 -- 172.21.15.112:0/2379346294 shutdown_connections 2024-04-25T18:55:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.297+0000 7f6c7ba00640 1 -- 172.21.15.112:0/2379346294 wait complete. 2024-04-25T18:55:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.298+0000 7f6c7ba00640 1 Processor -- start 2024-04-25T18:55:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.298+0000 7f6c7ba00640 1 -- start start 2024-04-25T18:55:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.298+0000 7f6c7ba00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c741043b0 0x7f6c741a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.299+0000 7f6c7ba00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c74109480 0x7f6c741a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.299+0000 7f6c7ba00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c7419a770 0x7f6c7419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.299+0000 7f6c7ba00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74114410 con 0x7f6c741043b0 2024-04-25T18:55:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.299+0000 7f6c7ba00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74114290 con 0x7f6c7419a770 2024-04-25T18:55:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.299+0000 7f6c7ba00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74114590 con 0x7f6c74109480 2024-04-25T18:55:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.299+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c74109480 0x7f6c741a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.299+0000 7f6c79a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c7419a770 0x7f6c7419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.299+0000 7f6c79000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c741043b0 0x7f6c741a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.299+0000 7f6c79000640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c741043b0 0x7f6c741a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41664/0 (socket says 172.21.15.112:41664) 2024-04-25T18:55:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.299+0000 7f6c79000640 1 -- 172.21.15.112:0/1674542716 learned_addr learned my addr 172.21.15.112:0/1674542716 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.300+0000 7f6c79000640 1 -- 172.21.15.112:0/1674542716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c74109480 msgr2=0x7f6c741a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.299+0000 7f6c79a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c7419a770 0x7f6c7419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:51662/0 (socket says 172.21.15.112:51662) 2024-04-25T18:55:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.301+0000 7f6c79000640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c74109480 0x7f6c741a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.301+0000 7f6c79000640 1 -- 172.21.15.112:0/1674542716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c7419a770 msgr2=0x7f6c7419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.301+0000 7f6c79000640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c7419a770 0x7f6c7419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.301+0000 7f6c79000640 1 -- 172.21.15.112:0/1674542716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7419b370 con 0x7f6c741043b0 2024-04-25T18:55:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.301+0000 7f6c73e00640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c74109480 0x7f6c741a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:55:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.301+0000 7f6c79a00640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c7419a770 0x7f6c7419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:55:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.301+0000 7f6c79000640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c741043b0 0x7f6c741a0660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800e9f0 tx=0x7f6c6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.301+0000 7f6c71600640 1 -- 172.21.15.112:0/1674542716 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800cd80 con 0x7f6c741043b0 2024-04-25T18:55:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.302+0000 7f6c7ba00640 1 -- 172.21.15.112:0/1674542716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c741a7400 con 0x7f6c741043b0 2024-04-25T18:55:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.302+0000 7f6c7ba00640 1 -- 172.21.15.112:0/1674542716 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c741a7800 con 0x7f6c741043b0 2024-04-25T18:55:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.302+0000 7f6c71600640 1 -- 172.21.15.112:0/1674542716 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c680054d0 con 0x7f6c741043b0 2024-04-25T18:55:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.302+0000 7f6c71600640 1 -- 172.21.15.112:0/1674542716 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68010690 con 0x7f6c741043b0 2024-04-25T18:55:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.306+0000 7f6c7e999740 1 Processor -- start 2024-04-25T18:55:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.307+0000 7f6c71600640 1 -- 172.21.15.112:0/1674542716 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f6c68010830 con 0x7f6c741043b0 2024-04-25T18:55:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.307+0000 7f6c7e999740 1 -- start start 2024-04-25T18:55:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.307+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef4999760 0x561ef4999b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.307+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 0x561ef49accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.307+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef49ad210 0x561ef49af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.307+0000 7f6c7e999740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561ef48ff900 con 0x561ef49ad210 2024-04-25T18:55:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.307+0000 7f6c7e999740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561ef48a5ef0 con 0x561ef499a080 2024-04-25T18:55:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.307+0000 7f6c7e999740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561ef48fb8c0 con 0x561ef4999760 2024-04-25T18:55:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.307+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef49ad210 0x561ef49af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.307+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef49ad210 0x561ef49af600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41670/0 (socket says 172.21.15.112:41670) 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.307+0000 7f6c73e00640 1 -- 172.21.15.112:0/3038522122 learned_addr learned my addr 172.21.15.112:0/3038522122 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.308+0000 7f6c79a00640 1 --2- 172.21.15.112:0/3038522122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef4999760 0x561ef4999b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.308+0000 7f6c79000640 1 --2- 172.21.15.112:0/3038522122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 0x561ef49accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.308+0000 7f6c79a00640 1 -- 172.21.15.112:0/3038522122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 msgr2=0x561ef49accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.308+0000 7f6c79a00640 1 --2- 172.21.15.112:0/3038522122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 0x561ef49accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.308+0000 7f6c79a00640 1 -- 172.21.15.112:0/3038522122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef49ad210 msgr2=0x561ef49af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.308+0000 7f6c79a00640 1 --2- 172.21.15.112:0/3038522122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef49ad210 0x561ef49af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.308+0000 7f6c79a00640 1 -- 172.21.15.112:0/3038522122 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef4612210 con 0x561ef4999760 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.308+0000 7f6c79000640 1 --2- 172.21.15.112:0/3038522122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 0x561ef49accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.308+0000 7f6c79a00640 1 --2- 172.21.15.112:0/3038522122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef4999760 0x561ef4999b40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005fd0 tx=0x7f6c6c00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604e19e3cdd4914a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.309+0000 7f6c66000640 1 -- 172.21.15.112:0/3038522122 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c016070 con 0x561ef4999760 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.309+0000 7f6c66000640 1 -- 172.21.15.112:0/3038522122 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c6c005020 con 0x561ef4999760 2024-04-25T18:55:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.309+0000 7f6c66000640 1 -- 172.21.15.112:0/3038522122 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0116b0 con 0x561ef4999760 2024-04-25T18:55:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.309+0000 7f6c71600640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c5c0a3dc0 0x7f6c5c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.309+0000 7f6c71600640 1 -- 172.21.15.112:0/1674542716 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f6c680d0cf0 con 0x7f6c741043b0 2024-04-25T18:55:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.310+0000 7f6c7e999740 1 -- 172.21.15.112:0/3038522122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef4999760 msgr2=0x561ef4999b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.310+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3038522122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef4999760 0x561ef4999b40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005fd0 tx=0x7f6c6c00e6c0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.310+0000 7f6c79000640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c5c0a3dc0 0x7f6c5c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.310+0000 7f6c7e999740 1 -- 172.21.15.112:0/3038522122 shutdown_connections 2024-04-25T18:55:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.310+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3038522122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef49ad210 0x561ef49af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.310+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3038522122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 0x561ef49accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.310+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3038522122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef4999760 0x561ef4999b40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.310+0000 7f6c7e999740 1 -- 172.21.15.112:0/3038522122 >> 172.21.15.112:0/3038522122 conn(0x561ef4995d30 msgr2=0x561ef4996110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.311+0000 7f6c7e999740 1 -- 172.21.15.112:0/3038522122 shutdown_connections 2024-04-25T18:55:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.311+0000 7f6c79000640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c5c0a3dc0 0x7f6c5c0a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6c68018000 tx=0x7f6c6800cf20 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.311+0000 7f6c7e999740 1 -- 172.21.15.112:0/3038522122 wait complete. 2024-04-25T18:55:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.312+0000 7f6c7e999740 1 -- start start 2024-04-25T18:55:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.312+0000 7f6c7e999740 1 Processor -- start 2024-04-25T18:55:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef4999760 0x561ef49aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 0x561ef49aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef49aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c7e999740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561ef49afde0 con 0x561ef4999760 2024-04-25T18:55:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c7e999740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561ef48b9310 con 0x561ef499a080 2024-04-25T18:55:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c7e999740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561ef49aff60 con 0x561ef49ad210 2024-04-25T18:55:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c7e999740 10 client.0 _ll_register_callbacks cb 0x561ef4887cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c79a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef4999760 0x561ef49aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c79a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef49aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c79a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef49aabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:33436/0 (socket says 172.21.15.112:33436) 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c79a00640 1 -- 172.21.15.112:0/3032400562 learned_addr learned my addr 172.21.15.112:0/3032400562 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c73e00640 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 0x561ef49aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c79a00640 1 -- 172.21.15.112:0/3032400562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 msgr2=0x561ef49aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.313+0000 7f6c79a00640 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 0x561ef49aa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c79a00640 1 -- 172.21.15.112:0/3032400562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef4999760 msgr2=0x561ef49aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c79a00640 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef4999760 0x561ef49aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c79a00640 1 -- 172.21.15.112:0/3032400562 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef46de620 con 0x561ef49ad210 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c73e00640 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 0x561ef49aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c79a00640 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef49aabd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c004f70 tx=0x7f6c6c009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d1d09eb4679c6af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c5aa00640 1 -- 172.21.15.112:0/3032400562 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c016030 con 0x561ef49ad210 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c5aa00640 1 -- 172.21.15.112:0/3032400562 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f6c6c003970 con 0x561ef49ad210 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c7e999740 1 -- 172.21.15.112:0/3032400562 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6c005020 con 0x561ef49ad210 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c5aa00640 1 -- 172.21.15.112:0/3032400562 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6c6c003b70 con 0x561ef49ad210 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c7e999740 10 client.25984 fetch_fsmap learned FSMap version 36 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.314+0000 7f6c7e999740 1 -- 172.21.15.112:0/3032400562 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561ef49bc0d0 con 0x561ef49ad210 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.315+0000 7f6c5aa00640 1 -- 172.21.15.112:0/3032400562 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c6c005020 con 0x561ef49ad210 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.315+0000 7f6c5aa00640 1 -- 172.21.15.112:0/3032400562 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f6c6c00f040 con 0x561ef49ad210 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.315+0000 7f6c7e999740 10 client.25984 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.315+0000 7f6c7e999740 10 client.25984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.315+0000 7f6c7e999740 20 client.25984 populate_metadata read hostname 'smithi112' 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.315+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561ef49bd530 0x561ef49bf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.315+0000 7f6c7e999740 4 client.25984 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.315+0000 7f6c7e999740 1 -- 172.21.15.112:0/3032400562 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ef49138b0 con 0x561ef49bd530 2024-04-25T18:55:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.315+0000 7f6c73e00640 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561ef49bd530 0x561ef49bf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.316+0000 7f6c5aa00640 10 client.25984 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.316+0000 7f6c73e00640 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561ef49bd530 0x561ef49bf920 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f6407aea30368 server_cookie=a81920e7f31cf4ae in_seq=0 out_seq=0 2024-04-25T18:55:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c5aa00640 1 -- 172.21.15.112:0/3032400562 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ef49138b0 con 0x561ef49bd530 2024-04-25T18:55:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c5aa00640 10 client.25984 handle_command_reply: tid=0 2024-04-25T18:55:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c7e999740 1 client.25984 shutdown 2024-04-25T18:55:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c7e999740 1 -- 172.21.15.112:0/3032400562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 msgr2=0x561ef49aabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef49aabd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c004f70 tx=0x7f6c6c009330 comp rx=0 tx=0).stop 2024-04-25T18:55:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c7e999740 1 -- 172.21.15.112:0/3032400562 shutdown_connections 2024-04-25T18:55:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef49aabd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef499a080 0x561ef49aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561ef49bd530 0x561ef49bf920 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3032400562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef4999760 0x561ef49aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c7e999740 1 -- 172.21.15.112:0/3032400562 >> 172.21.15.112:0/3032400562 conn(0x561ef4995d30 msgr2=0x561ef499ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.317+0000 7f6c5aa00640 0 client.25984 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.318+0000 7f6c7e999740 1 -- 172.21.15.112:0/3032400562 shutdown_connections 2024-04-25T18:55:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.318+0000 7f6c7e999740 1 -- 172.21.15.112:0/3032400562 wait complete. 2024-04-25T18:55:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.318+0000 7f6c7e999740 20 client.25984 trim_cache size 0 max 16384 2024-04-25T18:55:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.340+0000 7f6c7e999740 1 Processor -- start 2024-04-25T18:55:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.340+0000 7f6c7e999740 1 -- start start 2024-04-25T18:55:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.340+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef4999760 0x561ef49b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.340+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef499a080 0x561ef49c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.340+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef49ad210 0x561ef49c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.340+0000 7f6c7e999740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54011110 con 0x561ef4999760 2024-04-25T18:55:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.340+0000 7f6c7e999740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5400e9e0 con 0x561ef49ad210 2024-04-25T18:55:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.340+0000 7f6c7e999740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5400c850 con 0x561ef499a080 2024-04-25T18:55:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.340+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef499a080 0x561ef49c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.341+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef499a080 0x561ef49c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:33444/0 (socket says 172.21.15.112:33444) 2024-04-25T18:55:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.341+0000 7f6c79a00640 1 --2- 172.21.15.112:0/3022784823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef4999760 0x561ef49b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.341+0000 7f6c73e00640 1 -- 172.21.15.112:0/3022784823 learned_addr learned my addr 172.21.15.112:0/3022784823 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.341+0000 7f6c79a00640 1 --2- 172.21.15.112:0/3022784823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef49ad210 0x561ef49c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.341+0000 7f6c73e00640 1 -- 172.21.15.112:0/3022784823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef49ad210 msgr2=0x561ef49c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.341+0000 7f6c73e00640 1 --2- 172.21.15.112:0/3022784823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef49ad210 0x561ef49c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.341+0000 7f6c73e00640 1 -- 172.21.15.112:0/3022784823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef4999760 msgr2=0x561ef49b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.341+0000 7f6c73e00640 1 --2- 172.21.15.112:0/3022784823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef4999760 0x561ef49b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.341+0000 7f6c73e00640 1 -- 172.21.15.112:0/3022784823 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef499aa90 con 0x561ef499a080 2024-04-25T18:55:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.341+0000 7f6c79a00640 1 --2- 172.21.15.112:0/3022784823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef49ad210 0x561ef49c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:55:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.341+0000 7f6c73e00640 1 --2- 172.21.15.112:0/3022784823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef499a080 0x561ef49c5040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c60006010 tx=0x7f6c60005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67a062a706d9abd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.342+0000 7f6c65600640 1 -- 172.21.15.112:0/3022784823 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60047020 con 0x561ef499a080 2024-04-25T18:55:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.342+0000 7f6c65600640 1 -- 172.21.15.112:0/3022784823 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c60005040 con 0x561ef499a080 2024-04-25T18:55:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.342+0000 7f6c65600640 1 -- 172.21.15.112:0/3022784823 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60035040 con 0x561ef499a080 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.342+0000 7f6c7e999740 1 -- 172.21.15.112:0/3022784823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef499a080 msgr2=0x561ef49c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.342+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3022784823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef499a080 0x561ef49c5040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c60006010 tx=0x7f6c60005b70 comp rx=0 tx=0).stop 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.342+0000 7f6c7e999740 1 -- 172.21.15.112:0/3022784823 shutdown_connections 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.342+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3022784823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef49ad210 0x561ef49c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.342+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3022784823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef499a080 0x561ef49c5040 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.342+0000 7f6c7e999740 1 --2- 172.21.15.112:0/3022784823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef4999760 0x561ef49b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.342+0000 7f6c7e999740 1 -- 172.21.15.112:0/3022784823 >> 172.21.15.112:0/3022784823 conn(0x561ef4995d30 msgr2=0x561ef49a53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.343+0000 7f6c7e999740 1 -- 172.21.15.112:0/3022784823 shutdown_connections 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.343+0000 7f6c7e999740 1 -- 172.21.15.112:0/3022784823 wait complete. 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.343+0000 7f6c7e999740 1 -- start start 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.343+0000 7f6c7e999740 1 Processor -- start 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef4999760 0x561ef49b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef499a080 0x561ef49b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c7e999740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef4a218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c7e999740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5400cea0 con 0x561ef499a080 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c7e999740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5400e310 con 0x561ef4999760 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c7e999740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5400d570 con 0x561ef49ad210 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c7e999740 10 client.0 _ll_register_callbacks cb 0x7f6c6c004e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef499a080 0x561ef49b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c79a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef4a218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c79a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef4a218b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:33454/0 (socket says 172.21.15.112:33454) 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c79a00640 1 -- 172.21.15.112:0/2189330236 learned_addr learned my addr 172.21.15.112:0/2189330236 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef499a080 0x561ef49b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41696/0 (socket says 172.21.15.112:41696) 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.344+0000 7f6c79a00640 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef4999760 0x561ef49b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.345+0000 7f6c79a00640 1 -- 172.21.15.112:0/2189330236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef4999760 msgr2=0x561ef49b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.345+0000 7f6c79a00640 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef4999760 0x561ef49b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.345+0000 7f6c79a00640 1 -- 172.21.15.112:0/2189330236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef499a080 msgr2=0x561ef49b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.345+0000 7f6c79a00640 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef499a080 0x561ef49b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.345+0000 7f6c79a00640 1 -- 172.21.15.112:0/2189330236 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef4a21df0 con 0x561ef49ad210 2024-04-25T18:55:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.345+0000 7f6c73e00640 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef499a080 0x561ef49b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.345+0000 7f6c79a00640 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef4a218b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c004e50 tx=0x7f6c6c01c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d31c3466750ec12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.345+0000 7f6c5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:55:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.345+0000 7f6c5aa00640 1 -- 172.21.15.112:0/2189330236 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c016020 con 0x561ef49ad210 2024-04-25T18:55:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.345+0000 7f6c5aa00640 1 -- 172.21.15.112:0/2189330236 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f6c6c023040 con 0x561ef49ad210 2024-04-25T18:55:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.345+0000 7f6c7e999740 1 -- 172.21.15.112:0/2189330236 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c60005040 con 0x561ef49ad210 2024-04-25T18:55:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.346+0000 7f6c5aa00640 1 -- 172.21.15.112:0/2189330236 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6c6c045050 con 0x561ef49ad210 2024-04-25T18:55:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.346+0000 7f6c7e999740 10 client.25990 fetch_fsmap learned FSMap version 36 2024-04-25T18:55:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.346+0000 7f6c7e999740 1 -- 172.21.15.112:0/2189330236 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561ef4a22f10 con 0x561ef49ad210 2024-04-25T18:55:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.346+0000 7f6c5aa00640 1 -- 172.21.15.112:0/2189330236 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c60005040 con 0x561ef49ad210 2024-04-25T18:55:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.346+0000 7f6c5aa00640 1 -- 172.21.15.112:0/2189330236 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f6c6c03b040 con 0x561ef49ad210 2024-04-25T18:55:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.347+0000 7f6c7e999740 10 client.25990 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:55:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.347+0000 7f6c7e999740 10 client.25990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:55:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.347+0000 7f6c7e999740 20 client.25990 populate_metadata read hostname 'smithi112' 2024-04-25T18:55:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.347+0000 7f6c7e999740 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561ef4a23580 0x561ef4a25970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.347+0000 7f6c7e999740 4 client.25990 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:55:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.347+0000 7f6c7e999740 1 -- 172.21.15.112:0/2189330236 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ef490d320 con 0x561ef4a23580 2024-04-25T18:55:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.347+0000 7f6c73e00640 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561ef4a23580 0x561ef4a25970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.348+0000 7f6c73e00640 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561ef4a23580 0x561ef4a25970 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9fb0d9f3a8d5a91 server_cookie=272387a87eb11ba7 in_seq=0 out_seq=0 2024-04-25T18:55:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.348+0000 7f6c5aa00640 10 client.25990 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c5aa00640 1 -- 172.21.15.112:0/2189330236 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ef490d320 con 0x561ef4a23580 2024-04-25T18:55:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c5aa00640 10 client.25990 handle_command_reply: tid=0 2024-04-25T18:55:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 1 client.25990 shutdown 2024-04-25T18:55:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 1 -- 172.21.15.112:0/2189330236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 msgr2=0x561ef4a218b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef4a218b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c004e50 tx=0x7f6c6c01c040 comp rx=0 tx=0).stop 2024-04-25T18:55:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 1 -- 172.21.15.112:0/2189330236 shutdown_connections 2024-04-25T18:55:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561ef49ad210 0x561ef4a218b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x561ef4a23580 0x561ef4a25970 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ef499a080 0x561ef49b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 1 --2- 172.21.15.112:0/2189330236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561ef4999760 0x561ef49b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c5aa00640 0 client.25990 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 1 -- 172.21.15.112:0/2189330236 >> 172.21.15.112:0/2189330236 conn(0x561ef4995d30 msgr2=0x561ef4994600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 1 -- 172.21.15.112:0/2189330236 shutdown_connections 2024-04-25T18:55:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 1 -- 172.21.15.112:0/2189330236 wait complete. 2024-04-25T18:55:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.363+0000 7f6c7e999740 20 client.25990 trim_cache size 0 max 16384 2024-04-25T18:55:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:55:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:55:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:55:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:55:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.364+0000 7f6c7ba00640 1 -- 172.21.15.112:0/1674542716 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c5c0a3dc0 msgr2=0x7f6c5c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.364+0000 7f6c7ba00640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c5c0a3dc0 0x7f6c5c0a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6c68018000 tx=0x7f6c6800cf20 comp rx=0 tx=0).stop 2024-04-25T18:55:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.364+0000 7f6c7ba00640 1 -- 172.21.15.112:0/1674542716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c741043b0 msgr2=0x7f6c741a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.364+0000 7f6c7ba00640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c741043b0 0x7f6c741a0660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800e9f0 tx=0x7f6c6800edf0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.365+0000 7f6c7ba00640 1 -- 172.21.15.112:0/1674542716 shutdown_connections 2024-04-25T18:55:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.365+0000 7f6c7ba00640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c5c0a3dc0 0x7f6c5c0a6280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.365+0000 7f6c7ba00640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c7419a770 0x7f6c7419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.365+0000 7f6c7ba00640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c74109480 0x7f6c741a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.365+0000 7f6c7ba00640 1 --2- 172.21.15.112:0/1674542716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c741043b0 0x7f6c741a0660 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.365+0000 7f6c7ba00640 1 -- 172.21.15.112:0/1674542716 >> 172.21.15.112:0/1674542716 conn(0x7f6c740ffc50 msgr2=0x7f6c74101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.365+0000 7f6c7ba00640 1 -- 172.21.15.112:0/1674542716 shutdown_connections 2024-04-25T18:55:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:45.365+0000 7f6c7ba00640 1 -- 172.21.15.112:0/1674542716 wait complete. 2024-04-25T18:55:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:46 smithi171 ceph-mon[33266]: pgmap v260: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1011 KiB/s wr, 346 op/s 2024-04-25T18:55:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:46 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:55:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:46 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:55:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:46 smithi161 ceph-mon[32609]: pgmap v260: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1011 KiB/s wr, 346 op/s 2024-04-25T18:55:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:46 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:55:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:46 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:55:46.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:46 smithi112 ceph-mon[27864]: pgmap v260: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1011 KiB/s wr, 346 op/s 2024-04-25T18:55:46.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:46 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:55:46.476 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:46 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:55:47.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:47 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:47.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:47 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:47.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:47 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:55:47.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:47 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:47.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:47 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:47.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:47 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:55:47.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:47 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:47.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:47 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:47.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:47 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:55:48.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:48 smithi171 ceph-mon[33266]: pgmap v261: 97 pgs: 97 active+clean; 117 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 524 op/s 2024-04-25T18:55:48.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:48 smithi161 ceph-mon[32609]: pgmap v261: 97 pgs: 97 active+clean; 117 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 524 op/s 2024-04-25T18:55:48.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:48 smithi112 ceph-mon[27864]: pgmap v261: 97 pgs: 97 active+clean; 117 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 524 op/s 2024-04-25T18:55:50.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:50 smithi171 ceph-mon[33266]: pgmap v262: 97 pgs: 97 active+clean; 117 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.0 MiB/s wr, 348 op/s 2024-04-25T18:55:50.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:50 smithi161 ceph-mon[32609]: pgmap v262: 97 pgs: 97 active+clean; 117 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.0 MiB/s wr, 348 op/s 2024-04-25T18:55:50.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:50 smithi112 ceph-mon[27864]: pgmap v262: 97 pgs: 97 active+clean; 117 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.0 MiB/s wr, 348 op/s 2024-04-25T18:55:52.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:52 smithi171 ceph-mon[33266]: pgmap v263: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 516 op/s 2024-04-25T18:55:52.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:52 smithi161 ceph-mon[32609]: pgmap v263: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 516 op/s 2024-04-25T18:55:52.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:52 smithi112 ceph-mon[27864]: pgmap v263: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 516 op/s 2024-04-25T18:55:54.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:54 smithi171 ceph-mon[33266]: pgmap v264: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.0 MiB/s wr, 345 op/s 2024-04-25T18:55:54.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:54 smithi161 ceph-mon[32609]: pgmap v264: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.0 MiB/s wr, 345 op/s 2024-04-25T18:55:54.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:54 smithi112 ceph-mon[27864]: pgmap v264: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.0 MiB/s wr, 345 op/s 2024-04-25T18:55:56.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:56 smithi171 ceph-mon[33266]: pgmap v265: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.0 MiB/s wr, 345 op/s 2024-04-25T18:55:56.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:56 smithi161 ceph-mon[32609]: pgmap v265: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.0 MiB/s wr, 345 op/s 2024-04-25T18:55:56.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:56 smithi112 ceph-mon[27864]: pgmap v265: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.0 MiB/s wr, 345 op/s 2024-04-25T18:55:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.527+0000 7fccacc00640 1 -- 172.21.15.112:0/2780433152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca8109480 msgr2=0x7fcca8111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.527+0000 7fccacc00640 1 --2- 172.21.15.112:0/2780433152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca8109480 0x7fcca8111050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009ca0 tx=0x7fcca002a8c0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.529+0000 7fccacc00640 1 -- 172.21.15.112:0/2780433152 shutdown_connections 2024-04-25T18:55:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.529+0000 7fccacc00640 1 --2- 172.21.15.112:0/2780433152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca8109480 0x7fcca8111050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.529+0000 7fccacc00640 1 --2- 172.21.15.112:0/2780433152 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcca8104d60 0x7fcca8108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.529+0000 7fccacc00640 1 --2- 172.21.15.112:0/2780433152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca81043b0 0x7fcca8104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.529+0000 7fccacc00640 1 -- 172.21.15.112:0/2780433152 >> 172.21.15.112:0/2780433152 conn(0x7fcca80ffc50 msgr2=0x7fcca8102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.530+0000 7fccacc00640 1 -- 172.21.15.112:0/2780433152 shutdown_connections 2024-04-25T18:55:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.530+0000 7fccacc00640 1 -- 172.21.15.112:0/2780433152 wait complete. 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.532+0000 7fccacc00640 1 Processor -- start 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.532+0000 7fccacc00640 1 -- start start 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fccacc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcca81043b0 0x7fcca8076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fccacc00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca8109480 0x7fcca80769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fccacc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca8076ee0 0x7fcca81aae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fccacc00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcca8114420 con 0x7fcca8109480 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fccacc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81142a0 con 0x7fcca8076ee0 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fccacc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81145a0 con 0x7fcca81043b0 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fcca5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcca81043b0 0x7fcca8076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fcca5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcca81043b0 0x7fcca8076460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:50646/0 (socket says 172.21.15.112:50646) 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fcca5e00640 1 -- 172.21.15.112:0/793440532 learned_addr learned my addr 172.21.15.112:0/793440532 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fcca6800640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca8076ee0 0x7fcca81aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fcca5400640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca8109480 0x7fcca80769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fcca5e00640 1 -- 172.21.15.112:0/793440532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca8076ee0 msgr2=0x7fcca81aae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fcca5e00640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca8076ee0 0x7fcca81aae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fcca5e00640 1 -- 172.21.15.112:0/793440532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca8109480 msgr2=0x7fcca80769a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.533+0000 7fcca5e00640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca8109480 0x7fcca80769a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.534+0000 7fcca5e00640 1 -- 172.21.15.112:0/793440532 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca81ab370 con 0x7fcca81043b0 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.534+0000 7fcca5e00640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcca81043b0 0x7fcca8076460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800e9f0 tx=0x7fcc9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.534+0000 7fcc9ea00640 1 -- 172.21.15.112:0/793440532 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9800cd80 con 0x7fcca81043b0 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.534+0000 7fcc9ea00640 1 -- 172.21.15.112:0/793440532 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc980108e0 con 0x7fcca81043b0 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.534+0000 7fcc9ea00640 1 -- 172.21.15.112:0/793440532 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98018980 con 0x7fcca81043b0 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.535+0000 7fccacc00640 1 -- 172.21.15.112:0/793440532 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca81ab660 con 0x7fcca81043b0 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.535+0000 7fccacc00640 1 -- 172.21.15.112:0/793440532 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca81abb30 con 0x7fcca81043b0 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.539+0000 7fccafa54740 1 Processor -- start 2024-04-25T18:55:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.539+0000 7fccafa54740 1 -- start start 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.539+0000 7fcc9ea00640 1 -- 172.21.15.112:0/793440532 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fcc98010430 con 0x7fcca81043b0 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.539+0000 7fccafa54740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94c820 0x55b77c94cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.539+0000 7fccafa54740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94d140 0x55b77c9574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.539+0000 7fccafa54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 0x55b77c959df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.539+0000 7fccafa54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b77c8b2900 con 0x55b77c957a00 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fccafa54740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b77c858ef0 con 0x55b77c94d140 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fccafa54740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b77c8ae8c0 con 0x55b77c94c820 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fcca6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94c820 0x55b77c94cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fcca6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94c820 0x55b77c94cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:50654/0 (socket says 172.21.15.112:50654) 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fcca6800640 1 -- 172.21.15.112:0/2271527914 learned_addr learned my addr 172.21.15.112:0/2271527914 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fcca5e00640 1 --2- 172.21.15.112:0/2271527914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94d140 0x55b77c9574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fcca5400640 1 --2- 172.21.15.112:0/2271527914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 0x55b77c959df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fcca5e00640 1 -- 172.21.15.112:0/2271527914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94c820 msgr2=0x55b77c94cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fcca5e00640 1 --2- 172.21.15.112:0/2271527914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94c820 0x55b77c94cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fcca5e00640 1 -- 172.21.15.112:0/2271527914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 msgr2=0x55b77c959df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fcca5e00640 1 --2- 172.21.15.112:0/2271527914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 0x55b77c959df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fcca5e00640 1 -- 172.21.15.112:0/2271527914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77c5c5210 con 0x55b77c94d140 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.540+0000 7fcca5400640 1 --2- 172.21.15.112:0/2271527914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 0x55b77c959df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.541+0000 7fcca5e00640 1 --2- 172.21.15.112:0/2271527914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94d140 0x55b77c9574c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcc980ce270 tx=0x7fcc980d95f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79e2fd26e329e78b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.541+0000 7fcc93000640 1 -- 172.21.15.112:0/2271527914 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc980dd640 con 0x55b77c94d140 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.541+0000 7fcc93000640 1 -- 172.21.15.112:0/2271527914 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc980e2070 con 0x55b77c94d140 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.541+0000 7fcc93000640 1 -- 172.21.15.112:0/2271527914 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc980dc820 con 0x55b77c94d140 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.542+0000 7fcc9ea00640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcc8c0a3ec0 0x7fcc8c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.542+0000 7fcc9ea00640 1 -- 172.21.15.112:0/793440532 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fcc980d0bb0 con 0x7fcca81043b0 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.542+0000 7fcca6800640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcc8c0a3ec0 0x7fcc8c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.543+0000 7fccafa54740 1 -- 172.21.15.112:0/2271527914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94d140 msgr2=0x55b77c9574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.543+0000 7fccafa54740 1 --2- 172.21.15.112:0/2271527914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94d140 0x55b77c9574c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcc980ce270 tx=0x7fcc980d95f0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.543+0000 7fccafa54740 1 -- 172.21.15.112:0/2271527914 shutdown_connections 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.543+0000 7fccafa54740 1 --2- 172.21.15.112:0/2271527914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 0x55b77c959df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.543+0000 7fccafa54740 1 --2- 172.21.15.112:0/2271527914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94d140 0x55b77c9574c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.543+0000 7fccafa54740 1 --2- 172.21.15.112:0/2271527914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94c820 0x55b77c94cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.543+0000 7fccafa54740 1 -- 172.21.15.112:0/2271527914 >> 172.21.15.112:0/2271527914 conn(0x55b77c948d30 msgr2=0x55b77c949110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.543+0000 7fccafa54740 1 -- 172.21.15.112:0/2271527914 shutdown_connections 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.544+0000 7fccafa54740 1 -- 172.21.15.112:0/2271527914 wait complete. 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.544+0000 7fcca6800640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcc8c0a3ec0 0x7fcc8c0a6380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009a60 tx=0x7fcca0005d80 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.544+0000 7fccafa54740 1 -- start start 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.544+0000 7fccafa54740 1 Processor -- start 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fccafa54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c94c820 0x55b77c9549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fccafa54740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c954f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fccafa54740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c955450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fccafa54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b77c95a5d0 con 0x55b77c94c820 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fccafa54740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b77c86c310 con 0x55b77c957a00 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fccafa54740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b77c95a750 con 0x55b77c94d140 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fccafa54740 10 client.0 _ll_register_callbacks cb 0x55b77c83acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fcca5e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c94c820 0x55b77c9549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fcca6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c955450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fcca5e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c94c820 0x55b77c9549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35408/0 (socket says 172.21.15.112:35408) 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fcca6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c955450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:34792/0 (socket says 172.21.15.112:34792) 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fcca5e00640 1 -- 172.21.15.112:0/2193274846 learned_addr learned my addr 172.21.15.112:0/2193274846 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fcca5400640 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c954f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fcca6800640 1 -- 172.21.15.112:0/2193274846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 msgr2=0x55b77c954f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fcca6800640 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c954f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fcca6800640 1 -- 172.21.15.112:0/2193274846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c94c820 msgr2=0x55b77c9549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fcca6800640 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c94c820 0x55b77c9549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.545+0000 7fcca6800640 1 -- 172.21.15.112:0/2193274846 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77c691620 con 0x55b77c957a00 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.546+0000 7fcca5400640 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c954f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.546+0000 7fcca6800640 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c955450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcca002ade0 tx=0x7fcca002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55374699eb1c9486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.546+0000 7fcc87400640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.546+0000 7fcc87400640 1 -- 172.21.15.112:0/2193274846 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca002f610 con 0x55b77c957a00 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.546+0000 7fcc87400640 1 -- 172.21.15.112:0/2193274846 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fcca002f7b0 con 0x55b77c957a00 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.546+0000 7fccafa54740 1 -- 172.21.15.112:0/2193274846 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc980e2070 con 0x55b77c957a00 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.546+0000 7fcc87400640 1 -- 172.21.15.112:0/2193274846 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcca002f9b0 con 0x55b77c957a00 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.547+0000 7fccafa54740 10 client.25973 fetch_fsmap learned FSMap version 36 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.547+0000 7fccafa54740 1 -- 172.21.15.112:0/2193274846 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b77c966880 con 0x55b77c957a00 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.547+0000 7fcc87400640 1 -- 172.21.15.112:0/2193274846 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc980e2070 con 0x55b77c957a00 2024-04-25T18:55:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.547+0000 7fcc87400640 1 -- 172.21.15.112:0/2193274846 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fcca0051020 con 0x55b77c957a00 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.547+0000 7fccafa54740 10 client.25973 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.547+0000 7fccafa54740 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.547+0000 7fccafa54740 20 client.25973 populate_metadata read hostname 'smithi112' 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.547+0000 7fccafa54740 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55b77c967ce0 0x55b77c96a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.547+0000 7fccafa54740 4 client.25973 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.547+0000 7fccafa54740 1 -- 172.21.15.112:0/2193274846 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b77c8c68b0 con 0x55b77c967ce0 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.547+0000 7fcca5e00640 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55b77c967ce0 0x55b77c96a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.548+0000 7fcc87400640 10 client.25973 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.548+0000 7fcca5e00640 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55b77c967ce0 0x55b77c96a180 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3fbc05b3a37e74b server_cookie=f401300ba3f1923f in_seq=0 out_seq=0 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fcc87400640 1 -- 172.21.15.112:0/2193274846 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b77c8c68b0 con 0x55b77c967ce0 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fcc87400640 10 client.25973 handle_command_reply: tid=0 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fccafa54740 1 client.25973 shutdown 2024-04-25T18:55:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fccafa54740 1 -- 172.21.15.112:0/2193274846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 msgr2=0x55b77c955450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fccafa54740 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c955450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcca002ade0 tx=0x7fcca002a8c0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fccafa54740 1 -- 172.21.15.112:0/2193274846 shutdown_connections 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fccafa54740 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c955450 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fccafa54740 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c954f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fccafa54740 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55b77c967ce0 0x55b77c96a180 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fccafa54740 1 --2- 172.21.15.112:0/2193274846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c94c820 0x55b77c9549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fcc87400640 0 client.25973 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fccafa54740 1 -- 172.21.15.112:0/2193274846 >> 172.21.15.112:0/2193274846 conn(0x55b77c948d30 msgr2=0x55b77c94ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.549+0000 7fccafa54740 1 -- 172.21.15.112:0/2193274846 shutdown_connections 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.550+0000 7fccafa54740 1 -- 172.21.15.112:0/2193274846 wait complete. 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.550+0000 7fccafa54740 20 client.25973 trim_cache size 0 max 16384 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.566+0000 7fccafa54740 1 Processor -- start 2024-04-25T18:55:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.566+0000 7fccafa54740 1 -- start start 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fccafa54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c94c820 0x55b77c963490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fccafa54740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c96fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fccafa54740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c96ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fccafa54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b6c20 con 0x55b77c94c820 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fccafa54740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b44f0 con 0x55b77c957a00 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fccafa54740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b2360 con 0x55b77c94d140 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fcca6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c96ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fcca5400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c96fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fcca5400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c96fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:50660/0 (socket says 172.21.15.112:50660) 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fcca5e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c94c820 0x55b77c963490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fcca5400640 1 -- 172.21.15.112:0/778319672 learned_addr learned my addr 172.21.15.112:0/778319672 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fcca6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c96ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:34794/0 (socket says 172.21.15.112:34794) 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fcca6800640 1 -- 172.21.15.112:0/778319672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 msgr2=0x55b77c96fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fcca6800640 1 --2- 172.21.15.112:0/778319672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c96fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fcca6800640 1 -- 172.21.15.112:0/778319672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c94c820 msgr2=0x55b77c963490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fcca6800640 1 --2- 172.21.15.112:0/778319672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c94c820 0x55b77c963490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.567+0000 7fcca6800640 1 -- 172.21.15.112:0/778319672 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77c94cf70 con 0x55b77c957a00 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.568+0000 7fcca6800640 1 --2- 172.21.15.112:0/778319672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c96ffd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcca0004af0 tx=0x7fcca0005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa0a9897b36262e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.568+0000 7fcc92600640 1 -- 172.21.15.112:0/778319672 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca005f020 con 0x55b77c957a00 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.568+0000 7fcc92600640 1 -- 172.21.15.112:0/778319672 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca0036050 con 0x55b77c957a00 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.568+0000 7fcc92600640 1 -- 172.21.15.112:0/778319672 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0030960 con 0x55b77c957a00 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.569+0000 7fccafa54740 1 -- 172.21.15.112:0/778319672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 msgr2=0x55b77c96ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.569+0000 7fccafa54740 1 --2- 172.21.15.112:0/778319672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c96ffd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcca0004af0 tx=0x7fcca0005df0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.569+0000 7fccafa54740 1 -- 172.21.15.112:0/778319672 shutdown_connections 2024-04-25T18:55:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.569+0000 7fccafa54740 1 --2- 172.21.15.112:0/778319672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c957a00 0x55b77c96ffd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.569+0000 7fccafa54740 1 --2- 172.21.15.112:0/778319672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c96fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.569+0000 7fccafa54740 1 --2- 172.21.15.112:0/778319672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c94c820 0x55b77c963490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.569+0000 7fccafa54740 1 -- 172.21.15.112:0/778319672 >> 172.21.15.112:0/778319672 conn(0x55b77c948d30 msgr2=0x55b77c949110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.569+0000 7fccafa54740 1 -- 172.21.15.112:0/778319672 shutdown_connections 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.569+0000 7fccafa54740 1 -- 172.21.15.112:0/778319672 wait complete. 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.569+0000 7fccafa54740 1 -- start start 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fccafa54740 1 Processor -- start 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fccafa54740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94c820 0x55b77c95fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fccafa54740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c95d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fccafa54740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 0x55b77c95d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fccafa54740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b29b0 con 0x55b77c957a00 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fccafa54740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b3e20 con 0x55b77c94c820 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fccafa54740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b3080 con 0x55b77c94d140 2024-04-25T18:55:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fccafa54740 10 client.0 _ll_register_callbacks cb 0x7fcca0031f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fcca5e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94c820 0x55b77c95fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fcca6800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 0x55b77c95d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fcca5e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94c820 0x55b77c95fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:34798/0 (socket says 172.21.15.112:34798) 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fcca5e00640 1 -- 172.21.15.112:0/2717590350 learned_addr learned my addr 172.21.15.112:0/2717590350 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.570+0000 7fcca5400640 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c95d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fcca5e00640 1 -- 172.21.15.112:0/2717590350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 msgr2=0x55b77c95d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fcca5e00640 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c95d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fcca5e00640 1 -- 172.21.15.112:0/2717590350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 msgr2=0x55b77c95d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fcca5e00640 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 0x55b77c95d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fcca5e00640 1 -- 172.21.15.112:0/2717590350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77c95df80 con 0x55b77c94c820 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fcca6800640 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 0x55b77c95d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:55:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fcca5e00640 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94c820 0x55b77c95fe20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800c6a0 tx=0x7fcc980dc3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4acda1a4d95b3be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fcc87400640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fcc87400640 1 -- 172.21.15.112:0/2717590350 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98098d60 con 0x55b77c94c820 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fcc87400640 1 -- 172.21.15.112:0/2717590350 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fcc9809adc0 con 0x55b77c94c820 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fccafa54740 1 -- 172.21.15.112:0/2717590350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca0036050 con 0x55b77c94c820 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fcc87400640 1 -- 172.21.15.112:0/2717590350 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcc9809c420 con 0x55b77c94c820 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fccafa54740 10 client.25979 fetch_fsmap learned FSMap version 36 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.571+0000 7fccafa54740 1 -- 172.21.15.112:0/2717590350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b77c9ce3f0 con 0x55b77c94c820 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.572+0000 7fcc87400640 1 -- 172.21.15.112:0/2717590350 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca0036050 con 0x55b77c94c820 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.572+0000 7fcc87400640 1 -- 172.21.15.112:0/2717590350 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fcc98099130 con 0x55b77c94c820 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.572+0000 7fccafa54740 10 client.25979 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.572+0000 7fccafa54740 10 client.25979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.572+0000 7fccafa54740 20 client.25979 populate_metadata read hostname 'smithi112' 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.572+0000 7fccafa54740 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55b77c9ce6e0 0x55b77c9d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.572+0000 7fccafa54740 4 client.25979 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.572+0000 7fccafa54740 1 -- 172.21.15.112:0/2717590350 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b77c8c0320 con 0x55b77c9ce6e0 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.572+0000 7fcca5400640 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55b77c9ce6e0 0x55b77c9d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.573+0000 7fcca5400640 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55b77c9ce6e0 0x55b77c9d0ad0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf58a98b4e66ca00 server_cookie=55d331a66175ec43 in_seq=0 out_seq=0 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.573+0000 7fcc87400640 10 client.25979 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.586+0000 7fcc87400640 1 -- 172.21.15.112:0/2717590350 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b77c8c0320 con 0x55b77c9ce6e0 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.586+0000 7fcc87400640 10 client.25979 handle_command_reply: tid=0 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.586+0000 7fccafa54740 1 client.25979 shutdown 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fccafa54740 1 -- 172.21.15.112:0/2717590350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94c820 msgr2=0x55b77c95fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fccafa54740 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94c820 0x55b77c95fe20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800c6a0 tx=0x7fcc980dc3a0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fccafa54740 1 -- 172.21.15.112:0/2717590350 shutdown_connections 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fccafa54740 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55b77c9ce6e0 0x55b77c9d0ad0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fccafa54740 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b77c957a00 0x55b77c95d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fccafa54740 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b77c94d140 0x55b77c95d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fcc87400640 0 client.25979 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:55:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fccafa54740 1 --2- 172.21.15.112:0/2717590350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b77c94c820 0x55b77c95fe20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fccafa54740 1 -- 172.21.15.112:0/2717590350 >> 172.21.15.112:0/2717590350 conn(0x55b77c948d30 msgr2=0x55b77c94e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fccafa54740 1 -- 172.21.15.112:0/2717590350 shutdown_connections 2024-04-25T18:55:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fccafa54740 1 -- 172.21.15.112:0/2717590350 wait complete. 2024-04-25T18:55:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.587+0000 7fccafa54740 20 client.25979 trim_cache size 0 max 16384 2024-04-25T18:55:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:55:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:55:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:55:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:55:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 -- 172.21.15.112:0/793440532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcc8c0a3ec0 msgr2=0x7fcc8c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcc8c0a3ec0 0x7fcc8c0a6380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009a60 tx=0x7fcca0005d80 comp rx=0 tx=0).stop 2024-04-25T18:55:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 -- 172.21.15.112:0/793440532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcca81043b0 msgr2=0x7fcca8076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcca81043b0 0x7fcca8076460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800e9f0 tx=0x7fcc9800eeb0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 -- 172.21.15.112:0/793440532 shutdown_connections 2024-04-25T18:55:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fcc8c0a3ec0 0x7fcc8c0a6380 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca8076ee0 0x7fcca81aae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcca8109480 0x7fcca80769a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 --2- 172.21.15.112:0/793440532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcca81043b0 0x7fcca8076460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 -- 172.21.15.112:0/793440532 >> 172.21.15.112:0/793440532 conn(0x7fcca80ffc50 msgr2=0x7fcca81007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 -- 172.21.15.112:0/793440532 shutdown_connections 2024-04-25T18:55:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:55:58.588+0000 7fccacc00640 1 -- 172.21.15.112:0/793440532 wait complete. 2024-04-25T18:55:58.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:55:58 smithi171 ceph-mon[33266]: pgmap v266: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.5 MiB/s wr, 505 op/s 2024-04-25T18:55:58.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:55:58 smithi161 ceph-mon[32609]: pgmap v266: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.5 MiB/s wr, 505 op/s 2024-04-25T18:55:58.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:55:58 smithi112 ceph-mon[27864]: pgmap v266: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.5 MiB/s wr, 505 op/s 2024-04-25T18:56:00.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:00 smithi171 ceph-mon[33266]: pgmap v267: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1006 KiB/s wr, 327 op/s 2024-04-25T18:56:00.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:00 smithi161 ceph-mon[32609]: pgmap v267: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1006 KiB/s wr, 327 op/s 2024-04-25T18:56:00.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:00 smithi112 ceph-mon[27864]: pgmap v267: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1006 KiB/s wr, 327 op/s 2024-04-25T18:56:02.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:02 smithi171 ceph-mon[33266]: pgmap v268: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.6 MiB/s wr, 502 op/s 2024-04-25T18:56:02.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:02 smithi161 ceph-mon[32609]: pgmap v268: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.6 MiB/s wr, 502 op/s 2024-04-25T18:56:02.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:02 smithi112 ceph-mon[27864]: pgmap v268: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.6 MiB/s wr, 502 op/s 2024-04-25T18:56:03.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:03 smithi171 ceph-mon[33266]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T18:56:03.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:03 smithi161 ceph-mon[32609]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T18:56:03.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:03 smithi112 ceph-mon[27864]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T18:56:04.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:04 smithi171 ceph-mon[33266]: pgmap v270: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2024-04-25T18:56:04.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:04 smithi161 ceph-mon[32609]: pgmap v270: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2024-04-25T18:56:04.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:04 smithi112 ceph-mon[27864]: pgmap v270: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2024-04-25T18:56:06.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:06 smithi171 ceph-mon[33266]: pgmap v271: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2024-04-25T18:56:06.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:06 smithi161 ceph-mon[32609]: pgmap v271: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2024-04-25T18:56:06.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:06 smithi112 ceph-mon[27864]: pgmap v271: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2024-04-25T18:56:08.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:08 smithi171 ceph-mon[33266]: pgmap v272: 97 pgs: 97 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.4 MiB/s wr, 410 op/s 2024-04-25T18:56:08.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:08 smithi161 ceph-mon[32609]: pgmap v272: 97 pgs: 97 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.4 MiB/s wr, 410 op/s 2024-04-25T18:56:08.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:08 smithi112 ceph-mon[27864]: pgmap v272: 97 pgs: 97 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.4 MiB/s wr, 410 op/s 2024-04-25T18:56:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:09 smithi171 ceph-mon[33266]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T18:56:10.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:09 smithi171 ceph-mon[33266]: pgmap v274: 97 pgs: 97 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 820 KiB/s wr, 252 op/s 2024-04-25T18:56:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:09 smithi161 ceph-mon[32609]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T18:56:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:09 smithi161 ceph-mon[32609]: pgmap v274: 97 pgs: 97 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 820 KiB/s wr, 252 op/s 2024-04-25T18:56:10.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:09 smithi112 ceph-mon[27864]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T18:56:10.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:09 smithi112 ceph-mon[27864]: pgmap v274: 97 pgs: 97 active+clean; 139 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 820 KiB/s wr, 252 op/s 2024-04-25T18:56:11.721 INFO:tasks.workunit.client.0.smithi112.stdout:All 50000 operations completed A-OK! 2024-04-25T18:56:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.743+0000 7fb424200640 1 -- 172.21.15.112:0/3704825111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb41c06a780 msgr2=0x7fb41c06abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.743+0000 7fb424200640 1 --2- 172.21.15.112:0/3704825111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb41c06a780 0x7fb41c06abe0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb408009900 tx=0x7fb40802a480 comp rx=0 tx=0).stop 2024-04-25T18:56:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.743+0000 7fb424200640 1 -- 172.21.15.112:0/3704825111 shutdown_connections 2024-04-25T18:56:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.743+0000 7fb424200640 1 --2- 172.21.15.112:0/3704825111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41c06b120 0x7fb41c10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.743+0000 7fb424200640 1 --2- 172.21.15.112:0/3704825111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb41c06a780 0x7fb41c06abe0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.743+0000 7fb424200640 1 --2- 172.21.15.112:0/3704825111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb41c1080e0 0x7fb41c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.743+0000 7fb424200640 1 -- 172.21.15.112:0/3704825111 >> 172.21.15.112:0/3704825111 conn(0x7fb41c076c50 msgr2=0x7fb41c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.744+0000 7fb424200640 1 -- 172.21.15.112:0/3704825111 shutdown_connections 2024-04-25T18:56:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.744+0000 7fb424200640 1 -- 172.21.15.112:0/3704825111 wait complete. 2024-04-25T18:56:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.745+0000 7fb424200640 1 Processor -- start 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.745+0000 7fb424200640 1 -- start start 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.745+0000 7fb424200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb41c06a780 0x7fb41c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.745+0000 7fb424200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41c06b120 0x7fb41c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.745+0000 7fb424200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb41c1080e0 0x7fb41c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.745+0000 7fb424200640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c112180 con 0x7fb41c06a780 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.745+0000 7fb424200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c112000 con 0x7fb41c06b120 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.745+0000 7fb424200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c112300 con 0x7fb41c1080e0 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.745+0000 7fb421800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb41c06a780 0x7fb41c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.745+0000 7fb421800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb41c06a780 0x7fb41c198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33046/0 (socket says 172.21.15.112:33046) 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.746+0000 7fb421800640 1 -- 172.21.15.112:0/1397976326 learned_addr learned my addr 172.21.15.112:0/1397976326 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.746+0000 7fb422200640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb41c1080e0 0x7fb41c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.746+0000 7fb420e00640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41c06b120 0x7fb41c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.746+0000 7fb421800640 1 -- 172.21.15.112:0/1397976326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb41c1080e0 msgr2=0x7fb41c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.746+0000 7fb421800640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb41c1080e0 0x7fb41c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.746+0000 7fb421800640 1 -- 172.21.15.112:0/1397976326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41c06b120 msgr2=0x7fb41c198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.746+0000 7fb421800640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41c06b120 0x7fb41c198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.746+0000 7fb421800640 1 -- 172.21.15.112:0/1397976326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c19d5f0 con 0x7fb41c06a780 2024-04-25T18:56:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.746+0000 7fb421800640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb41c06a780 0x7fb41c198600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb41000e9f0 tx=0x7fb41000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.747+0000 7fb41a000640 1 -- 172.21.15.112:0/1397976326 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41000cd80 con 0x7fb41c06a780 2024-04-25T18:56:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.747+0000 7fb41a000640 1 -- 172.21.15.112:0/1397976326 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4100108e0 con 0x7fb41c06a780 2024-04-25T18:56:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.747+0000 7fb424200640 1 -- 172.21.15.112:0/1397976326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c19d8e0 con 0x7fb41c06a780 2024-04-25T18:56:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.747+0000 7fb41a000640 1 -- 172.21.15.112:0/1397976326 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410018980 con 0x7fb41c06a780 2024-04-25T18:56:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.747+0000 7fb424200640 1 -- 172.21.15.112:0/1397976326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c1a5170 con 0x7fb41c06a780 2024-04-25T18:56:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.750+0000 7fb41a000640 1 -- 172.21.15.112:0/1397976326 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fb410010430 con 0x7fb41c06a780 2024-04-25T18:56:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.750+0000 7fb426ffe740 1 Processor -- start 2024-04-25T18:56:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb426ffe740 1 -- start start 2024-04-25T18:56:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87041c30 0x55be87042010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.754 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T18:56:11.754 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T18:56:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be8704c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8704ce10 0x55be8704f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb426ffe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be86fa3900 con 0x55be87041c30 2024-04-25T18:56:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb426ffe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55be86f49ef0 con 0x55be8704ce10 2024-04-25T18:56:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb426ffe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55be86f9f8c0 con 0x55be87042550 2024-04-25T18:56:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb421800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be8704c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb422200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87041c30 0x55be87042010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb420e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8704ce10 0x55be8704f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb421800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be8704c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:57078/0 (socket says 172.21.15.112:57078) 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.751+0000 7fb421800640 1 -- 172.21.15.112:0/1240077983 learned_addr learned my addr 172.21.15.112:0/1240077983 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.752+0000 7fb420e00640 1 -- 172.21.15.112:0/1240077983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 msgr2=0x55be8704c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.752+0000 7fb420e00640 1 --2- 172.21.15.112:0/1240077983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be8704c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.752+0000 7fb420e00640 1 -- 172.21.15.112:0/1240077983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87041c30 msgr2=0x55be87042010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.752+0000 7fb420e00640 1 --2- 172.21.15.112:0/1240077983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87041c30 0x55be87042010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.752+0000 7fb420e00640 1 -- 172.21.15.112:0/1240077983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be86cb6210 con 0x55be8704ce10 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.752+0000 7fb41a000640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb4040a3dc0 0x7fb4040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.752+0000 7fb420e00640 1 --2- 172.21.15.112:0/1240077983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8704ce10 0x55be8704f200 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb408009230 tx=0x7fb408032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a2d5718e4aead62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.752+0000 7fb41a000640 1 -- 172.21.15.112:0/1397976326 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7fb4100d0c00 con 0x7fb41c06a780 2024-04-25T18:56:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.752+0000 7fb422200640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb4040a3dc0 0x7fb4040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.752+0000 7fb40ea00640 1 -- 172.21.15.112:0/1240077983 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40803f070 con 0x55be8704ce10 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.753+0000 7fb40ea00640 1 -- 172.21.15.112:0/1240077983 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb408032e40 con 0x55be8704ce10 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.753+0000 7fb40ea00640 1 -- 172.21.15.112:0/1240077983 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408035070 con 0x55be8704ce10 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.753+0000 7fb426ffe740 1 -- 172.21.15.112:0/1240077983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8704ce10 msgr2=0x55be8704f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.753+0000 7fb422200640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb4040a3dc0 0x7fb4040a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb414005fd0 tx=0x7fb414009500 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.753+0000 7fb426ffe740 1 --2- 172.21.15.112:0/1240077983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8704ce10 0x55be8704f200 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb408009230 tx=0x7fb408032b90 comp rx=0 tx=0).stop 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.754+0000 7fb426ffe740 1 -- 172.21.15.112:0/1240077983 shutdown_connections 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.754+0000 7fb426ffe740 1 --2- 172.21.15.112:0/1240077983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be8704ce10 0x55be8704f200 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.754+0000 7fb426ffe740 1 --2- 172.21.15.112:0/1240077983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be8704c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.754+0000 7fb426ffe740 1 --2- 172.21.15.112:0/1240077983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87041c30 0x55be87042010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.754+0000 7fb426ffe740 1 -- 172.21.15.112:0/1240077983 >> 172.21.15.112:0/1240077983 conn(0x55be8703e140 msgr2=0x55be8703e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.754+0000 7fb426ffe740 1 -- 172.21.15.112:0/1240077983 shutdown_connections 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.754+0000 7fb426ffe740 1 -- 172.21.15.112:0/1240077983 wait complete. 2024-04-25T18:56:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.754+0000 7fb426ffe740 1 -- start start 2024-04-25T18:56:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.754+0000 7fb426ffe740 1 Processor -- start 2024-04-25T18:56:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be87049e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87042550 0x55be8704a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be8704ce10 0x55be8704a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb426ffe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be8704f9e0 con 0x55be87042550 2024-04-25T18:56:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb426ffe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55be86f5d310 con 0x55be87041c30 2024-04-25T18:56:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb426ffe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55be8704fb60 con 0x55be8704ce10 2024-04-25T18:56:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb426ffe740 10 client.0 _ll_register_callbacks cb 0x55be86f2bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:56:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb420e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87042550 0x55be8704a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb422200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be8704ce10 0x55be8704a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb420e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87042550 0x55be8704a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33068/0 (socket says 172.21.15.112:33068) 2024-04-25T18:56:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb420e00640 1 -- 172.21.15.112:0/3344925057 learned_addr learned my addr 172.21.15.112:0/3344925057 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb421800640 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be87049e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb422200640 1 -- 172.21.15.112:0/3344925057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 msgr2=0x55be87049e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.755+0000 7fb422200640 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be87049e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb422200640 1 -- 172.21.15.112:0/3344925057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87042550 msgr2=0x55be8704a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb422200640 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87042550 0x55be8704a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb422200640 1 -- 172.21.15.112:0/3344925057 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be86d82620 con 0x55be8704ce10 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb420e00640 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87042550 0x55be8704a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb422200640 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be8704ce10 0x55be8704a890 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb414003f20 tx=0x7fb414004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5dc46e1c9f5f1a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb403400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb403400640 1 -- 172.21.15.112:0/3344925057 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41400f7b0 con 0x55be8704ce10 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb426ffe740 1 -- 172.21.15.112:0/3344925057 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be86fb78b0 con 0x55be8704ce10 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb403400640 1 -- 172.21.15.112:0/3344925057 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7fb41400f950 con 0x55be8704ce10 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb403400640 1 -- 172.21.15.112:0/3344925057 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb41400fb50 con 0x55be8704ce10 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb426ffe740 10 client.26020 fetch_fsmap learned FSMap version 36 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.756+0000 7fb426ffe740 1 -- 172.21.15.112:0/3344925057 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55be8705bd00 con 0x55be8704ce10 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.757+0000 7fb403400640 1 -- 172.21.15.112:0/3344925057 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55be86fb78b0 con 0x55be8704ce10 2024-04-25T18:56:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.757+0000 7fb403400640 1 -- 172.21.15.112:0/3344925057 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fb414026070 con 0x55be8704ce10 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.757+0000 7fb426ffe740 10 client.26020 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.757+0000 7fb426ffe740 10 client.26020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.757+0000 7fb426ffe740 20 client.26020 populate_metadata read hostname 'smithi112' 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.757+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55be8705d280 0x55be8705f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.757+0000 7fb426ffe740 4 client.26020 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.757+0000 7fb426ffe740 1 -- 172.21.15.112:0/3344925057 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be86fb1320 con 0x55be8705d280 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.757+0000 7fb421800640 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55be8705d280 0x55be8705f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.758+0000 7fb403400640 10 client.26020 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.758+0000 7fb421800640 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55be8705d280 0x55be8705f690 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927d80fe114ba53a server_cookie=feb8b34f25295ae6 in_seq=0 out_seq=0 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.758+0000 7fb403400640 1 -- 172.21.15.112:0/3344925057 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be86fb1320 con 0x55be8705d280 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.758+0000 7fb403400640 10 client.26020 handle_command_reply: tid=0 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 1 client.26020 shutdown 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 1 -- 172.21.15.112:0/3344925057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be8704ce10 msgr2=0x55be8704a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be8704ce10 0x55be8704a890 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb414003f20 tx=0x7fb414004ab0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 1 -- 172.21.15.112:0/3344925057 shutdown_connections 2024-04-25T18:56:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be8704ce10 0x55be8704a890 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55be8705d280 0x55be8705f690 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be87042550 0x55be8704a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3344925057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be87049e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb403400640 0 client.26020 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 1 -- 172.21.15.112:0/3344925057 >> 172.21.15.112:0/3344925057 conn(0x55be8703e140 msgr2=0x55be87043ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 1 -- 172.21.15.112:0/3344925057 shutdown_connections 2024-04-25T18:56:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 1 -- 172.21.15.112:0/3344925057 wait complete. 2024-04-25T18:56:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.759+0000 7fb426ffe740 20 client.26020 trim_cache size 0 max 16384 2024-04-25T18:56:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.775+0000 7fb426ffe740 1 Processor -- start 2024-04-25T18:56:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.775+0000 7fb426ffe740 1 -- start start 2024-04-25T18:56:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.775+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be870588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be87044640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be8704ce10 0x55be87038d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb426ffe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1ae1a0 con 0x55be8704ce10 2024-04-25T18:56:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb426ffe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1aba70 con 0x55be87041c30 2024-04-25T18:56:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb426ffe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1a98e0 con 0x55be87042550 2024-04-25T18:56:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb422200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be8704ce10 0x55be87038d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb422200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be8704ce10 0x55be87038d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33084/0 (socket says 172.21.15.112:33084) 2024-04-25T18:56:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb422200640 1 -- 172.21.15.112:0/3987172032 learned_addr learned my addr 172.21.15.112:0/3987172032 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb420e00640 1 --2- 172.21.15.112:0/3987172032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be87044640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb421800640 1 --2- 172.21.15.112:0/3987172032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be870588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb420e00640 1 -- 172.21.15.112:0/3987172032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 msgr2=0x55be870588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb420e00640 1 --2- 172.21.15.112:0/3987172032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be870588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb420e00640 1 -- 172.21.15.112:0/3987172032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be8704ce10 msgr2=0x55be87038d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb420e00640 1 --2- 172.21.15.112:0/3987172032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be8704ce10 0x55be87038d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.776+0000 7fb420e00640 1 -- 172.21.15.112:0/3987172032 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be87042c90 con 0x55be87042550 2024-04-25T18:56:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.777+0000 7fb420e00640 1 --2- 172.21.15.112:0/3987172032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be87044640 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb4080051b0 tx=0x7fb408009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9116887c785d89db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.777+0000 7fb40e000640 1 -- 172.21.15.112:0/3987172032 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408035050 con 0x55be87042550 2024-04-25T18:56:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.777+0000 7fb40e000640 1 -- 172.21.15.112:0/3987172032 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4080053e0 con 0x55be87042550 2024-04-25T18:56:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.777+0000 7fb40e000640 1 -- 172.21.15.112:0/3987172032 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40803f040 con 0x55be87042550 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.778+0000 7fb426ffe740 1 -- 172.21.15.112:0/3987172032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 msgr2=0x55be87044640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.778+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3987172032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be87044640 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb4080051b0 tx=0x7fb408009260 comp rx=0 tx=0).stop 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.778+0000 7fb426ffe740 1 -- 172.21.15.112:0/3987172032 shutdown_connections 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.778+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3987172032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be8704ce10 0x55be87038d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.778+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3987172032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be87044640 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.778+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3987172032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be870588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.778+0000 7fb426ffe740 1 -- 172.21.15.112:0/3987172032 >> 172.21.15.112:0/3987172032 conn(0x55be8703e140 msgr2=0x55be87038600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.778+0000 7fb426ffe740 1 -- 172.21.15.112:0/3987172032 shutdown_connections 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.778+0000 7fb426ffe740 1 -- 172.21.15.112:0/3987172032 wait complete. 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.778+0000 7fb426ffe740 1 -- start start 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb426ffe740 1 Processor -- start 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be87052810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be87052d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb426ffe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be8704ce10 0x55be87053290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb426ffe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1a9f30 con 0x55be8704ce10 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb426ffe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1ab3a0 con 0x55be87041c30 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb426ffe740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1aa600 con 0x55be87042550 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb426ffe740 10 client.0 _ll_register_callbacks cb 0x7fb414003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb420e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be87052d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb420e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be87052d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:57116/0 (socket says 172.21.15.112:57116) 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb420e00640 1 -- 172.21.15.112:0/3477998733 learned_addr learned my addr 172.21.15.112:0/3477998733 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb422200640 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be8704ce10 0x55be87053290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.779+0000 7fb421800640 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be87052810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.780+0000 7fb420e00640 1 -- 172.21.15.112:0/3477998733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 msgr2=0x55be87052810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.780+0000 7fb420e00640 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be87052810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.780+0000 7fb420e00640 1 -- 172.21.15.112:0/3477998733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be8704ce10 msgr2=0x55be87053290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.780+0000 7fb420e00640 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be8704ce10 0x55be87053290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.780+0000 7fb420e00640 1 -- 172.21.15.112:0/3477998733 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be870c2850 con 0x55be87042550 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.780+0000 7fb420e00640 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be87052d50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55be87049670 tx=0x7fb408032d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc26b054af6d45c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.780+0000 7fb403400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.780+0000 7fb403400640 1 -- 172.21.15.112:0/3477998733 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40803f030 con 0x55be87042550 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb403400640 1 -- 172.21.15.112:0/3477998733 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7fb4080317b0 con 0x55be87042550 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb426ffe740 1 -- 172.21.15.112:0/3477998733 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4080053e0 con 0x55be87042550 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb403400640 1 -- 172.21.15.112:0/3477998733 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb408060050 con 0x55be87042550 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb426ffe740 10 client.26026 fetch_fsmap learned FSMap version 36 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb426ffe740 1 -- 172.21.15.112:0/3477998733 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55be870c3b40 con 0x55be87042550 2024-04-25T18:56:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb403400640 1 -- 172.21.15.112:0/3477998733 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4080053e0 con 0x55be87042550 2024-04-25T18:56:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb403400640 1 -- 172.21.15.112:0/3477998733 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fb4080046c0 con 0x55be87042550 2024-04-25T18:56:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb426ffe740 10 client.26026 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T18:56:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb426ffe740 10 client.26026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:56:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb426ffe740 20 client.26026 populate_metadata read hostname 'smithi112' 2024-04-25T18:56:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55be870c3de0 0x55be870c6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb426ffe740 4 client.26026 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:56:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.781+0000 7fb426ffe740 1 -- 172.21.15.112:0/3477998733 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be86fab190 con 0x55be870c3de0 2024-04-25T18:56:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.782+0000 7fb421800640 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55be870c3de0 0x55be870c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.782+0000 7fb421800640 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55be870c3de0 0x55be870c6200 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a772ce8626fba92 server_cookie=29f52d989734e5eb in_seq=0 out_seq=0 2024-04-25T18:56:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.782+0000 7fb403400640 10 client.26026 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.797+0000 7fb403400640 1 -- 172.21.15.112:0/3477998733 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55be86fab190 con 0x55be870c3de0 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.797+0000 7fb403400640 10 client.26026 handle_command_reply: tid=0 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.797+0000 7fb426ffe740 1 client.26026 shutdown 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.797+0000 7fb426ffe740 1 -- 172.21.15.112:0/3477998733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 msgr2=0x55be87052d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.797+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be87052d50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55be87049670 tx=0x7fb408032d00 comp rx=0 tx=0).stop 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.797+0000 7fb426ffe740 1 -- 172.21.15.112:0/3477998733 shutdown_connections 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.797+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55be870c3de0 0x55be870c6200 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.797+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be8704ce10 0x55be87053290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.798+0000 7fb403400640 0 client.26026 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.797+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55be87042550 0x55be87052d50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.798+0000 7fb426ffe740 1 --2- 172.21.15.112:0/3477998733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be87041c30 0x55be87052810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.798+0000 7fb426ffe740 1 -- 172.21.15.112:0/3477998733 >> 172.21.15.112:0/3477998733 conn(0x55be8703e140 msgr2=0x55be87038e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.798+0000 7fb426ffe740 1 -- 172.21.15.112:0/3477998733 shutdown_connections 2024-04-25T18:56:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.798+0000 7fb426ffe740 1 -- 172.21.15.112:0/3477998733 wait complete. 2024-04-25T18:56:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.798+0000 7fb426ffe740 20 client.26026 trim_cache size 0 max 16384 2024-04-25T18:56:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:56:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:56:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:56:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 -- 172.21.15.112:0/1397976326 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb4040a3dc0 msgr2=0x7fb4040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb4040a3dc0 0x7fb4040a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb414005fd0 tx=0x7fb414009500 comp rx=0 tx=0).stop 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 -- 172.21.15.112:0/1397976326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb41c06a780 msgr2=0x7fb41c198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb41c06a780 0x7fb41c198600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb41000e9f0 tx=0x7fb41000eeb0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 -- 172.21.15.112:0/1397976326 shutdown_connections 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fb4040a3dc0 0x7fb4040a6280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb41c1080e0 0x7fb41c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41c06b120 0x7fb41c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 --2- 172.21.15.112:0/1397976326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb41c06a780 0x7fb41c198600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 -- 172.21.15.112:0/1397976326 >> 172.21.15.112:0/1397976326 conn(0x7fb41c076c50 msgr2=0x7fb41c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 -- 172.21.15.112:0/1397976326 shutdown_connections 2024-04-25T18:56:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:11.799+0000 7fb424200640 1 -- 172.21.15.112:0/1397976326 wait complete. 2024-04-25T18:56:12.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:11 smithi171 ceph-mon[33266]: pgmap v275: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.4 MiB/s wr, 471 op/s 2024-04-25T18:56:12.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:11 smithi161 ceph-mon[32609]: pgmap v275: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.4 MiB/s wr, 471 op/s 2024-04-25T18:56:12.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:11 smithi112 ceph-mon[27864]: pgmap v275: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.4 MiB/s wr, 471 op/s 2024-04-25T18:56:14.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:13 smithi171 ceph-mon[33266]: pgmap v276: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:56:14.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:13 smithi161 ceph-mon[32609]: pgmap v276: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:56:14.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:13 smithi112 ceph-mon[27864]: pgmap v276: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:56:16.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:15 smithi171 ceph-mon[33266]: pgmap v277: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:56:16.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:15 smithi161 ceph-mon[32609]: pgmap v277: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:56:16.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:15 smithi112 ceph-mon[27864]: pgmap v277: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-25T18:56:18.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:17 smithi161 ceph-mon[32609]: pgmap v278: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 303 op/s 2024-04-25T18:56:18.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:17 smithi112 ceph-mon[27864]: pgmap v278: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 303 op/s 2024-04-25T18:56:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:17 smithi171 ceph-mon[33266]: pgmap v278: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 303 op/s 2024-04-25T18:56:20.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:19 smithi161 ceph-mon[32609]: pgmap v279: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 302 op/s 2024-04-25T18:56:20.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:19 smithi112 ceph-mon[27864]: pgmap v279: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 302 op/s 2024-04-25T18:56:20.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:19 smithi171 ceph-mon[33266]: pgmap v279: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 302 op/s 2024-04-25T18:56:22.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:21 smithi161 ceph-mon[32609]: pgmap v280: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.6 MiB/s wr, 411 op/s 2024-04-25T18:56:22.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:21 smithi112 ceph-mon[27864]: pgmap v280: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.6 MiB/s wr, 411 op/s 2024-04-25T18:56:22.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:21 smithi171 ceph-mon[33266]: pgmap v280: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.6 MiB/s wr, 411 op/s 2024-04-25T18:56:24.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:23 smithi161 ceph-mon[32609]: pgmap v281: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 246 op/s 2024-04-25T18:56:24.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:23 smithi112 ceph-mon[27864]: pgmap v281: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 246 op/s 2024-04-25T18:56:24.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:23 smithi171 ceph-mon[33266]: pgmap v281: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 246 op/s 2024-04-25T18:56:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.957+0000 7fd06a800640 1 -- 172.21.15.112:0/81398664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0641043a0 msgr2=0x7fd064104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.957+0000 7fd06a800640 1 --2- 172.21.15.112:0/81398664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0641043a0 0x7fd064104780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd054009900 tx=0x7fd05402a410 comp rx=0 tx=0).stop 2024-04-25T18:56:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.957+0000 7fd06a800640 1 -- 172.21.15.112:0/81398664 shutdown_connections 2024-04-25T18:56:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.957+0000 7fd06a800640 1 --2- 172.21.15.112:0/81398664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd064109470 0x7fd064111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.957+0000 7fd06a800640 1 --2- 172.21.15.112:0/81398664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd064104d50 0x7fd064108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.957+0000 7fd06a800640 1 --2- 172.21.15.112:0/81398664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0641043a0 0x7fd064104780 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.957+0000 7fd06a800640 1 -- 172.21.15.112:0/81398664 >> 172.21.15.112:0/81398664 conn(0x7fd0640ffc40 msgr2=0x7fd064102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.957+0000 7fd06a800640 1 -- 172.21.15.112:0/81398664 shutdown_connections 2024-04-25T18:56:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.957+0000 7fd06a800640 1 -- 172.21.15.112:0/81398664 wait complete. 2024-04-25T18:56:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.958+0000 7fd06a800640 1 Processor -- start 2024-04-25T18:56:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.958+0000 7fd06a800640 1 -- start start 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd06a800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0641043a0 0x7fd0641a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd06a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd064109470 0x7fd0641a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd06a800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd06419a6a0 0x7fd06419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd06a800640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd064114400 con 0x7fd0641043a0 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd06a800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd064114280 con 0x7fd064109470 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd06a800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd064114580 con 0x7fd06419a6a0 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd063400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0641043a0 0x7fd0641a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd063400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0641043a0 0x7fd0641a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60006/0 (socket says 172.21.15.112:60006) 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd063400640 1 -- 172.21.15.112:0/325811868 learned_addr learned my addr 172.21.15.112:0/325811868 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd062a00640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd064109470 0x7fd0641a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd063e00640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd06419a6a0 0x7fd06419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd062a00640 1 -- 172.21.15.112:0/325811868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd06419a6a0 msgr2=0x7fd06419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd062a00640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd06419a6a0 0x7fd06419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd062a00640 1 -- 172.21.15.112:0/325811868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0641043a0 msgr2=0x7fd0641a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.959+0000 7fd062a00640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0641043a0 0x7fd0641a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.960+0000 7fd062a00640 1 -- 172.21.15.112:0/325811868 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06419b2a0 con 0x7fd064109470 2024-04-25T18:56:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.960+0000 7fd063400640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0641043a0 0x7fd0641a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.960+0000 7fd062a00640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd064109470 0x7fd0641a0b20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd05000e9c0 tx=0x7fd05000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.960+0000 7fd05be00640 1 -- 172.21.15.112:0/325811868 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05000ccc0 con 0x7fd064109470 2024-04-25T18:56:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.960+0000 7fd05be00640 1 -- 172.21.15.112:0/325811868 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0500108e0 con 0x7fd064109470 2024-04-25T18:56:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.960+0000 7fd05be00640 1 -- 172.21.15.112:0/325811868 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500187c0 con 0x7fd064109470 2024-04-25T18:56:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.960+0000 7fd06a800640 1 -- 172.21.15.112:0/325811868 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0641a7380 con 0x7fd064109470 2024-04-25T18:56:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.960+0000 7fd06a800640 1 -- 172.21.15.112:0/325811868 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0641a77e0 con 0x7fd064109470 2024-04-25T18:56:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.964+0000 7fd06d781740 1 Processor -- start 2024-04-25T18:56:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.964+0000 7fd06d781740 1 -- start start 2024-04-25T18:56:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.964+0000 7fd05be00640 1 -- 172.21.15.112:0/325811868 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fd050010430 con 0x7fd064109470 2024-04-25T18:56:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd06d781740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f201ebb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd06d781740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ec060 0x560f201fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd06d781740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 0x560f20201600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd06d781740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560f20151900 con 0x560f201ff210 2024-04-25T18:56:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd06d781740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560f200f7ef0 con 0x560f201ec060 2024-04-25T18:56:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd06d781740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560f2014d8c0 con 0x560f201eb740 2024-04-25T18:56:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd063400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ec060 0x560f201fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd063e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f201ebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd063e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f201ebb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:59346/0 (socket says 172.21.15.112:59346) 2024-04-25T18:56:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd063e00640 1 -- 172.21.15.112:0/3217361117 learned_addr learned my addr 172.21.15.112:0/3217361117 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd063400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ec060 0x560f201fecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:50424/0 (socket says 172.21.15.112:50424) 2024-04-25T18:56:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd062a00640 1 --2- 172.21.15.112:0/3217361117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 0x560f20201600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd063e00640 1 -- 172.21.15.112:0/3217361117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ec060 msgr2=0x560f201fecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd063e00640 1 --2- 172.21.15.112:0/3217361117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ec060 0x560f201fecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd063e00640 1 -- 172.21.15.112:0/3217361117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 msgr2=0x560f20201600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd063e00640 1 --2- 172.21.15.112:0/3217361117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 0x560f20201600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd063e00640 1 -- 172.21.15.112:0/3217361117 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1fe64210 con 0x560f201eb740 2024-04-25T18:56:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.965+0000 7fd062a00640 1 --2- 172.21.15.112:0/3217361117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 0x560f20201600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:56:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.966+0000 7fd063e00640 1 --2- 172.21.15.112:0/3217361117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f201ebb20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009f50 tx=0x7fd05c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80d973b98515afd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.966+0000 7fd05be00640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd0480a3e40 0x7fd0480a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.967+0000 7fd05be00640 1 -- 172.21.15.112:0/325811868 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7fd0500d0b30 con 0x7fd064109470 2024-04-25T18:56:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.967+0000 7fd063400640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd0480a3e40 0x7fd0480a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.967+0000 7fd063400640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd0480a3e40 0x7fd0480a6300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x560f201ee340 tx=0x7fd054005a20 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.968+0000 7fd058c00640 1 -- 172.21.15.112:0/3217361117 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c004d20 con 0x560f201eb740 2024-04-25T18:56:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.969+0000 7fd058c00640 1 -- 172.21.15.112:0/3217361117 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd05c004050 con 0x560f201eb740 2024-04-25T18:56:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.969+0000 7fd058c00640 1 -- 172.21.15.112:0/3217361117 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c005280 con 0x560f201eb740 2024-04-25T18:56:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.970+0000 7fd06d781740 1 -- 172.21.15.112:0/3217361117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 msgr2=0x560f201ebb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.970+0000 7fd06d781740 1 --2- 172.21.15.112:0/3217361117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f201ebb20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009f50 tx=0x7fd05c0118d0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.970+0000 7fd06d781740 1 -- 172.21.15.112:0/3217361117 shutdown_connections 2024-04-25T18:56:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.970+0000 7fd06d781740 1 --2- 172.21.15.112:0/3217361117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 0x560f20201600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.970+0000 7fd06d781740 1 --2- 172.21.15.112:0/3217361117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ec060 0x560f201fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.970+0000 7fd06d781740 1 --2- 172.21.15.112:0/3217361117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f201ebb20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.970+0000 7fd06d781740 1 -- 172.21.15.112:0/3217361117 >> 172.21.15.112:0/3217361117 conn(0x560f201e7d30 msgr2=0x560f201e8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.970+0000 7fd06d781740 1 -- 172.21.15.112:0/3217361117 shutdown_connections 2024-04-25T18:56:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.970+0000 7fd06d781740 1 -- 172.21.15.112:0/3217361117 wait complete. 2024-04-25T18:56:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.971+0000 7fd06d781740 1 -- start start 2024-04-25T18:56:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.971+0000 7fd06d781740 1 Processor -- start 2024-04-25T18:56:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.971+0000 7fd06d781740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 0x560f201fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.971+0000 7fd06d781740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ec060 0x560f201fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.971+0000 7fd06d781740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ff210 0x560f201fcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.971+0000 7fd06d781740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560f20201e40 con 0x560f201ec060 2024-04-25T18:56:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.971+0000 7fd06d781740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560f2010b310 con 0x560f201eb740 2024-04-25T18:56:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.971+0000 7fd06d781740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560f20201fc0 con 0x560f201ff210 2024-04-25T18:56:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.971+0000 7fd06d781740 10 client.0 _ll_register_callbacks cb 0x560f200d9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:56:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.972+0000 7fd063400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 0x560f201fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.972+0000 7fd063e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ff210 0x560f201fcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.972+0000 7fd063400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 0x560f201fc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:50430/0 (socket says 172.21.15.112:50430) 2024-04-25T18:56:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.972+0000 7fd063400640 1 -- 172.21.15.112:0/4021171237 learned_addr learned my addr 172.21.15.112:0/4021171237 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.972+0000 7fd062a00640 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ec060 0x560f201fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.972+0000 7fd063400640 1 -- 172.21.15.112:0/4021171237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ff210 msgr2=0x560f201fcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.972+0000 7fd063400640 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ff210 0x560f201fcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.972+0000 7fd063400640 1 -- 172.21.15.112:0/4021171237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ec060 msgr2=0x560f201fc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.972+0000 7fd063400640 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ec060 0x560f201fc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.972+0000 7fd063400640 1 -- 172.21.15.112:0/4021171237 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f1ff30620 con 0x560f201eb740 2024-04-25T18:56:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.972+0000 7fd063e00640 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ff210 0x560f201fcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.973+0000 7fd063400640 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 0x560f201fc1a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd05402aa00 tx=0x7fd054009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a909766ab45774dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.973+0000 7fd04d600640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:56:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.974+0000 7fd04d600640 1 -- 172.21.15.112:0/4021171237 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05404e070 con 0x560f201eb740 2024-04-25T18:56:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.974+0000 7fd06d781740 1 -- 172.21.15.112:0/4021171237 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd05c004050 con 0x560f201eb740 2024-04-25T18:56:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.974+0000 7fd04d600640 1 -- 172.21.15.112:0/4021171237 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7fd0540318b0 con 0x560f201eb740 2024-04-25T18:56:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.976+0000 7fd04d600640 1 -- 172.21.15.112:0/4021171237 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd05405a050 con 0x560f201eb740 2024-04-25T18:56:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.976+0000 7fd06d781740 10 client.26009 fetch_fsmap learned FSMap version 37 2024-04-25T18:56:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.976+0000 7fd06d781740 1 -- 172.21.15.112:0/4021171237 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560f2020e060 con 0x560f201eb740 2024-04-25T18:56:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.977+0000 7fd04d600640 1 -- 172.21.15.112:0/4021171237 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd05c004050 con 0x560f201eb740 2024-04-25T18:56:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.977+0000 7fd04d600640 1 -- 172.21.15.112:0/4021171237 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 37) ==== 8342+0+0 (secure 0 0 0) 0x7fd05404aa50 con 0x560f201eb740 2024-04-25T18:56:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.977+0000 7fd06d781740 10 client.26009 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T18:56:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.977+0000 7fd06d781740 10 client.26009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:56:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.977+0000 7fd06d781740 20 client.26009 populate_metadata read hostname 'smithi112' 2024-04-25T18:56:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.977+0000 7fd06d781740 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560f2020f4d0 0x560f20211900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.977+0000 7fd06d781740 4 client.26009 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:56:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.977+0000 7fd06d781740 1 -- 172.21.15.112:0/4021171237 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f201658b0 con 0x560f2020f4d0 2024-04-25T18:56:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.977+0000 7fd062a00640 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560f2020f4d0 0x560f20211900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.978+0000 7fd062a00640 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560f2020f4d0 0x560f20211900 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8602e67638f1a52 server_cookie=13e5608020afbb0f in_seq=0 out_seq=0 2024-04-25T18:56:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.978+0000 7fd04d600640 10 client.26009 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.978+0000 7fd04d600640 1 -- 172.21.15.112:0/4021171237 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f201658b0 con 0x560f2020f4d0 2024-04-25T18:56:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.978+0000 7fd04d600640 10 client.26009 handle_command_reply: tid=0 2024-04-25T18:56:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.978+0000 7fd06d781740 1 client.26009 shutdown 2024-04-25T18:56:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd06d781740 1 -- 172.21.15.112:0/4021171237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 msgr2=0x560f201fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd06d781740 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 0x560f201fc1a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd05402aa00 tx=0x7fd054009260 comp rx=0 tx=0).stop 2024-04-25T18:56:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd06d781740 1 -- 172.21.15.112:0/4021171237 shutdown_connections 2024-04-25T18:56:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd06d781740 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ff210 0x560f201fcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd06d781740 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560f2020f4d0 0x560f20211900 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd06d781740 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ec060 0x560f201fc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd06d781740 1 --2- 172.21.15.112:0/4021171237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 0x560f201fc1a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd06d781740 1 -- 172.21.15.112:0/4021171237 >> 172.21.15.112:0/4021171237 conn(0x560f201e7d30 msgr2=0x560f201edba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd04d600640 0 client.26009 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd06d781740 1 -- 172.21.15.112:0/4021171237 shutdown_connections 2024-04-25T18:56:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd06d781740 1 -- 172.21.15.112:0/4021171237 wait complete. 2024-04-25T18:56:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.979+0000 7fd06d781740 20 client.26009 trim_cache size 0 max 16384 2024-04-25T18:56:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.997+0000 7fd06d781740 1 Processor -- start 2024-04-25T18:56:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.997+0000 7fd06d781740 1 -- start start 2024-04-25T18:56:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.997+0000 7fd06d781740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 0x560f2020ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.997+0000 7fd06d781740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ec060 0x560f20217210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.997+0000 7fd06d781740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 0x560f20217750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.997+0000 7fd06d781740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641ae2b0 con 0x560f201ff210 2024-04-25T18:56:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.997+0000 7fd06d781740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641abb80 con 0x560f201eb740 2024-04-25T18:56:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.997+0000 7fd06d781740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641a9a10 con 0x560f201ec060 2024-04-25T18:56:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd063e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 0x560f20217750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd063e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 0x560f20217750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60014/0 (socket says 172.21.15.112:60014) 2024-04-25T18:56:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd063e00640 1 -- 172.21.15.112:0/4223448012 learned_addr learned my addr 172.21.15.112:0/4223448012 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd063400640 1 --2- 172.21.15.112:0/4223448012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 0x560f2020ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd062a00640 1 --2- 172.21.15.112:0/4223448012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ec060 0x560f20217210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd063400640 1 -- 172.21.15.112:0/4223448012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ec060 msgr2=0x560f20217210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd063400640 1 --2- 172.21.15.112:0/4223448012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ec060 0x560f20217210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd063400640 1 -- 172.21.15.112:0/4223448012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 msgr2=0x560f20217750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd063400640 1 --2- 172.21.15.112:0/4223448012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 0x560f20217750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd063400640 1 -- 172.21.15.112:0/4223448012 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f201e3470 con 0x560f201eb740 2024-04-25T18:56:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd062a00640 1 --2- 172.21.15.112:0/4223448012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ec060 0x560f20217210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd063e00640 1 --2- 172.21.15.112:0/4223448012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 0x560f20217750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.998+0000 7fd063400640 1 --2- 172.21.15.112:0/4223448012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 0x560f2020ad00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd054005ab0 tx=0x7fd05402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f26b7d3b143b56c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.999+0000 7fd04fe00640 1 -- 172.21.15.112:0/4223448012 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054063070 con 0x560f201eb740 2024-04-25T18:56:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.999+0000 7fd04fe00640 1 -- 172.21.15.112:0/4223448012 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0540036b0 con 0x560f201eb740 2024-04-25T18:56:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:24.999+0000 7fd04fe00640 1 -- 172.21.15.112:0/4223448012 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05404e040 con 0x560f201eb740 2024-04-25T18:56:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.000+0000 7fd06d781740 1 -- 172.21.15.112:0/4223448012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 msgr2=0x560f2020ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.000+0000 7fd06d781740 1 --2- 172.21.15.112:0/4223448012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 0x560f2020ad00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd054005ab0 tx=0x7fd05402a410 comp rx=0 tx=0).stop 2024-04-25T18:56:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.000+0000 7fd06d781740 1 -- 172.21.15.112:0/4223448012 shutdown_connections 2024-04-25T18:56:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.000+0000 7fd06d781740 1 --2- 172.21.15.112:0/4223448012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ff210 0x560f20217750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.000+0000 7fd06d781740 1 --2- 172.21.15.112:0/4223448012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201ec060 0x560f20217210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.000+0000 7fd06d781740 1 --2- 172.21.15.112:0/4223448012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201eb740 0x560f2020ad00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.000+0000 7fd06d781740 1 -- 172.21.15.112:0/4223448012 >> 172.21.15.112:0/4223448012 conn(0x560f201e7d30 msgr2=0x560f201f74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.000+0000 7fd06d781740 1 -- 172.21.15.112:0/4223448012 shutdown_connections 2024-04-25T18:56:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.000+0000 7fd06d781740 1 -- 172.21.15.112:0/4223448012 wait complete. 2024-04-25T18:56:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.000+0000 7fd06d781740 1 -- start start 2024-04-25T18:56:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.000+0000 7fd06d781740 1 Processor -- start 2024-04-25T18:56:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd06d781740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f202073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd06d781740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ec060 0x560f20204760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd06d781740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ff210 0x560f20204ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd06d781740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641aa060 con 0x560f201ec060 2024-04-25T18:56:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd06d781740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641ab4b0 con 0x560f201ff210 2024-04-25T18:56:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd06d781740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641aa710 con 0x560f201eb740 2024-04-25T18:56:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd06d781740 10 client.0 _ll_register_callbacks cb 0x7fd0540044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:56:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd063400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f202073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd062a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ec060 0x560f20204760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd063400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f202073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:59378/0 (socket says 172.21.15.112:59378) 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd063400640 1 -- 172.21.15.112:0/1818314163 learned_addr learned my addr 172.21.15.112:0/1818314163 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.001+0000 7fd063e00640 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ff210 0x560f20204ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.002+0000 7fd063400640 1 -- 172.21.15.112:0/1818314163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ff210 msgr2=0x560f20204ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.002+0000 7fd063400640 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ff210 0x560f20204ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.002+0000 7fd063400640 1 -- 172.21.15.112:0/1818314163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ec060 msgr2=0x560f20204760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.002+0000 7fd063400640 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ec060 0x560f20204760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.002+0000 7fd063400640 1 -- 172.21.15.112:0/1818314163 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f20205290 con 0x560f201eb740 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.002+0000 7fd063e00640 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ff210 0x560f20204ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.002+0000 7fd063400640 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f202073d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd054005df0 tx=0x7fd05403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9212316988c06cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.002+0000 7fd04d600640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.002+0000 7fd04d600640 1 -- 172.21.15.112:0/1818314163 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054063070 con 0x560f201eb740 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.002+0000 7fd04d600640 1 -- 172.21.15.112:0/1818314163 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7fd0540316b0 con 0x560f201eb740 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.002+0000 7fd06d781740 1 -- 172.21.15.112:0/1818314163 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0540036b0 con 0x560f201eb740 2024-04-25T18:56:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.003+0000 7fd04d600640 1 -- 172.21.15.112:0/1818314163 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd05404e040 con 0x560f201eb740 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.003+0000 7fd06d781740 10 client.26044 fetch_fsmap learned FSMap version 37 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.003+0000 7fd06d781740 1 -- 172.21.15.112:0/1818314163 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560f202758a0 con 0x560f201eb740 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.003+0000 7fd04d600640 1 -- 172.21.15.112:0/1818314163 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0540036b0 con 0x560f201eb740 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.003+0000 7fd04d600640 1 -- 172.21.15.112:0/1818314163 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 37) ==== 8342+0+0 (secure 0 0 0) 0x7fd05406e070 con 0x560f201eb740 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.003+0000 7fd06d781740 10 client.26044 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.003+0000 7fd06d781740 10 client.26044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.003+0000 7fd06d781740 20 client.26044 populate_metadata read hostname 'smithi112' 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.003+0000 7fd06d781740 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560f20275f30 0x560f20278350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.003+0000 7fd06d781740 4 client.26044 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.003+0000 7fd06d781740 1 -- 172.21.15.112:0/1818314163 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f2015f320 con 0x560f20275f30 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.004+0000 7fd062a00640 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560f20275f30 0x560f20278350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.004+0000 7fd062a00640 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560f20275f30 0x560f20278350 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=191ce6968c5a066a server_cookie=f5293813759ffdbf in_seq=0 out_seq=0 2024-04-25T18:56:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.004+0000 7fd04d600640 10 client.26044 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.169+0000 7fd04d600640 1 -- 172.21.15.112:0/1818314163 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f2015f320 con 0x560f20275f30 2024-04-25T18:56:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.169+0000 7fd04d600640 10 client.26044 handle_command_reply: tid=0 2024-04-25T18:56:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.169+0000 7fd06d781740 1 client.26044 shutdown 2024-04-25T18:56:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.170+0000 7fd06d781740 1 -- 172.21.15.112:0/1818314163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 msgr2=0x560f202073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.170+0000 7fd06d781740 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f202073d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd054005df0 tx=0x7fd05403f000 comp rx=0 tx=0).stop 2024-04-25T18:56:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.171+0000 7fd06d781740 1 -- 172.21.15.112:0/1818314163 shutdown_connections 2024-04-25T18:56:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.171+0000 7fd06d781740 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f201ff210 0x560f20204ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.171+0000 7fd06d781740 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x560f20275f30 0x560f20278350 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.171+0000 7fd06d781740 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f201ec060 0x560f20204760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.171+0000 7fd06d781740 1 --2- 172.21.15.112:0/1818314163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560f201eb740 0x560f202073d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.171+0000 7fd06d781740 1 -- 172.21.15.112:0/1818314163 >> 172.21.15.112:0/1818314163 conn(0x560f201e7d30 msgr2=0x560f201ed200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.171+0000 7fd04d600640 0 client.26044 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.171+0000 7fd06d781740 1 -- 172.21.15.112:0/1818314163 shutdown_connections 2024-04-25T18:56:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.171+0000 7fd06d781740 1 -- 172.21.15.112:0/1818314163 wait complete. 2024-04-25T18:56:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.171+0000 7fd06d781740 20 client.26044 trim_cache size 0 max 16384 2024-04-25T18:56:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T18:56:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T18:56:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T18:56:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 -- 172.21.15.112:0/325811868 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd0480a3e40 msgr2=0x7fd0480a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd0480a3e40 0x7fd0480a6300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x560f201ee340 tx=0x7fd054005a20 comp rx=0 tx=0).stop 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 -- 172.21.15.112:0/325811868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd064109470 msgr2=0x7fd0641a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd064109470 0x7fd0641a0b20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd05000e9c0 tx=0x7fd05000ee80 comp rx=0 tx=0).stop 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 -- 172.21.15.112:0/325811868 shutdown_connections 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd0480a3e40 0x7fd0480a6300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd06419a6a0 0x7fd06419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd064109470 0x7fd0641a0b20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 --2- 172.21.15.112:0/325811868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0641043a0 0x7fd0641a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 -- 172.21.15.112:0/325811868 >> 172.21.15.112:0/325811868 conn(0x7fd0640ffc40 msgr2=0x7fd064101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 -- 172.21.15.112:0/325811868 shutdown_connections 2024-04-25T18:56:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T18:56:25.173+0000 7fd06a800640 1 -- 172.21.15.112:0/325811868 wait complete. 2024-04-25T18:56:25.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:24 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:56:25.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:24 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:56:25.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:24 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:56:26.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:25 smithi171 ceph-mon[33266]: pgmap v282: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 246 op/s 2024-04-25T18:56:26.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:25 smithi161 ceph-mon[32609]: pgmap v282: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 246 op/s 2024-04-25T18:56:26.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:25 smithi112 ceph-mon[27864]: pgmap v282: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 246 op/s 2024-04-25T18:56:27.626 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-25T18:56:27.626 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T18:56:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:27 smithi171 ceph-mon[33266]: pgmap v283: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 453 op/s 2024-04-25T18:56:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:27 smithi161 ceph-mon[32609]: pgmap v283: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 453 op/s 2024-04-25T18:56:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:27 smithi112 ceph-mon[27864]: pgmap v283: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 453 op/s 2024-04-25T18:56:28.881 DEBUG:teuthology.parallel:result is None 2024-04-25T18:56:28.882 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T18:56:28.967 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T18:56:28.968 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T18:56:28.979 INFO:tasks.quiescer:joining Quiescers 2024-04-25T18:56:28.980 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-25T18:56:28.980 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:56:29.400 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:30.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:29 smithi112 ceph-mon[27864]: pgmap v284: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 365 op/s 2024-04-25T18:56:30.321 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:29 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:56:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:29 smithi171 ceph-mon[33266]: pgmap v284: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 365 op/s 2024-04-25T18:56:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:29 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:56:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:29 smithi161 ceph-mon[32609]: pgmap v284: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 365 op/s 2024-04-25T18:56:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:29 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:56:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.145+0000 7fa580600700 1 -- 172.21.15.112:0/4069503815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa578109c80 msgr2=0x7fa57810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.145+0000 7fa580600700 1 --2- 172.21.15.112:0/4069503815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa578109c80 0x7fa57810dcd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa5640099c0 tx=0x7fa564009cd0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.146+0000 7fa580600700 1 -- 172.21.15.112:0/4069503815 shutdown_connections 2024-04-25T18:56:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.146+0000 7fa580600700 1 --2- 172.21.15.112:0/4069503815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa57810e3b0 0x7fa578115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.146+0000 7fa580600700 1 --2- 172.21.15.112:0/4069503815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5781092d0 0x7fa5781096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.146+0000 7fa580600700 1 --2- 172.21.15.112:0/4069503815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa578109c80 0x7fa57810dcd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.146+0000 7fa580600700 1 -- 172.21.15.112:0/4069503815 >> 172.21.15.112:0/4069503815 conn(0x7fa578076210 msgr2=0x7fa578107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.146+0000 7fa580600700 1 -- 172.21.15.112:0/4069503815 shutdown_connections 2024-04-25T18:56:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.146+0000 7fa580600700 1 -- 172.21.15.112:0/4069503815 wait complete. 2024-04-25T18:56:32.149 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:31 smithi112 ceph-mon[27864]: pgmap v285: 97 pgs: 97 active+clean; 151 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 430 op/s 2024-04-25T18:56:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.147+0000 7fa580600700 1 Processor -- start 2024-04-25T18:56:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.148+0000 7fa580600700 1 -- start start 2024-04-25T18:56:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.148+0000 7fa580600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5781092d0 0x7fa5781a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.148+0000 7fa580600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa578109c80 0x7fa5781a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.148+0000 7fa580600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57810e3b0 0x7fa57819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.148+0000 7fa580600700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa578119290 con 0x7fa57810e3b0 2024-04-25T18:56:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.148+0000 7fa580600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa578119110 con 0x7fa578109c80 2024-04-25T18:56:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.148+0000 7fa580600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa578119410 con 0x7fa5781092d0 2024-04-25T18:56:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.148+0000 7fa57e600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57810e3b0 0x7fa57819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.149+0000 7fa57dc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5781092d0 0x7fa5781a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.149+0000 7fa57d200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa578109c80 0x7fa5781a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.149+0000 7fa57dc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5781092d0 0x7fa5781a5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:51518/0 (socket says 172.21.15.112:51518) 2024-04-25T18:56:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.149+0000 7fa57e600700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57810e3b0 0x7fa57819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36098/0 (socket says 172.21.15.112:36098) 2024-04-25T18:56:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.149+0000 7fa57dc00700 1 -- 172.21.15.112:0/38127802 learned_addr learned my addr 172.21.15.112:0/38127802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.149+0000 7fa57d200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa578109c80 0x7fa5781a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:60366/0 (socket says 172.21.15.112:60366) 2024-04-25T18:56:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.149+0000 7fa57e600700 1 -- 172.21.15.112:0/38127802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5781092d0 msgr2=0x7fa5781a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.149+0000 7fa57e600700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5781092d0 0x7fa5781a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.149+0000 7fa57e600700 1 -- 172.21.15.112:0/38127802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa578109c80 msgr2=0x7fa5781a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.149+0000 7fa57e600700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa578109c80 0x7fa5781a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.149+0000 7fa57e600700 1 -- 172.21.15.112:0/38127802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57819fd70 con 0x7fa57810e3b0 2024-04-25T18:56:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.150+0000 7fa57e600700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57810e3b0 0x7fa57819f610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa57000efc0 tx=0x7fa57000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.150+0000 7fa576000700 1 -- 172.21.15.112:0/38127802 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5700092e0 con 0x7fa57810e3b0 2024-04-25T18:56:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.150+0000 7fa576000700 1 -- 172.21.15.112:0/38127802 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa570004bb0 con 0x7fa57810e3b0 2024-04-25T18:56:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.150+0000 7fa580600700 1 -- 172.21.15.112:0/38127802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5781a0000 con 0x7fa57810e3b0 2024-04-25T18:56:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.150+0000 7fa576000700 1 -- 172.21.15.112:0/38127802 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570004020 con 0x7fa57810e3b0 2024-04-25T18:56:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.150+0000 7fa580600700 1 -- 172.21.15.112:0/38127802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5781ac320 con 0x7fa57810e3b0 2024-04-25T18:56:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.151+0000 7fa580600700 1 -- 172.21.15.112:0/38127802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57810a7b0 con 0x7fa57810e3b0 2024-04-25T18:56:32.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.154+0000 7fa576000700 1 -- 172.21.15.112:0/38127802 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fa57001b030 con 0x7fa57810e3b0 2024-04-25T18:56:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.155+0000 7fa576000700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa5600a3db0 0x7fa5600a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.155+0000 7fa576000700 1 -- 172.21.15.112:0/38127802 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7fa5700d0ca0 con 0x7fa57810e3b0 2024-04-25T18:56:32.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.155+0000 7fa57dc00700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa5600a3db0 0x7fa5600a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:32.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.156+0000 7fa57dc00700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa5600a3db0 0x7fa5600a6270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009d60 tx=0x7fa56c009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:32.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.158+0000 7fa576000700 1 -- 172.21.15.112:0/38127802 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa570022020 con 0x7fa57810e3b0 2024-04-25T18:56:32.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:31 smithi171 ceph-mon[33266]: pgmap v285: 97 pgs: 97 active+clean; 151 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 430 op/s 2024-04-25T18:56:32.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.438+0000 7fa580600700 1 -- 172.21.15.112:0/38127802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5781a0b40 con 0x7fa57810e3b0 2024-04-25T18:56:32.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.439+0000 7fa576000700 1 -- 172.21.15.112:0/38127802 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11895 (secure 0 0 0) 0x7fa570010040 con 0x7fa57810e3b0 2024-04-25T18:56:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:56:32.442 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:56:28.660237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25739":{"gid":25739,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3065773788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3065773788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3065773788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24493,24511,24512]},"id":1}]} 2024-04-25T18:56:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.442+0000 7fa580600700 1 -- 172.21.15.112:0/38127802 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa5600a3db0 msgr2=0x7fa5600a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.443+0000 7fa580600700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa5600a3db0 0x7fa5600a6270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009d60 tx=0x7fa56c009260 comp rx=0 tx=0).stop 2024-04-25T18:56:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.443+0000 7fa580600700 1 -- 172.21.15.112:0/38127802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57810e3b0 msgr2=0x7fa57819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.443+0000 7fa580600700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57810e3b0 0x7fa57819f610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa57000efc0 tx=0x7fa57000c3a0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.443+0000 7fa580600700 1 -- 172.21.15.112:0/38127802 shutdown_connections 2024-04-25T18:56:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.443+0000 7fa580600700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa5600a3db0 0x7fa5600a6270 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.443+0000 7fa580600700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa57810e3b0 0x7fa57819f610 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.443+0000 7fa580600700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5781092d0 0x7fa5781a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.443+0000 7fa580600700 1 --2- 172.21.15.112:0/38127802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa578109c80 0x7fa5781a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.443+0000 7fa580600700 1 -- 172.21.15.112:0/38127802 >> 172.21.15.112:0/38127802 conn(0x7fa578076210 msgr2=0x7fa578077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.443+0000 7fa580600700 1 -- 172.21.15.112:0/38127802 shutdown_connections 2024-04-25T18:56:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:32.443+0000 7fa580600700 1 -- 172.21.15.112:0/38127802 wait complete. 2024-04-25T18:56:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 38 2024-04-25T18:56:32.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:31 smithi161 ceph-mon[32609]: pgmap v285: 97 pgs: 97 active+clean; 151 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 430 op/s 2024-04-25T18:56:33.130 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:32 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/38127802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:56:33.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:32 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/38127802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:56:33.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:32 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/38127802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:56:33.799 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24512 quiesce db --cancel --all' 2024-04-25T18:56:33.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24512 quiesce db --cancel --all 2024-04-25T18:56:34.218 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:34.365 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:34 smithi112 ceph-mon[27864]: pgmap v286: 97 pgs: 97 active+clean; 151 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 917 KiB/s wr, 272 op/s 2024-04-25T18:56:34.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:34 smithi171 ceph-mon[33266]: pgmap v286: 97 pgs: 97 active+clean; 151 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 917 KiB/s wr, 272 op/s 2024-04-25T18:56:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:34 smithi161 ceph-mon[32609]: pgmap v286: 97 pgs: 97 active+clean; 151 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 917 KiB/s wr, 272 op/s 2024-04-25T18:56:36.210 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:36 smithi112 ceph-mon[27864]: pgmap v287: 97 pgs: 97 active+clean; 151 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 917 KiB/s wr, 272 op/s 2024-04-25T18:56:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:36 smithi171 ceph-mon[33266]: pgmap v287: 97 pgs: 97 active+clean; 151 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 917 KiB/s wr, 272 op/s 2024-04-25T18:56:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:36 smithi161 ceph-mon[32609]: pgmap v287: 97 pgs: 97 active+clean; 151 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 917 KiB/s wr, 272 op/s 2024-04-25T18:56:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.843+0000 7f14d6a00700 1 -- 172.21.15.112:0/2482545362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d0109c80 msgr2=0x7f14d010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.843+0000 7f14d6a00700 1 --2- 172.21.15.112:0/2482545362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d0109c80 0x7f14d010dcd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0099c0 tx=0x7f14bc009cd0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.843+0000 7f14d6a00700 1 -- 172.21.15.112:0/2482545362 shutdown_connections 2024-04-25T18:56:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.843+0000 7f14d6a00700 1 --2- 172.21.15.112:0/2482545362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14d010e3b0 0x7f14d0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.843+0000 7f14d6a00700 1 --2- 172.21.15.112:0/2482545362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14d01092d0 0x7f14d01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.843+0000 7f14d6a00700 1 --2- 172.21.15.112:0/2482545362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d0109c80 0x7f14d010dcd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.843+0000 7f14d6a00700 1 -- 172.21.15.112:0/2482545362 >> 172.21.15.112:0/2482545362 conn(0x7f14d0076210 msgr2=0x7f14d0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.843+0000 7f14d6a00700 1 -- 172.21.15.112:0/2482545362 shutdown_connections 2024-04-25T18:56:36.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.843+0000 7f14d6a00700 1 -- 172.21.15.112:0/2482545362 wait complete. 2024-04-25T18:56:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.844+0000 7f14d6a00700 1 Processor -- start 2024-04-25T18:56:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.844+0000 7f14d6a00700 1 -- start start 2024-04-25T18:56:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14d6a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14d01092d0 0x7f14d019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14d6a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14d0109c80 0x7f14d019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14d6a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d010e3b0 0x7f14d01a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14d6a00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14d0119220 con 0x7f14d010e3b0 2024-04-25T18:56:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14d6a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f14d01190a0 con 0x7f14d01092d0 2024-04-25T18:56:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14d6a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f14d01193a0 con 0x7f14d0109c80 2024-04-25T18:56:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14d4a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d010e3b0 0x7f14d01a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14cf400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14d0109c80 0x7f14d019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14cfe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14d01092d0 0x7f14d019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14d4a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d010e3b0 0x7f14d01a4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36122/0 (socket says 172.21.15.112:36122) 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14cf400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14d0109c80 0x7f14d019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:51532/0 (socket says 172.21.15.112:51532) 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14d4a00700 1 -- 172.21.15.112:0/3293730794 learned_addr learned my addr 172.21.15.112:0/3293730794 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14cf400700 1 -- 172.21.15.112:0/3293730794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14d01092d0 msgr2=0x7f14d019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14cf400700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14d01092d0 0x7f14d019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14cf400700 1 -- 172.21.15.112:0/3293730794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d010e3b0 msgr2=0x7f14d01a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14cf400700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d010e3b0 0x7f14d01a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.845+0000 7f14cf400700 1 -- 172.21.15.112:0/3293730794 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d01a4750 con 0x7f14d0109c80 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.846+0000 7f14cf400700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14d0109c80 0x7f14d019fcd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0098e0 tx=0x7f14bc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.846+0000 7f14ccc00700 1 -- 172.21.15.112:0/3293730794 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc026070 con 0x7f14d0109c80 2024-04-25T18:56:36.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.846+0000 7f14d6a00700 1 -- 172.21.15.112:0/3293730794 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d01a49e0 con 0x7f14d0109c80 2024-04-25T18:56:36.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.846+0000 7f14ccc00700 1 -- 172.21.15.112:0/3293730794 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14bc003c70 con 0x7f14d0109c80 2024-04-25T18:56:36.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.846+0000 7f14ccc00700 1 -- 172.21.15.112:0/3293730794 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc00f9c0 con 0x7f14d0109c80 2024-04-25T18:56:36.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.846+0000 7f14d6a00700 1 -- 172.21.15.112:0/3293730794 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d01ac2c0 con 0x7f14d0109c80 2024-04-25T18:56:36.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.850+0000 7f14ccc00700 1 -- 172.21.15.112:0/3293730794 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f14bc029070 con 0x7f14d0109c80 2024-04-25T18:56:36.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.851+0000 7f14e1365b80 1 Processor -- start 2024-04-25T18:56:36.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.851+0000 7f14e1365b80 1 -- start start 2024-04-25T18:56:36.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.851+0000 7f14ccc00700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f14b40a3e30 0x7f14b40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.851+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.851+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166eaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.851+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166eb440 0x55f6166ed830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.851+0000 7f14e1365b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f616632f00 con 0x55f6166eb440 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.851+0000 7f14e1365b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f6165d97a0 con 0x55f6166dfcb0 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.851+0000 7f14ccc00700 1 -- 172.21.15.112:0/3293730794 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7f14bc0d0f50 con 0x7f14d0109c80 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.851+0000 7f14e1365b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f61662eec0 con 0x55f6166e05d0 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14cfe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14d4a00700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f14b40a3e30 0x7f14b40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14cfe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:60400/0 (socket says 172.21.15.112:60400) 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14cfe00700 1 -- 172.21.15.112:0/2917952038 learned_addr learned my addr 172.21.15.112:0/2917952038 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14d4a00700 1 --2- 172.21.15.112:0/2917952038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166eb440 0x55f6166ed830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14cf400700 1 --2- 172.21.15.112:0/2917952038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166eaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14cfe00700 1 -- 172.21.15.112:0/2917952038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 msgr2=0x55f6166eaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:36.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14cfe00700 1 --2- 172.21.15.112:0/2917952038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166eaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14cfe00700 1 -- 172.21.15.112:0/2917952038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166eb440 msgr2=0x55f6166ed830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:36.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14cfe00700 1 --2- 172.21.15.112:0/2917952038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166eb440 0x55f6166ed830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14cfe00700 1 -- 172.21.15.112:0/2917952038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6165cf6f0 con 0x55f6166dfcb0 2024-04-25T18:56:36.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14cf400700 1 --2- 172.21.15.112:0/2917952038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166eaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:56:36.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.852+0000 7f14cfe00700 1 --2- 172.21.15.112:0/2917952038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166e0090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009d80 tx=0x7f14c0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2ca391bc9ed302 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:36.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.853+0000 7f14d4a00700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f14b40a3e30 0x7f14b40a62f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55f6166e1680 tx=0x7f14c8009920 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:36.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.853+0000 7f14c5600700 1 -- 172.21.15.112:0/2917952038 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0004230 con 0x55f6166dfcb0 2024-04-25T18:56:36.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.853+0000 7f14c5600700 1 -- 172.21.15.112:0/2917952038 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c00043d0 con 0x55f6166dfcb0 2024-04-25T18:56:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.853+0000 7f14c5600700 1 -- 172.21.15.112:0/2917952038 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0019830 con 0x55f6166dfcb0 2024-04-25T18:56:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.854+0000 7f14e1365b80 1 -- 172.21.15.112:0/2917952038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 msgr2=0x55f6166e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.854+0000 7f14e1365b80 1 --2- 172.21.15.112:0/2917952038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166e0090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009d80 tx=0x7f14c0011720 comp rx=0 tx=0).stop 2024-04-25T18:56:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.854+0000 7f14e1365b80 1 -- 172.21.15.112:0/2917952038 shutdown_connections 2024-04-25T18:56:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.854+0000 7f14e1365b80 1 --2- 172.21.15.112:0/2917952038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166eb440 0x55f6166ed830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.854+0000 7f14e1365b80 1 --2- 172.21.15.112:0/2917952038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166e0090 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.854+0000 7f14e1365b80 1 --2- 172.21.15.112:0/2917952038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166eaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.854+0000 7f14e1365b80 1 -- 172.21.15.112:0/2917952038 >> 172.21.15.112:0/2917952038 conn(0x55f61660efc0 msgr2=0x55f6166dc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.854+0000 7f14e1365b80 1 -- 172.21.15.112:0/2917952038 shutdown_connections 2024-04-25T18:56:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.854+0000 7f14e1365b80 1 -- 172.21.15.112:0/2917952038 wait complete. 2024-04-25T18:56:36.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.854+0000 7f14e1365b80 1 -- start start 2024-04-25T18:56:36.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14e1365b80 1 Processor -- start 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166eab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166e05d0 0x55f6166e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166eb440 0x55f6166e8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14e1365b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f6166ede10 con 0x55f6166e05d0 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14e1365b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f6165ec8d0 con 0x55f6166dfcb0 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14e1365b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f6166edf90 con 0x55f6166eb440 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14e1365b80 10 client.0 _ll_register_callbacks cb 0x55f6165cb790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14cf400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166e05d0 0x55f6166e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14cfe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14cf400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166e05d0 0x55f6166e7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36142/0 (socket says 172.21.15.112:36142) 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14cf400700 1 -- 172.21.15.112:0/3292965569 learned_addr learned my addr 172.21.15.112:0/3292965569 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.855+0000 7f14d4a00700 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166eb440 0x55f6166e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.856+0000 7f14cfe00700 1 -- 172.21.15.112:0/3292965569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166eb440 msgr2=0x55f6166e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:36.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.856+0000 7f14cfe00700 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166eb440 0x55f6166e8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.856+0000 7f14cfe00700 1 -- 172.21.15.112:0/3292965569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166e05d0 msgr2=0x55f6166e7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.856+0000 7f14cfe00700 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166e05d0 0x55f6166e7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.856+0000 7f14cfe00700 1 -- 172.21.15.112:0/3292965569 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f616580940 con 0x55f6166dfcb0 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.856+0000 7f14cfe00700 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166eab10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009d80 tx=0x7f14c00052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f3164212a8de47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.856+0000 7f14ba000700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.856+0000 7f14ba000700 1 -- 172.21.15.112:0/3292965569 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0027070 con 0x55f6166dfcb0 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.857+0000 7f14ba000700 1 -- 172.21.15.112:0/3292965569 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7f14c0004950 con 0x55f6166dfcb0 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.857+0000 7f14e1365b80 1 -- 172.21.15.112:0/3292965569 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c00043d0 con 0x55f6166dfcb0 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.857+0000 7f14ba000700 1 -- 172.21.15.112:0/3292965569 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f14c0033050 con 0x55f6166dfcb0 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.857+0000 7f14e1365b80 10 client.26033 fetch_fsmap learned FSMap version 38 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.857+0000 7f14e1365b80 1 -- 172.21.15.112:0/3292965569 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f6165c9b60 con 0x55f6166dfcb0 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.857+0000 7f14ba000700 1 -- 172.21.15.112:0/3292965569 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c00043d0 con 0x55f6166dfcb0 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.857+0000 7f14ba000700 1 -- 172.21.15.112:0/3292965569 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f14c0003c10 con 0x55f6166dfcb0 2024-04-25T18:56:36.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.857+0000 7f14e1365b80 10 client.26033 fetch_fsmap finished waiting for FSMap version 38 2024-04-25T18:56:36.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.857+0000 7f14e1365b80 10 client.26033 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-25T18:56:36.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.857+0000 7f14e1365b80 20 client.26033 populate_metadata read hostname 'smithi112' 2024-04-25T18:56:36.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.858+0000 7f14e1365b80 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f6166fa9d0 0x55f6166fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.858+0000 7f14e1365b80 4 client.26033 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:56:36.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.858+0000 7f14e1365b80 1 -- 172.21.15.112:0/3292965569 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6166406f0 con 0x55f6166fa9d0 2024-04-25T18:56:36.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.858+0000 7f14cf400700 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f6166fa9d0 0x55f6166fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.858+0000 7f14cf400700 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f6166fa9d0 0x55f6166fcdc0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7a23d823a62aad7 server_cookie=d96e7abae2c09a93 in_seq=0 out_seq=0 2024-04-25T18:56:36.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.858+0000 7f14ba000700 10 client.26033 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:36.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14ba000700 1 -- 172.21.15.112:0/3292965569 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6166406f0 con 0x55f6166fa9d0 2024-04-25T18:56:36.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14ba000700 10 client.26033 handle_command_reply: tid=0 2024-04-25T18:56:36.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14e1365b80 1 client.26033 shutdown 2024-04-25T18:56:36.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14e1365b80 1 -- 172.21.15.112:0/3292965569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 msgr2=0x55f6166eab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14e1365b80 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166eab10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009d80 tx=0x7f14c00052f0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14e1365b80 1 -- 172.21.15.112:0/3292965569 shutdown_connections 2024-04-25T18:56:36.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14e1365b80 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f6166fa9d0 0x55f6166fcdc0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14e1365b80 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166eb440 0x55f6166e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14e1365b80 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166eab10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14e1365b80 1 --2- 172.21.15.112:0/3292965569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166e05d0 0x55f6166e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14ba000700 0 client.26033 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:36.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.859+0000 7f14e1365b80 1 -- 172.21.15.112:0/3292965569 >> 172.21.15.112:0/3292965569 conn(0x55f61660efc0 msgr2=0x55f6166dbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:36.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.860+0000 7f14e1365b80 1 -- 172.21.15.112:0/3292965569 shutdown_connections 2024-04-25T18:56:36.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.860+0000 7f14e1365b80 1 -- 172.21.15.112:0/3292965569 wait complete. 2024-04-25T18:56:36.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.860+0000 7f14e1365b80 20 client.26033 trim_cache size 0 max 16384 2024-04-25T18:56:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.882+0000 7f14e1365b80 1 Processor -- start 2024-04-25T18:56:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.882+0000 7f14e1365b80 1 -- start start 2024-04-25T18:56:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.882+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166dfcb0 0x55f6166e9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.882+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.882+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166eb440 0x55f6166f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.882+0000 7f14e1365b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14d01b6270 con 0x55f6166dfcb0 2024-04-25T18:56:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.882+0000 7f14e1365b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f14d01b3ad0 con 0x55f6166eb440 2024-04-25T18:56:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.882+0000 7f14e1365b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f14d01b1940 con 0x55f6166e05d0 2024-04-25T18:56:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.882+0000 7f14cfe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166dfcb0 0x55f6166e9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.882+0000 7f14cfe00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166dfcb0 0x55f6166e9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36158/0 (socket says 172.21.15.112:36158) 2024-04-25T18:56:36.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.883+0000 7f14d4a00700 1 --2- 172.21.15.112:0/3118486996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166eb440 0x55f6166f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.882+0000 7f14cfe00700 1 -- 172.21.15.112:0/3118486996 learned_addr learned my addr 172.21.15.112:0/3118486996 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.883+0000 7f14cf400700 1 --2- 172.21.15.112:0/3118486996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.883+0000 7f14cfe00700 1 -- 172.21.15.112:0/3118486996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 msgr2=0x55f6166dc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.883+0000 7f14cfe00700 1 --2- 172.21.15.112:0/3118486996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166dc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.883+0000 7f14cfe00700 1 -- 172.21.15.112:0/3118486996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166eb440 msgr2=0x55f6166f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.883+0000 7f14cfe00700 1 --2- 172.21.15.112:0/3118486996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166eb440 0x55f6166f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.883+0000 7f14cfe00700 1 -- 172.21.15.112:0/3118486996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6166fd350 con 0x55f6166dfcb0 2024-04-25T18:56:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.883+0000 7f14d4a00700 1 --2- 172.21.15.112:0/3118486996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166eb440 0x55f6166f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.883+0000 7f14cfe00700 1 --2- 172.21.15.112:0/3118486996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166dfcb0 0x55f6166e9cb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f14c00092a0 tx=0x7f14c0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08799bd0ae17d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.884+0000 7f14c4c00700 1 -- 172.21.15.112:0/3118486996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0030020 con 0x55f6166dfcb0 2024-04-25T18:56:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.884+0000 7f14c4c00700 1 -- 172.21.15.112:0/3118486996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c0023750 con 0x55f6166dfcb0 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.884+0000 7f14c4c00700 1 -- 172.21.15.112:0/3118486996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0027030 con 0x55f6166dfcb0 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.884+0000 7f14e1365b80 1 -- 172.21.15.112:0/3118486996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166dfcb0 msgr2=0x55f6166e9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.884+0000 7f14e1365b80 1 --2- 172.21.15.112:0/3118486996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166dfcb0 0x55f6166e9cb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f14c00092a0 tx=0x7f14c0009f90 comp rx=0 tx=0).stop 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.884+0000 7f14e1365b80 1 -- 172.21.15.112:0/3118486996 shutdown_connections 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.884+0000 7f14e1365b80 1 --2- 172.21.15.112:0/3118486996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166eb440 0x55f6166f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.884+0000 7f14e1365b80 1 --2- 172.21.15.112:0/3118486996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166dfcb0 0x55f6166e9cb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.884+0000 7f14e1365b80 1 --2- 172.21.15.112:0/3118486996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166dc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.884+0000 7f14e1365b80 1 -- 172.21.15.112:0/3118486996 >> 172.21.15.112:0/3118486996 conn(0x55f61660efc0 msgr2=0x55f6166dbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.885+0000 7f14e1365b80 1 -- 172.21.15.112:0/3118486996 shutdown_connections 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.885+0000 7f14e1365b80 1 -- 172.21.15.112:0/3118486996 wait complete. 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.885+0000 7f14e1365b80 1 -- start start 2024-04-25T18:56:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.885+0000 7f14e1365b80 1 Processor -- start 2024-04-25T18:56:36.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166efe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14e1365b80 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166eb440 0x55f6166f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14e1365b80 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14d01b1f90 con 0x55f6166eb440 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14e1365b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f14d01b3400 con 0x55f6166dfcb0 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14e1365b80 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f14d01b2660 con 0x55f6166e05d0 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14e1365b80 10 client.0 _ll_register_callbacks cb 0x55f6165cb790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14d4a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166eb440 0x55f6166f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14cf400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14d4a00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166eb440 0x55f6166f08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36170/0 (socket says 172.21.15.112:36170) 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14d4a00700 1 -- 172.21.15.112:0/133021058 learned_addr learned my addr 172.21.15.112:0/133021058 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.886+0000 7f14cfe00700 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.887+0000 7f14cf400700 1 -- 172.21.15.112:0/133021058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 msgr2=0x55f6166efe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.887+0000 7f14cf400700 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166efe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.887+0000 7f14cf400700 1 -- 172.21.15.112:0/133021058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166eb440 msgr2=0x55f6166f08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.887+0000 7f14cf400700 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166eb440 0x55f6166f08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.887+0000 7f14cf400700 1 -- 172.21.15.112:0/133021058 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f616760fe0 con 0x55f6166e05d0 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.887+0000 7f14cf400700 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166f03a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f14bc099350 tx=0x7f14bc00b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1832c5d2fa1d147 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.887+0000 7f14ba000700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.887+0000 7f14ba000700 1 -- 172.21.15.112:0/133021058 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc003e20 con 0x55f6166e05d0 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.887+0000 7f14ba000700 1 -- 172.21.15.112:0/133021058 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7f14bc0b6170 con 0x55f6166e05d0 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.887+0000 7f14e1365b80 1 -- 172.21.15.112:0/133021058 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c0023750 con 0x55f6166e05d0 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.887+0000 7f14ba000700 1 -- 172.21.15.112:0/133021058 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f14bc09cef0 con 0x55f6166e05d0 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.888+0000 7f14e1365b80 10 client.26068 fetch_fsmap learned FSMap version 38 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.888+0000 7f14e1365b80 1 -- 172.21.15.112:0/133021058 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f616762340 con 0x55f6166e05d0 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.888+0000 7f14ba000700 1 -- 172.21.15.112:0/133021058 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c0023750 con 0x55f6166e05d0 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.888+0000 7f14ba000700 1 -- 172.21.15.112:0/133021058 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f14bc0a5640 con 0x55f6166e05d0 2024-04-25T18:56:36.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.888+0000 7f14e1365b80 10 client.26068 fetch_fsmap finished waiting for FSMap version 38 2024-04-25T18:56:36.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.888+0000 7f14e1365b80 10 client.26068 resolve_mds: validated gid 24512 aka daemon mds.b 2024-04-25T18:56:36.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.888+0000 7f14e1365b80 20 client.26068 populate_metadata read hostname 'smithi112' 2024-04-25T18:56:36.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.888+0000 7f14e1365b80 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f616762a00 0x55f616764e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.888+0000 7f14e1365b80 4 client.26068 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-25T18:56:36.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.888+0000 7f14e1365b80 1 -- 172.21.15.112:0/133021058 --> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55f61663a790 con 0x55f616762a00 2024-04-25T18:56:36.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.888+0000 7f14cfe00700 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f616762a00 0x55f616764e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.889+0000 7f14cfe00700 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f616762a00 0x55f616764e20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e96d0c7bc6ec8fc5 server_cookie=eca9ea5a5c0dd28e in_seq=0 out_seq=0 2024-04-25T18:56:36.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.889+0000 7f14ba000700 10 client.26068 ms_handle_connect on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:36.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.889+0000 7f14ba000700 1 -- 172.21.15.112:0/133021058 <== mds.0 v2:172.21.15.161:6838/2315334000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x55f61663a790 con 0x55f616762a00 2024-04-25T18:56:36.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.889+0000 7f14ba000700 10 client.26068 handle_command_reply: tid=0 2024-04-25T18:56:36.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.889+0000 7f14e1365b80 1 client.26068 shutdown 2024-04-25T18:56:36.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14e1365b80 1 -- 172.21.15.112:0/133021058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 msgr2=0x55f6166f03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14e1365b80 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166f03a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f14bc099350 tx=0x7f14bc00b540 comp rx=0 tx=0).stop 2024-04-25T18:56:36.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14e1365b80 1 -- 172.21.15.112:0/133021058 shutdown_connections 2024-04-25T18:56:36.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14e1365b80 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.161:6838/2315334000,v1:172.21.15.161:6839/2315334000] conn(0x55f616762a00 0x55f616764e20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14e1365b80 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f6166eb440 0x55f6166f08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14e1365b80 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6166dfcb0 0x55f6166efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14e1365b80 1 --2- 172.21.15.112:0/133021058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f6166e05d0 0x55f6166f03a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14ba000700 0 client.26068 ms_handle_reset on v2:172.21.15.161:6838/2315334000 2024-04-25T18:56:36.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14e1365b80 1 -- 172.21.15.112:0/133021058 >> 172.21.15.112:0/133021058 conn(0x55f61660efc0 msgr2=0x55f6166f5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:36.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14e1365b80 1 -- 172.21.15.112:0/133021058 shutdown_connections 2024-04-25T18:56:36.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14e1365b80 1 -- 172.21.15.112:0/133021058 wait complete. 2024-04-25T18:56:36.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.890+0000 7f14e1365b80 20 client.26068 trim_cache size 0 max 16384 2024-04-25T18:56:36.894 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:56:36.894 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 38, 2024-04-25T18:56:36.895 INFO:teuthology.orchestra.run.smithi112.stdout: "leader": 24512, 2024-04-25T18:56:36.895 INFO:teuthology.orchestra.run.smithi112.stdout: "set_version": 6, 2024-04-25T18:56:36.895 INFO:teuthology.orchestra.run.smithi112.stdout: "sets": {} 2024-04-25T18:56:36.895 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:56:36.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 -- 172.21.15.112:0/3293730794 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f14b40a3e30 msgr2=0x7f14b40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f14b40a3e30 0x7f14b40a62f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55f6166e1680 tx=0x7f14c8009920 comp rx=0 tx=0).stop 2024-04-25T18:56:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 -- 172.21.15.112:0/3293730794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14d0109c80 msgr2=0x7f14d019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14d0109c80 0x7f14d019fcd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0098e0 tx=0x7f14bc004270 comp rx=0 tx=0).stop 2024-04-25T18:56:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 -- 172.21.15.112:0/3293730794 shutdown_connections 2024-04-25T18:56:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f14b40a3e30 0x7f14b40a62f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14d010e3b0 0x7f14d01a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14d01092d0 0x7f14d019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 --2- 172.21.15.112:0/3293730794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14d0109c80 0x7f14d019fcd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 -- 172.21.15.112:0/3293730794 >> 172.21.15.112:0/3293730794 conn(0x7f14d0076210 msgr2=0x7f14d010f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 -- 172.21.15.112:0/3293730794 shutdown_connections 2024-04-25T18:56:36.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:56:36.891+0000 7f14d6a00700 1 -- 172.21.15.112:0/3293730794 wait complete. 2024-04-25T18:56:37.855 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa/tasks/quiescer.py", line 411, in task stop_all_quiescers(ctx.ceph[cluster_name].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa/tasks/quiescer.py", line 370, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error quiescing set '5aed9937': 110 (ETIMEDOUT) 2024-04-25T18:56:37.856 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T18:56:37.867 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T18:56:37.895 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T18:56:37.906 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T18:56:37.906 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T18:56:37.906 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T18:56:37.906 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T18:56:38.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:38 smithi112 ceph-mon[27864]: pgmap v288: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 961 KiB/s wr, 281 op/s 2024-04-25T18:56:38.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T18:56:38.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:38 smithi171 ceph-mon[33266]: pgmap v288: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 961 KiB/s wr, 281 op/s 2024-04-25T18:56:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:38 smithi161 ceph-mon[32609]: pgmap v288: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 961 KiB/s wr, 281 op/s 2024-04-25T18:56:40.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:40 smithi171 ceph-mon[33266]: pgmap v289: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 356 KiB/s wr, 75 op/s 2024-04-25T18:56:40.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:40 smithi161 ceph-mon[32609]: pgmap v289: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 356 KiB/s wr, 75 op/s 2024-04-25T18:56:40.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:40 smithi112 ceph-mon[27864]: pgmap v289: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 356 KiB/s wr, 75 op/s 2024-04-25T18:56:42.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:42 smithi171 ceph-mon[33266]: pgmap v290: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 359 KiB/s wr, 79 op/s 2024-04-25T18:56:42.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:42 smithi161 ceph-mon[32609]: pgmap v290: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 359 KiB/s wr, 79 op/s 2024-04-25T18:56:42.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:42 smithi112 ceph-mon[27864]: pgmap v290: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 359 KiB/s wr, 79 op/s 2024-04-25T18:56:44.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:44 smithi171 ceph-mon[33266]: pgmap v291: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-04-25T18:56:44.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:44 smithi161 ceph-mon[32609]: pgmap v291: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-04-25T18:56:44.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:44 smithi112 ceph-mon[27864]: pgmap v291: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-04-25T18:56:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:46 smithi171 ceph-mon[33266]: pgmap v292: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-04-25T18:56:46.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:46 smithi161 ceph-mon[32609]: pgmap v292: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-04-25T18:56:46.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:46 smithi112 ceph-mon[27864]: pgmap v292: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-04-25T18:56:47.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:47 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:47.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:47 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:47.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:47 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:48 smithi171 ceph-mon[33266]: pgmap v293: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-04-25T18:56:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:48 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:48 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:48.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:48 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:56:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:48 smithi161 ceph-mon[32609]: pgmap v293: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-04-25T18:56:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:48 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:48 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:48 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:56:48.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:48 smithi112 ceph-mon[27864]: pgmap v293: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-04-25T18:56:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:48 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:48 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:48.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:48 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:56:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:50 smithi171 ceph-mon[33266]: pgmap v294: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-25T18:56:50.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:50 smithi161 ceph-mon[32609]: pgmap v294: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-25T18:56:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:50 smithi112 ceph-mon[27864]: pgmap v294: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-25T18:56:52.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:52 smithi171 ceph-mon[33266]: pgmap v295: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-04-25T18:56:52.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:52 smithi161 ceph-mon[32609]: pgmap v295: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-04-25T18:56:52.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:52 smithi112 ceph-mon[27864]: pgmap v295: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-04-25T18:56:54.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:54 smithi171 ceph-mon[33266]: pgmap v296: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-25T18:56:54.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:54 smithi161 ceph-mon[32609]: pgmap v296: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-25T18:56:54.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:54 smithi112 ceph-mon[27864]: pgmap v296: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-25T18:56:56.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:56 smithi171 ceph-mon[33266]: pgmap v297: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-25T18:56:56.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:56 smithi161 ceph-mon[32609]: pgmap v297: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-25T18:56:56.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:56 smithi112 ceph-mon[27864]: pgmap v297: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-25T18:56:58.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:56:58 smithi171 ceph-mon[33266]: pgmap v298: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-25T18:56:58.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:56:58 smithi161 ceph-mon[32609]: pgmap v298: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-25T18:56:58.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:56:58 smithi112 ceph-mon[27864]: pgmap v298: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-25T18:57:00.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:00 smithi112 ceph-mon[27864]: pgmap v299: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-25T18:57:00.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:00 smithi171 ceph-mon[33266]: pgmap v299: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-25T18:57:00.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:00 smithi161 ceph-mon[32609]: pgmap v299: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-25T18:57:02.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:02 smithi171 ceph-mon[33266]: pgmap v300: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-25T18:57:02.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:02 smithi161 ceph-mon[32609]: pgmap v300: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-25T18:57:02.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:02 smithi112 ceph-mon[27864]: pgmap v300: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-25T18:57:04.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:04 smithi171 ceph-mon[33266]: pgmap v301: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T18:57:04.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:04 smithi161 ceph-mon[32609]: pgmap v301: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T18:57:04.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:04 smithi112 ceph-mon[27864]: pgmap v301: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T18:57:06.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:06 smithi171 ceph-mon[33266]: pgmap v302: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T18:57:06.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:06 smithi161 ceph-mon[32609]: pgmap v302: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T18:57:06.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:06 smithi112 ceph-mon[27864]: pgmap v302: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-25T18:57:08.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:08 smithi171 ceph-mon[33266]: pgmap v303: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-25T18:57:08.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:08 smithi171 ceph-mon[33266]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T18:57:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:08 smithi161 ceph-mon[32609]: pgmap v303: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-25T18:57:08.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:08 smithi161 ceph-mon[32609]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T18:57:08.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:08 smithi112 ceph-mon[27864]: pgmap v303: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-04-25T18:57:08.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:08 smithi112 ceph-mon[27864]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T18:57:10.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:10 smithi171 ceph-mon[33266]: pgmap v305: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T18:57:10.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:10 smithi161 ceph-mon[32609]: pgmap v305: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T18:57:10.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:10 smithi112 ceph-mon[27864]: pgmap v305: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T18:57:12.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:12 smithi171 ceph-mon[33266]: pgmap v306: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T18:57:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:12 smithi161 ceph-mon[32609]: pgmap v306: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T18:57:12.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:12 smithi112 ceph-mon[27864]: pgmap v306: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T18:57:14.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:14 smithi171 ceph-mon[33266]: pgmap v307: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T18:57:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:14 smithi171 ceph-mon[33266]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T18:57:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:14 smithi161 ceph-mon[32609]: pgmap v307: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T18:57:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:14 smithi161 ceph-mon[32609]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T18:57:14.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:14 smithi112 ceph-mon[27864]: pgmap v307: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T18:57:14.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:14 smithi112 ceph-mon[27864]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T18:57:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:16 smithi171 ceph-mon[33266]: pgmap v309: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T18:57:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:16 smithi161 ceph-mon[32609]: pgmap v309: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T18:57:16.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:16 smithi112 ceph-mon[27864]: pgmap v309: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T18:57:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:18 smithi171 ceph-mon[33266]: pgmap v310: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-04-25T18:57:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:18 smithi161 ceph-mon[32609]: pgmap v310: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-04-25T18:57:18.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:18 smithi112 ceph-mon[27864]: pgmap v310: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-04-25T18:57:20.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:20 smithi171 ceph-mon[33266]: pgmap v311: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T18:57:20.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:20 smithi161 ceph-mon[32609]: pgmap v311: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T18:57:20.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:20 smithi112 ceph-mon[27864]: pgmap v311: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T18:57:22.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:22 smithi171 ceph-mon[33266]: pgmap v312: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T18:57:22.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:22 smithi161 ceph-mon[32609]: pgmap v312: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T18:57:22.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:22 smithi112 ceph-mon[27864]: pgmap v312: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T18:57:24.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:24 smithi171 ceph-mon[33266]: pgmap v313: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T18:57:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:24 smithi161 ceph-mon[32609]: pgmap v313: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T18:57:24.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:24 smithi112 ceph-mon[27864]: pgmap v313: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T18:57:26.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:26 smithi171 ceph-mon[33266]: pgmap v314: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-04-25T18:57:26.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:26 smithi161 ceph-mon[32609]: pgmap v314: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-04-25T18:57:26.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:26 smithi112 ceph-mon[27864]: pgmap v314: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-04-25T18:57:28.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T18:57:28.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.329+0000 7f984a400640 1 -- 172.21.15.112:0/4270287374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98441092d0 msgr2=0x7f9844110ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.329+0000 7f984a400640 1 --2- 172.21.15.112:0/4270287374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98441092d0 0x7f9844110ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f983c009be0 tx=0x7f983c02a6c0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.329+0000 7f984a400640 1 -- 172.21.15.112:0/4270287374 shutdown_connections 2024-04-25T18:57:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.329+0000 7f984a400640 1 --2- 172.21.15.112:0/4270287374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98441092d0 0x7f9844110ea0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.329+0000 7f984a400640 1 --2- 172.21.15.112:0/4270287374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9844104bb0 0x7f9844108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.329+0000 7f984a400640 1 --2- 172.21.15.112:0/4270287374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9844104200 0x7f98441045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.329+0000 7f984a400640 1 -- 172.21.15.112:0/4270287374 >> 172.21.15.112:0/4270287374 conn(0x7f98440ffa60 msgr2=0x7f9844101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.329+0000 7f984a400640 1 -- 172.21.15.112:0/4270287374 shutdown_connections 2024-04-25T18:57:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.329+0000 7f984a400640 1 -- 172.21.15.112:0/4270287374 wait complete. 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.330+0000 7f984a400640 1 Processor -- start 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f984a400640 1 -- start start 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f984a400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9844104200 0x7f98441a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f984a400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98441092d0 0x7f98441a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f984a400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984419a5a0 0x7f984419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f984a400640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9844114240 con 0x7f984419a5a0 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f984a400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f98441140c0 con 0x7f98441092d0 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f984a400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f98441143c0 con 0x7f9844104200 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f9849000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9844104200 0x7f98441a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f9849000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9844104200 0x7f98441a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:51474/0 (socket says 172.21.15.112:51474) 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f9849000640 1 -- 172.21.15.112:0/63111309 learned_addr learned my addr 172.21.15.112:0/63111309 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f9849a00640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984419a5a0 0x7f984419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.331+0000 7f9843e00640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98441092d0 0x7f98441a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.332+0000 7f9849000640 1 -- 172.21.15.112:0/63111309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98441092d0 msgr2=0x7f98441a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.332+0000 7f9849000640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98441092d0 0x7f98441a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.332+0000 7f9849000640 1 -- 172.21.15.112:0/63111309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984419a5a0 msgr2=0x7f984419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.332+0000 7f9849000640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984419a5a0 0x7f984419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.332+0000 7f9849000640 1 -- 172.21.15.112:0/63111309 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984419b0c0 con 0x7f9844104200 2024-04-25T18:57:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.332+0000 7f9849000640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9844104200 0x7f98441a03f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f983800e9f0 tx=0x7f983800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.332+0000 7f9841600640 1 -- 172.21.15.112:0/63111309 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983800cd80 con 0x7f9844104200 2024-04-25T18:57:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.332+0000 7f984a400640 1 -- 172.21.15.112:0/63111309 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984419b3b0 con 0x7f9844104200 2024-04-25T18:57:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.332+0000 7f9841600640 1 -- 172.21.15.112:0/63111309 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98380054d0 con 0x7f9844104200 2024-04-25T18:57:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.332+0000 7f9841600640 1 -- 172.21.15.112:0/63111309 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838010430 con 0x7f9844104200 2024-04-25T18:57:28.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.332+0000 7f984a400640 1 -- 172.21.15.112:0/63111309 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441a7600 con 0x7f9844104200 2024-04-25T18:57:28.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.334+0000 7f984a400640 1 -- 172.21.15.112:0/63111309 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9844050530 con 0x7f9844104200 2024-04-25T18:57:28.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.336+0000 7f9841600640 1 -- 172.21.15.112:0/63111309 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f98380046c0 con 0x7f9844104200 2024-04-25T18:57:28.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.337+0000 7f9841600640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f982c0a3d90 0x7f982c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.338+0000 7f9841600640 1 -- 172.21.15.112:0/63111309 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8571+0+0 (secure 0 0 0) 0x7f98380d0ba0 con 0x7f9844104200 2024-04-25T18:57:28.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.338+0000 7f9843e00640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f982c0a3d90 0x7f982c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.339+0000 7f9843e00640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f982c0a3d90 0x7f982c0a6250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f984410a820 tx=0x7f9830009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:28.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.342+0000 7f9841600640 1 -- 172.21.15.112:0/63111309 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f983809d210 con 0x7f9844104200 2024-04-25T18:57:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:28 smithi171 ceph-mon[33266]: pgmap v315: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T18:57:28.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:28 smithi161 ceph-mon[32609]: pgmap v315: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T18:57:28.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:28 smithi112 ceph-mon[27864]: pgmap v315: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T18:57:28.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.567+0000 7f984a400640 1 -- 172.21.15.112:0/63111309 --> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9844104bb0 con 0x7f982c0a3d90 2024-04-25T18:57:28.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.583+0000 7f9841600640 1 -- 172.21.15.112:0/63111309 <== mgr.14922 v2:172.21.15.112:6800/2236031982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f9844104bb0 con 0x7f982c0a3d90 2024-04-25T18:57:28.585 INFO:teuthology.orchestra.run.smithi112.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T18:47:26", "first": "2024-04-25T18:48:00", "last": "2024-04-25T18:57:00", "last_pruned": "2024-04-25T18:57:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T18:57:28.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 -- 172.21.15.112:0/63111309 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f982c0a3d90 msgr2=0x7f982c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f982c0a3d90 0x7f982c0a6250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f984410a820 tx=0x7f9830009260 comp rx=0 tx=0).stop 2024-04-25T18:57:28.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 -- 172.21.15.112:0/63111309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9844104200 msgr2=0x7f98441a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9844104200 0x7f98441a03f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f983800e9f0 tx=0x7f983800edf0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 -- 172.21.15.112:0/63111309 shutdown_connections 2024-04-25T18:57:28.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f982c0a3d90 0x7f982c0a6250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984419a5a0 0x7f984419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98441092d0 0x7f98441a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 --2- 172.21.15.112:0/63111309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9844104200 0x7f98441a03f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 -- 172.21.15.112:0/63111309 >> 172.21.15.112:0/63111309 conn(0x7f98440ffa60 msgr2=0x7f9844101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:28.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 -- 172.21.15.112:0/63111309 shutdown_connections 2024-04-25T18:57:28.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.586+0000 7f984a400640 1 -- 172.21.15.112:0/63111309 wait complete. 2024-04-25T18:57:28.606 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T18:57:28.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.805+0000 7f6caf200640 1 -- 172.21.15.112:0/2113457576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cb0108e10 msgr2=0x7f6cb0115250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.805+0000 7f6caf200640 1 --2- 172.21.15.112:0/2113457576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cb0108e10 0x7f6cb0115250 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009cd0 tx=0x7f6ca802a8f0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.805+0000 7f6caf200640 1 -- 172.21.15.112:0/2113457576 shutdown_connections 2024-04-25T18:57:28.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.805+0000 7f6caf200640 1 --2- 172.21.15.112:0/2113457576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cb0108e10 0x7f6cb0115250 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.805+0000 7f6caf200640 1 --2- 172.21.15.112:0/2113457576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cb0108490 0x7f6cb01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.805+0000 7f6caf200640 1 --2- 172.21.15.112:0/2113457576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb010e420 0x7f6cb010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.805+0000 7f6caf200640 1 -- 172.21.15.112:0/2113457576 >> 172.21.15.112:0/2113457576 conn(0x7f6cb0077d30 msgr2=0x7f6cb0106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:28.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.805+0000 7f6caf200640 1 -- 172.21.15.112:0/2113457576 shutdown_connections 2024-04-25T18:57:28.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.805+0000 7f6caf200640 1 -- 172.21.15.112:0/2113457576 wait complete. 2024-04-25T18:57:28.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.806+0000 7f6caf200640 1 Processor -- start 2024-04-25T18:57:28.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.807+0000 7f6caf200640 1 -- start start 2024-04-25T18:57:28.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.807+0000 7f6caf200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cb0108490 0x7f6cb01a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.807+0000 7f6caf200640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb0108e10 0x7f6cb01a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.807+0000 7f6caf200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cb010e420 0x7f6cb019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.807+0000 7f6caf200640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0118600 con 0x7f6cb0108e10 2024-04-25T18:57:28.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.807+0000 7f6caf200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0118480 con 0x7f6cb0108490 2024-04-25T18:57:28.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.807+0000 7f6caf200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0118780 con 0x7f6cb010e420 2024-04-25T18:57:28.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.807+0000 7f6cad400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb0108e10 0x7f6cb01a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.808+0000 7f6cade00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cb0108490 0x7f6cb01a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.808+0000 7f6cad400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb0108e10 0x7f6cb01a4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59696/0 (socket says 172.21.15.112:59696) 2024-04-25T18:57:28.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.808+0000 7f6cad400640 1 -- 172.21.15.112:0/2464888786 learned_addr learned my addr 172.21.15.112:0/2464888786 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:57:28.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.808+0000 7f6cae800640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cb010e420 0x7f6cb019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.808+0000 7f6cad400640 1 -- 172.21.15.112:0/2464888786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cb010e420 msgr2=0x7f6cb019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.808+0000 7f6cad400640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cb010e420 0x7f6cb019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.808+0000 7f6cad400640 1 -- 172.21.15.112:0/2464888786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cb0108490 msgr2=0x7f6cb01a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.808+0000 7f6cad400640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cb0108490 0x7f6cb01a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.808+0000 7f6cad400640 1 -- 172.21.15.112:0/2464888786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb019f100 con 0x7f6cb0108e10 2024-04-25T18:57:28.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.808+0000 7f6cade00640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cb0108490 0x7f6cb01a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:28.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.808+0000 7f6cad400640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb0108e10 0x7f6cb01a4d10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000e9f0 tx=0x7f6ca000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:28.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.809+0000 7f6ca6a00640 1 -- 172.21.15.112:0/2464888786 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000cd80 con 0x7f6cb0108e10 2024-04-25T18:57:28.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.809+0000 7f6ca6a00640 1 -- 172.21.15.112:0/2464888786 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca00054d0 con 0x7f6cb0108e10 2024-04-25T18:57:28.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.809+0000 7f6caf200640 1 -- 172.21.15.112:0/2464888786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb019f3f0 con 0x7f6cb0108e10 2024-04-25T18:57:28.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.809+0000 7f6ca6a00640 1 -- 172.21.15.112:0/2464888786 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0010690 con 0x7f6cb0108e10 2024-04-25T18:57:28.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.809+0000 7f6caf200640 1 -- 172.21.15.112:0/2464888786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb01ab690 con 0x7f6cb0108e10 2024-04-25T18:57:28.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.810+0000 7f6caf200640 1 -- 172.21.15.112:0/2464888786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb0050530 con 0x7f6cb0108e10 2024-04-25T18:57:28.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.812+0000 7f6ca6a00640 1 -- 172.21.15.112:0/2464888786 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f6ca0004e90 con 0x7f6cb0108e10 2024-04-25T18:57:28.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.813+0000 7f6ca6a00640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c940a3d10 0x7f6c940a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.814+0000 7f6ca6a00640 1 -- 172.21.15.112:0/2464888786 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8571+0+0 (secure 0 0 0) 0x7f6ca00d0c80 con 0x7f6cb0108e10 2024-04-25T18:57:28.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.814+0000 7f6cade00640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c940a3d10 0x7f6c940a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.814+0000 7f6cade00640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c940a3d10 0x7f6c940a61d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6cb019fac0 tx=0x7f6c98005f90 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:28.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:28.818+0000 7f6ca6a00640 1 -- 172.21.15.112:0/2464888786 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ca009d290 con 0x7f6cb0108e10 2024-04-25T18:57:29.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.044+0000 7f6caf200640 1 -- 172.21.15.112:0/2464888786 --> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f6cb0062390 con 0x7f6c940a3d10 2024-04-25T18:57:29.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.059+0000 7f6ca6a00640 1 -- 172.21.15.112:0/2464888786 <== mgr.14922 v2:172.21.15.112:6800/2236031982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f6cb0062390 con 0x7f6c940a3d10 2024-04-25T18:57:29.061 INFO:teuthology.orchestra.run.smithi112.stdout:/ 1m 6m3h 2024-04-25T18:57:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 -- 172.21.15.112:0/2464888786 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c940a3d10 msgr2=0x7f6c940a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c940a3d10 0x7f6c940a61d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6cb019fac0 tx=0x7f6c98005f90 comp rx=0 tx=0).stop 2024-04-25T18:57:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 -- 172.21.15.112:0/2464888786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb0108e10 msgr2=0x7f6cb01a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb0108e10 0x7f6cb01a4d10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000e9f0 tx=0x7f6ca000edf0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 -- 172.21.15.112:0/2464888786 shutdown_connections 2024-04-25T18:57:29.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f6c940a3d10 0x7f6c940a61d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6cb010e420 0x7f6cb019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb0108e10 0x7f6cb01a4d10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 --2- 172.21.15.112:0/2464888786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cb0108490 0x7f6cb01a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 -- 172.21.15.112:0/2464888786 >> 172.21.15.112:0/2464888786 conn(0x7f6cb0077d30 msgr2=0x7f6cb0078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 -- 172.21.15.112:0/2464888786 shutdown_connections 2024-04-25T18:57:29.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.062+0000 7f6caf200640 1 -- 172.21.15.112:0/2464888786 wait complete. 2024-04-25T18:57:29.083 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T18:57:29.097 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T18:57:29.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.296+0000 7ff4ca400640 1 -- 172.21.15.112:0/1799419974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c41041b0 msgr2=0x7ff4c4104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.296+0000 7ff4ca400640 1 --2- 172.21.15.112:0/1799419974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c41041b0 0x7ff4c4104590 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009860 tx=0x7ff4b002a340 comp rx=0 tx=0).stop 2024-04-25T18:57:29.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.296+0000 7ff4ca400640 1 -- 172.21.15.112:0/1799419974 shutdown_connections 2024-04-25T18:57:29.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.296+0000 7ff4ca400640 1 --2- 172.21.15.112:0/1799419974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4109280 0x7ff4c4110e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.296+0000 7ff4ca400640 1 --2- 172.21.15.112:0/1799419974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c4104b60 0x7ff4c4108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.296+0000 7ff4ca400640 1 --2- 172.21.15.112:0/1799419974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c41041b0 0x7ff4c4104590 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.296+0000 7ff4ca400640 1 -- 172.21.15.112:0/1799419974 >> 172.21.15.112:0/1799419974 conn(0x7ff4c40ff9f0 msgr2=0x7ff4c4101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.296+0000 7ff4ca400640 1 -- 172.21.15.112:0/1799419974 shutdown_connections 2024-04-25T18:57:29.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.296+0000 7ff4ca400640 1 -- 172.21.15.112:0/1799419974 wait complete. 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.297+0000 7ff4ca400640 1 Processor -- start 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.297+0000 7ff4ca400640 1 -- start start 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4ca400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c41041b0 0x7ff4c41a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4ca400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4109280 0x7ff4c41a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4ca400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c419a5e0 0x7ff4c419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4ca400640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41141c0 con 0x7ff4c4109280 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4ca400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4114040 con 0x7ff4c419a5e0 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4ca400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4114340 con 0x7ff4c41041b0 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4c3e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4109280 0x7ff4c41a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4c9000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c41041b0 0x7ff4c41a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4c3e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4109280 0x7ff4c41a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59724/0 (socket says 172.21.15.112:59724) 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4c3e00640 1 -- 172.21.15.112:0/3064214308 learned_addr learned my addr 172.21.15.112:0/3064214308 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:57:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4c9a00640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c419a5e0 0x7ff4c419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4c9000640 1 -- 172.21.15.112:0/3064214308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c419a5e0 msgr2=0x7ff4c419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.298+0000 7ff4c9000640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c419a5e0 0x7ff4c419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.299+0000 7ff4c9000640 1 -- 172.21.15.112:0/3064214308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4109280 msgr2=0x7ff4c41a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.299+0000 7ff4c9000640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4109280 0x7ff4c41a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.299+0000 7ff4c9000640 1 -- 172.21.15.112:0/3064214308 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c419b100 con 0x7ff4c41041b0 2024-04-25T18:57:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.299+0000 7ff4c3e00640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4109280 0x7ff4c41a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.299+0000 7ff4c9000640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c41041b0 0x7ff4c41a03e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0035040 tx=0x7ff4b0032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.299+0000 7ff4c1600640 1 -- 172.21.15.112:0/3064214308 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0037040 con 0x7ff4c41041b0 2024-04-25T18:57:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.299+0000 7ff4ca400640 1 -- 172.21.15.112:0/3064214308 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c419b390 con 0x7ff4c41041b0 2024-04-25T18:57:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.299+0000 7ff4c1600640 1 -- 172.21.15.112:0/3064214308 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b0030c10 con 0x7ff4c41041b0 2024-04-25T18:57:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.299+0000 7ff4c1600640 1 -- 172.21.15.112:0/3064214308 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00035f0 con 0x7ff4c41041b0 2024-04-25T18:57:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.299+0000 7ff4ca400640 1 -- 172.21.15.112:0/3064214308 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c41a7530 con 0x7ff4c41041b0 2024-04-25T18:57:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.300+0000 7ff4ca400640 1 -- 172.21.15.112:0/3064214308 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c40690e0 con 0x7ff4c41041b0 2024-04-25T18:57:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.303+0000 7ff4c1600640 1 -- 172.21.15.112:0/3064214308 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7ff4b0043020 con 0x7ff4c41041b0 2024-04-25T18:57:29.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.305+0000 7ff4c1600640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff4ac0a3da0 0x7ff4ac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:29.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.305+0000 7ff4c1600640 1 -- 172.21.15.112:0/3064214308 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8571+0+0 (secure 0 0 0) 0x7ff4b00ef9a0 con 0x7ff4c41041b0 2024-04-25T18:57:29.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.305+0000 7ff4c3e00640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff4ac0a3da0 0x7ff4ac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:29.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.305+0000 7ff4c3e00640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff4ac0a3da0 0x7ff4ac0a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff4c410a860 tx=0x7ff4b8005f90 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:29.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.307+0000 7ff4c1600640 1 -- 172.21.15.112:0/3064214308 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4b004a060 con 0x7ff4c41041b0 2024-04-25T18:57:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.511+0000 7ff4ca400640 1 -- 172.21.15.112:0/3064214308 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4c410c4b0 con 0x7ff4c41041b0 2024-04-25T18:57:29.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.513+0000 7ff4c1600640 1 -- 172.21.15.112:0/3064214308 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+23182 (secure 0 0 0) 0x7ff4b00bc1c0 con 0x7ff4c41041b0 2024-04-25T18:57:29.516 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:57:29.516 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":106,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","created":"2024-04-25T18:33:37.379089+0000","modified":"2024-04-25T18:57:13.859217+0000","last_up_change":"2024-04-25T18:42:00.961646+0000","last_in_change":"2024-04-25T18:41:40.647162+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:37:45.106199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:43:11.017975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:43:15.116917+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efb4bc01-e29b-4402-8a7a-3387d1731ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6803","nonce":232254535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6805","nonce":232254535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6809","nonce":232254535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6807","nonce":232254535}]},"public_addr":"172.21.15.112:6803/232254535","cluster_addr":"172.21.15.112:6805/232254535","heartbeat_back_addr":"172.21.15.112:6809/232254535","heartbeat_front_addr":"172.21.15.112:6807/232254535","state":["exists","up"]},{"osd":1,"uuid":"65a83fa2-96a6-4743-8c17-21531f5e885e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6801","nonce":664224744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6803","nonce":664224744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6807","nonce":664224744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6805","nonce":664224744}]},"public_addr":"172.21.15.161:6801/664224744","cluster_addr":"172.21.15.161:6803/664224744","heartbeat_back_addr":"172.21.15.161:6807/664224744","heartbeat_front_addr":"172.21.15.161:6805/664224744","state":["exists","up"]},{"osd":2,"uuid":"7562ec4c-c126-48ec-a51c-56fa461a2e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6801","nonce":345172073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6803","nonce":345172073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6807","nonce":345172073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6805","nonce":345172073}]},"public_addr":"172.21.15.171:6801/345172073","cluster_addr":"172.21.15.171:6803/345172073","heartbeat_back_addr":"172.21.15.171:6807/345172073","heartbeat_front_addr":"172.21.15.171:6805/345172073","state":["exists","up"]},{"osd":3,"uuid":"d7aae8cd-fec7-46d9-8773-25eaaca3cc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6811","nonce":1472407156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6813","nonce":1472407156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6817","nonce":1472407156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6815","nonce":1472407156}]},"public_addr":"172.21.15.112:6811/1472407156","cluster_addr":"172.21.15.112:6813/1472407156","heartbeat_back_addr":"172.21.15.112:6817/1472407156","heartbeat_front_addr":"172.21.15.112:6815/1472407156","state":["exists","up"]},{"osd":4,"uuid":"b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6809","nonce":373204306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6811","nonce":373204306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6815","nonce":373204306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6813","nonce":373204306}]},"public_addr":"172.21.15.161:6809/373204306","cluster_addr":"172.21.15.161:6811/373204306","heartbeat_back_addr":"172.21.15.161:6815/373204306","heartbeat_front_addr":"172.21.15.161:6813/373204306","state":["exists","up"]},{"osd":5,"uuid":"56124d12-bc1f-4cba-8430-938248851776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6809","nonce":2643512313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6811","nonce":2643512313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6815","nonce":2643512313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6813","nonce":2643512313}]},"public_addr":"172.21.15.171:6809/2643512313","cluster_addr":"172.21.15.171:6811/2643512313","heartbeat_back_addr":"172.21.15.171:6815/2643512313","heartbeat_front_addr":"172.21.15.171:6813/2643512313","state":["exists","up"]},{"osd":6,"uuid":"9563a9d3-352b-4686-975c-282ec167248b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6819","nonce":1933654933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6821","nonce":1933654933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6825","nonce":1933654933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6823","nonce":1933654933}]},"public_addr":"172.21.15.112:6819/1933654933","cluster_addr":"172.21.15.112:6821/1933654933","heartbeat_back_addr":"172.21.15.112:6825/1933654933","heartbeat_front_addr":"172.21.15.112:6823/1933654933","state":["exists","up"]},{"osd":7,"uuid":"f2d06e86-6aac-40f4-956e-18fe0b0e3543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6817","nonce":3060201138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6819","nonce":3060201138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6823","nonce":3060201138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6821","nonce":3060201138}]},"public_addr":"172.21.15.161:6817/3060201138","cluster_addr":"172.21.15.161:6819/3060201138","heartbeat_back_addr":"172.21.15.161:6823/3060201138","heartbeat_front_addr":"172.21.15.161:6821/3060201138","state":["exists","up"]},{"osd":8,"uuid":"f006e45f-2bb9-45d0-9020-be7c33c6a825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6817","nonce":835756571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6819","nonce":835756571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6823","nonce":835756571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6821","nonce":835756571}]},"public_addr":"172.21.15.171:6817/835756571","cluster_addr":"172.21.15.171:6819/835756571","heartbeat_back_addr":"172.21.15.171:6823/835756571","heartbeat_front_addr":"172.21.15.171:6821/835756571","state":["exists","up"]},{"osd":9,"uuid":"c5a96491-4151-4633-b3bc-2c214fe51c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6827","nonce":349758528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6829","nonce":349758528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6833","nonce":349758528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6831","nonce":349758528}]},"public_addr":"172.21.15.112:6827/349758528","cluster_addr":"172.21.15.112:6829/349758528","heartbeat_back_addr":"172.21.15.112:6833/349758528","heartbeat_front_addr":"172.21.15.112:6831/349758528","state":["exists","up"]},{"osd":10,"uuid":"244eda3e-25a0-490e-879f-c1e2726c1be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6825","nonce":3013292765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6827","nonce":3013292765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6831","nonce":3013292765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6829","nonce":3013292765}]},"public_addr":"172.21.15.161:6825/3013292765","cluster_addr":"172.21.15.161:6827/3013292765","heartbeat_back_addr":"172.21.15.161:6831/3013292765","heartbeat_front_addr":"172.21.15.161:6829/3013292765","state":["exists","up"]},{"osd":11,"uuid":"356e5c54-bbc3-45ee-bda2-85164ce3cb92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6825","nonce":1010336458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6827","nonce":1010336458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6831","nonce":1010336458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6829","nonce":1010336458}]},"public_addr":"172.21.15.171:6825/1010336458","cluster_addr":"172.21.15.171:6827/1010336458","heartbeat_back_addr":"172.21.15.171:6831/1010336458","heartbeat_front_addr":"172.21.15.171:6829/1010336458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:36:51.024650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:17.026638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:41.239726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:15.148968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:41.029118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:06.225858+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:41.639444+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:07.753400+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:32.292085+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:07.159926+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:33.749782+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:58.594447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/2665612708":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/698167759":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/722306877":"2024-04-26T18:47:24.619297+0000","172.21.15.112:6801/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/181862246":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/3393454453":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6800/605196912":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/3195850527":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/2936464788":"2024-04-26T18:47:24.619297+0000","172.21.15.112:6801/3468460974":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3392048544":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6801/605196912":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/1329430933":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/735578423":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/3468460974":"2024-04-26T18:34:20.605775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:57:29.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 -- 172.21.15.112:0/3064214308 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff4ac0a3da0 msgr2=0x7ff4ac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff4ac0a3da0 0x7ff4ac0a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff4c410a860 tx=0x7ff4b8005f90 comp rx=0 tx=0).stop 2024-04-25T18:57:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 -- 172.21.15.112:0/3064214308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c41041b0 msgr2=0x7ff4c41a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c41041b0 0x7ff4c41a03e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0035040 tx=0x7ff4b0032d50 comp rx=0 tx=0).stop 2024-04-25T18:57:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 -- 172.21.15.112:0/3064214308 shutdown_connections 2024-04-25T18:57:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff4ac0a3da0 0x7ff4ac0a6260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4c419a5e0 0x7ff4c419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c4109280 0x7ff4c41a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 --2- 172.21.15.112:0/3064214308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c41041b0 0x7ff4c41a03e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 -- 172.21.15.112:0/3064214308 >> 172.21.15.112:0/3064214308 conn(0x7ff4c40ff9f0 msgr2=0x7ff4c4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:29.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 -- 172.21.15.112:0/3064214308 shutdown_connections 2024-04-25T18:57:29.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.516+0000 7ff4ca400640 1 -- 172.21.15.112:0/3064214308 wait complete. 2024-04-25T18:57:29.538 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T18:37:45.106199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T18:43:11.017975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T18:43:15.116917+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T18:57:29.538 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T18:57:29.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.742+0000 7fd58ca00640 1 -- 172.21.15.112:0/464049780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd588104d50 msgr2=0x7fd588108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.742+0000 7fd58ca00640 1 --2- 172.21.15.112:0/464049780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd588104d50 0x7fd588108d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd570009900 tx=0x7fd57002a3f0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.742+0000 7fd58ca00640 1 -- 172.21.15.112:0/464049780 shutdown_connections 2024-04-25T18:57:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.742+0000 7fd58ca00640 1 --2- 172.21.15.112:0/464049780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd588109470 0x7fd588111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.742+0000 7fd58ca00640 1 --2- 172.21.15.112:0/464049780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd588104d50 0x7fd588108d90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.742+0000 7fd58ca00640 1 --2- 172.21.15.112:0/464049780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5881043a0 0x7fd588104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.742+0000 7fd58ca00640 1 -- 172.21.15.112:0/464049780 >> 172.21.15.112:0/464049780 conn(0x7fd5880ffbc0 msgr2=0x7fd588101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.742+0000 7fd58ca00640 1 -- 172.21.15.112:0/464049780 shutdown_connections 2024-04-25T18:57:29.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.742+0000 7fd58ca00640 1 -- 172.21.15.112:0/464049780 wait complete. 2024-04-25T18:57:29.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.743+0000 7fd58ca00640 1 Processor -- start 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.743+0000 7fd58ca00640 1 -- start start 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.744+0000 7fd58ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5881043a0 0x7fd5881a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.744+0000 7fd58ca00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd588109470 0x7fd5881a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.744+0000 7fd58ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd58819a7e0 0x7fd58819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.744+0000 7fd58ca00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd588114400 con 0x7fd588109470 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.744+0000 7fd58ca00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd588114280 con 0x7fd5881043a0 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.744+0000 7fd58ca00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd588114580 con 0x7fd58819a7e0 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.744+0000 7fd57fe00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd588109470 0x7fd5881a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.744+0000 7fd585e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5881043a0 0x7fd5881a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.744+0000 7fd57fe00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd588109470 0x7fd5881a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59732/0 (socket says 172.21.15.112:59732) 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.744+0000 7fd57fe00640 1 -- 172.21.15.112:0/2094614900 learned_addr learned my addr 172.21.15.112:0/2094614900 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.744+0000 7fd586800640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd58819a7e0 0x7fd58819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:29.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd57fe00640 1 -- 172.21.15.112:0/2094614900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd58819a7e0 msgr2=0x7fd58819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd57fe00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd58819a7e0 0x7fd58819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd57fe00640 1 -- 172.21.15.112:0/2094614900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5881043a0 msgr2=0x7fd5881a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd57fe00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5881043a0 0x7fd5881a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd57fe00640 1 -- 172.21.15.112:0/2094614900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58819b370 con 0x7fd588109470 2024-04-25T18:57:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd585e00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5881043a0 0x7fd5881a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd57fe00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd588109470 0x7fd5881a0bb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd570005ab0 tx=0x7fd570003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd57ea00640 1 -- 172.21.15.112:0/2094614900 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57003f070 con 0x7fd588109470 2024-04-25T18:57:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd58ca00640 1 -- 172.21.15.112:0/2094614900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58819b600 con 0x7fd588109470 2024-04-25T18:57:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd57ea00640 1 -- 172.21.15.112:0/2094614900 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd570031b50 con 0x7fd588109470 2024-04-25T18:57:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd57ea00640 1 -- 172.21.15.112:0/2094614900 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570035030 con 0x7fd588109470 2024-04-25T18:57:29.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.745+0000 7fd58ca00640 1 -- 172.21.15.112:0/2094614900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5881a77b0 con 0x7fd588109470 2024-04-25T18:57:29.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.746+0000 7fd58ca00640 1 -- 172.21.15.112:0/2094614900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd588105870 con 0x7fd588109470 2024-04-25T18:57:29.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.748+0000 7fd57ea00640 1 -- 172.21.15.112:0/2094614900 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fd570042070 con 0x7fd588109470 2024-04-25T18:57:29.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.751+0000 7fd57ea00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd56c0a3d20 0x7fd56c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:29.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.751+0000 7fd57ea00640 1 -- 172.21.15.112:0/2094614900 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8571+0+0 (secure 0 0 0) 0x7fd5700f0a60 con 0x7fd588109470 2024-04-25T18:57:29.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.751+0000 7fd585e00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd56c0a3d20 0x7fd56c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:29.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.752+0000 7fd585e00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd56c0a3d20 0x7fd56c0a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd578009fc0 tx=0x7fd578005f90 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:29.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.756+0000 7fd57ea00640 1 -- 172.21.15.112:0/2094614900 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5700bca20 con 0x7fd588109470 2024-04-25T18:57:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.958+0000 7fd58ca00640 1 -- 172.21.15.112:0/2094614900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd58810c6a0 con 0x7fd588109470 2024-04-25T18:57:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.959+0000 7fd57ea00640 1 -- 172.21.15.112:0/2094614900 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7fd5700c0d20 con 0x7fd588109470 2024-04-25T18:57:29.961 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T18:57:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.961+0000 7fd58ca00640 1 -- 172.21.15.112:0/2094614900 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd56c0a3d20 msgr2=0x7fd56c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.961+0000 7fd58ca00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd56c0a3d20 0x7fd56c0a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd578009fc0 tx=0x7fd578005f90 comp rx=0 tx=0).stop 2024-04-25T18:57:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.961+0000 7fd58ca00640 1 -- 172.21.15.112:0/2094614900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd588109470 msgr2=0x7fd5881a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.962+0000 7fd58ca00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd588109470 0x7fd5881a0bb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd570005ab0 tx=0x7fd570003b90 comp rx=0 tx=0).stop 2024-04-25T18:57:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.962+0000 7fd58ca00640 1 -- 172.21.15.112:0/2094614900 shutdown_connections 2024-04-25T18:57:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.962+0000 7fd58ca00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fd56c0a3d20 0x7fd56c0a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.962+0000 7fd58ca00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd58819a7e0 0x7fd58819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.962+0000 7fd58ca00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd588109470 0x7fd5881a0bb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.962+0000 7fd58ca00640 1 --2- 172.21.15.112:0/2094614900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5881043a0 0x7fd5881a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.962+0000 7fd58ca00640 1 -- 172.21.15.112:0/2094614900 >> 172.21.15.112:0/2094614900 conn(0x7fd5880ffbc0 msgr2=0x7fd588100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.962+0000 7fd58ca00640 1 -- 172.21.15.112:0/2094614900 shutdown_connections 2024-04-25T18:57:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:29.962+0000 7fd58ca00640 1 -- 172.21.15.112:0/2094614900 wait complete. 2024-04-25T18:57:29.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T18:57:30.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.193+0000 7eff2ae00640 1 -- 172.21.15.112:0/2597975387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff24076480 msgr2=0x7eff24076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.193+0000 7eff2ae00640 1 --2- 172.21.15.112:0/2597975387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff24076480 0x7eff24076860 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7eff14009900 tx=0x7eff1402a3f0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.193+0000 7eff2ae00640 1 -- 172.21.15.112:0/2597975387 shutdown_connections 2024-04-25T18:57:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.193+0000 7eff2ae00640 1 --2- 172.21.15.112:0/2597975387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff24112dd0 0x7eff241151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.193+0000 7eff2ae00640 1 --2- 172.21.15.112:0/2597975387 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff24076da0 0x7eff24112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.193+0000 7eff2ae00640 1 --2- 172.21.15.112:0/2597975387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff24076480 0x7eff24076860 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.193+0000 7eff2ae00640 1 -- 172.21.15.112:0/2597975387 >> 172.21.15.112:0/2597975387 conn(0x7eff240ffbc0 msgr2=0x7eff24101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.193+0000 7eff2ae00640 1 -- 172.21.15.112:0/2597975387 shutdown_connections 2024-04-25T18:57:30.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.193+0000 7eff2ae00640 1 -- 172.21.15.112:0/2597975387 wait complete. 2024-04-25T18:57:30.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.194+0000 7eff2ae00640 1 Processor -- start 2024-04-25T18:57:30.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.194+0000 7eff2ae00640 1 -- start start 2024-04-25T18:57:30.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff2ae00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff24076480 0x7eff2419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff2ae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff24076da0 0x7eff2419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff2ae00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff24112dd0 0x7eff241a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff2ae00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff24118820 con 0x7eff24076480 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff2ae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff241186a0 con 0x7eff24076da0 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff2ae00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7eff241189a0 con 0x7eff24112dd0 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff23e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff24076480 0x7eff2419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff23e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff24076480 0x7eff2419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44882/0 (socket says 172.21.15.112:44882) 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff23e00640 1 -- 172.21.15.112:0/1702627940 learned_addr learned my addr 172.21.15.112:0/1702627940 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff23400640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff24076da0 0x7eff2419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff28e00640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff24112dd0 0x7eff241a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff23e00640 1 -- 172.21.15.112:0/1702627940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff24112dd0 msgr2=0x7eff241a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff23e00640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff24112dd0 0x7eff241a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff23e00640 1 -- 172.21.15.112:0/1702627940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff24076da0 msgr2=0x7eff2419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff23e00640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff24076da0 0x7eff2419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.195+0000 7eff23e00640 1 -- 172.21.15.112:0/1702627940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff241a3db0 con 0x7eff24076480 2024-04-25T18:57:30.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.196+0000 7eff23e00640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff24076480 0x7eff2419edc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7eff1402a9e0 tx=0x7eff14003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:30.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.196+0000 7eff20c00640 1 -- 172.21.15.112:0/1702627940 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1403f070 con 0x7eff24076480 2024-04-25T18:57:30.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.196+0000 7eff20c00640 1 -- 172.21.15.112:0/1702627940 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff1403bb20 con 0x7eff24076480 2024-04-25T18:57:30.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.196+0000 7eff2ae00640 1 -- 172.21.15.112:0/1702627940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff241a4040 con 0x7eff24076480 2024-04-25T18:57:30.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.196+0000 7eff20c00640 1 -- 172.21.15.112:0/1702627940 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14035030 con 0x7eff24076480 2024-04-25T18:57:30.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.196+0000 7eff2ae00640 1 -- 172.21.15.112:0/1702627940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff241ab930 con 0x7eff24076480 2024-04-25T18:57:30.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.197+0000 7eff2ae00640 1 -- 172.21.15.112:0/1702627940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff24068d70 con 0x7eff24076480 2024-04-25T18:57:30.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.199+0000 7eff20c00640 1 -- 172.21.15.112:0/1702627940 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7eff14042070 con 0x7eff24076480 2024-04-25T18:57:30.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.201+0000 7eff20c00640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7eff080a3d20 0x7eff080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.201+0000 7eff20c00640 1 -- 172.21.15.112:0/1702627940 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8571+0+0 (secure 0 0 0) 0x7eff140f0b90 con 0x7eff24076480 2024-04-25T18:57:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.201+0000 7eff23400640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7eff080a3d20 0x7eff080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.202+0000 7eff23400640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7eff080a3d20 0x7eff080a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7eff240790d0 tx=0x7eff0c00b410 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:30.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.205+0000 7eff20c00640 1 -- 172.21.15.112:0/1702627940 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff140bd200 con 0x7eff24076480 2024-04-25T18:57:30.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.406+0000 7eff2ae00640 1 -- 172.21.15.112:0/1702627940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7eff2410df60 con 0x7eff24076480 2024-04-25T18:57:30.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.407+0000 7eff20c00640 1 -- 172.21.15.112:0/1702627940 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7eff140bcb00 con 0x7eff24076480 2024-04-25T18:57:30.409 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T18:57:30.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.410+0000 7eff2ae00640 1 -- 172.21.15.112:0/1702627940 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7eff080a3d20 msgr2=0x7eff080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.410+0000 7eff2ae00640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7eff080a3d20 0x7eff080a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7eff240790d0 tx=0x7eff0c00b410 comp rx=0 tx=0).stop 2024-04-25T18:57:30.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.410+0000 7eff2ae00640 1 -- 172.21.15.112:0/1702627940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff24076480 msgr2=0x7eff2419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.410+0000 7eff2ae00640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff24076480 0x7eff2419edc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7eff1402a9e0 tx=0x7eff14003c10 comp rx=0 tx=0).stop 2024-04-25T18:57:30.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.410+0000 7eff2ae00640 1 -- 172.21.15.112:0/1702627940 shutdown_connections 2024-04-25T18:57:30.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.410+0000 7eff2ae00640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7eff080a3d20 0x7eff080a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.410+0000 7eff2ae00640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff24112dd0 0x7eff241a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.410+0000 7eff2ae00640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff24076da0 0x7eff2419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.410+0000 7eff2ae00640 1 --2- 172.21.15.112:0/1702627940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff24076480 0x7eff2419edc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.410+0000 7eff2ae00640 1 -- 172.21.15.112:0/1702627940 >> 172.21.15.112:0/1702627940 conn(0x7eff240ffbc0 msgr2=0x7eff24101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:30.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.410+0000 7eff2ae00640 1 -- 172.21.15.112:0/1702627940 shutdown_connections 2024-04-25T18:57:30.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.411+0000 7eff2ae00640 1 -- 172.21.15.112:0/1702627940 wait complete. 2024-04-25T18:57:30.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:30 smithi112 ceph-mon[27864]: from='client.26074 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:30.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:30 smithi112 ceph-mon[27864]: pgmap v316: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:30.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:30 smithi112 ceph-mon[27864]: from='client.15906 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:30.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:30 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3064214308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:57:30.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:30 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2094614900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:57:30.432 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T18:57:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:30 smithi171 ceph-mon[33266]: from='client.26074 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:30 smithi171 ceph-mon[33266]: pgmap v316: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:30 smithi171 ceph-mon[33266]: from='client.15906 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:30 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3064214308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:57:30.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:30 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2094614900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:57:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:30 smithi161 ceph-mon[32609]: from='client.26074 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:30 smithi161 ceph-mon[32609]: pgmap v316: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:30 smithi161 ceph-mon[32609]: from='client.15906 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:57:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:30 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3064214308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:57:30.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:30 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2094614900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:57:30.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.637+0000 7fa366a00640 1 -- 172.21.15.112:0/2103807860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360074ca0 msgr2=0x7fa36010e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.637+0000 7fa366a00640 1 --2- 172.21.15.112:0/2103807860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360074ca0 0x7fa36010e0f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009900 tx=0x7fa34c02a480 comp rx=0 tx=0).stop 2024-04-25T18:57:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.637+0000 7fa366a00640 1 -- 172.21.15.112:0/2103807860 shutdown_connections 2024-04-25T18:57:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.637+0000 7fa366a00640 1 --2- 172.21.15.112:0/2103807860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa36010e630 0x7fa360110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.637+0000 7fa366a00640 1 --2- 172.21.15.112:0/2103807860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360074ca0 0x7fa36010e0f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.637+0000 7fa366a00640 1 --2- 172.21.15.112:0/2103807860 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360074380 0x7fa360074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.637+0000 7fa366a00640 1 -- 172.21.15.112:0/2103807860 >> 172.21.15.112:0/2103807860 conn(0x7fa3600fd920 msgr2=0x7fa3600ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.637+0000 7fa366a00640 1 -- 172.21.15.112:0/2103807860 shutdown_connections 2024-04-25T18:57:30.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.637+0000 7fa366a00640 1 -- 172.21.15.112:0/2103807860 wait complete. 2024-04-25T18:57:30.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.638+0000 7fa366a00640 1 Processor -- start 2024-04-25T18:57:30.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa366a00640 1 -- start start 2024-04-25T18:57:30.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa366a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360074380 0x7fa3601a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa366a00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360074ca0 0x7fa3601a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa366a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa36010e630 0x7fa36019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa366a00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601142a0 con 0x7fa360074ca0 2024-04-25T18:57:30.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa366a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa360114120 con 0x7fa36010e630 2024-04-25T18:57:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa366a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa360114420 con 0x7fa360074380 2024-04-25T18:57:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa364c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360074ca0 0x7fa3601a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa364c00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360074ca0 0x7fa3601a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44896/0 (socket says 172.21.15.112:44896) 2024-04-25T18:57:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa364c00640 1 -- 172.21.15.112:0/672739728 learned_addr learned my addr 172.21.15.112:0/672739728 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:57:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa366000640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa36010e630 0x7fa36019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.639+0000 7fa365600640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360074380 0x7fa3601a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.640+0000 7fa364c00640 1 -- 172.21.15.112:0/672739728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360074380 msgr2=0x7fa3601a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.640+0000 7fa364c00640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360074380 0x7fa3601a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.640+0000 7fa364c00640 1 -- 172.21.15.112:0/672739728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa36010e630 msgr2=0x7fa36019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.640+0000 7fa364c00640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa36010e630 0x7fa36019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.640+0000 7fa364c00640 1 -- 172.21.15.112:0/672739728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36019aff0 con 0x7fa360074ca0 2024-04-25T18:57:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.640+0000 7fa366000640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa36010e630 0x7fa36019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.640+0000 7fa364c00640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360074ca0 0x7fa3601a0b60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005ab0 tx=0x7fa34c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.641+0000 7fa35e000640 1 -- 172.21.15.112:0/672739728 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c03f070 con 0x7fa360074ca0 2024-04-25T18:57:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.641+0000 7fa35e000640 1 -- 172.21.15.112:0/672739728 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34c031b60 con 0x7fa360074ca0 2024-04-25T18:57:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.641+0000 7fa366a00640 1 -- 172.21.15.112:0/672739728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36019b280 con 0x7fa360074ca0 2024-04-25T18:57:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.641+0000 7fa35e000640 1 -- 172.21.15.112:0/672739728 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c035030 con 0x7fa360074ca0 2024-04-25T18:57:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.641+0000 7fa366a00640 1 -- 172.21.15.112:0/672739728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3601a75a0 con 0x7fa360074ca0 2024-04-25T18:57:30.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.642+0000 7fa366a00640 1 -- 172.21.15.112:0/672739728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa360075a30 con 0x7fa360074ca0 2024-04-25T18:57:30.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.645+0000 7fa35e000640 1 -- 172.21.15.112:0/672739728 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fa34c042070 con 0x7fa360074ca0 2024-04-25T18:57:30.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.646+0000 7fa35e000640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa3440a3da0 0x7fa3440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.646+0000 7fa35e000640 1 -- 172.21.15.112:0/672739728 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8571+0+0 (secure 0 0 0) 0x7fa34c0f1650 con 0x7fa360074ca0 2024-04-25T18:57:30.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.646+0000 7fa365600640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa3440a3da0 0x7fa3440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.647+0000 7fa365600640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa3440a3da0 0x7fa3440a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa350005fd0 tx=0x7fa350009500 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:30.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.648+0000 7fa35e000640 1 -- 172.21.15.112:0/672739728 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa34c0bd610 con 0x7fa360074ca0 2024-04-25T18:57:30.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.852+0000 7fa366a00640 1 -- 172.21.15.112:0/672739728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa36019bc10 con 0x7fa360074ca0 2024-04-25T18:57:30.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.853+0000 7fa35e000640 1 -- 172.21.15.112:0/672739728 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7fa34c0c1910 con 0x7fa360074ca0 2024-04-25T18:57:30.855 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.855+0000 7fa366a00640 1 -- 172.21.15.112:0/672739728 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa3440a3da0 msgr2=0x7fa3440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.855+0000 7fa366a00640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa3440a3da0 0x7fa3440a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa350005fd0 tx=0x7fa350009500 comp rx=0 tx=0).stop 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.855+0000 7fa366a00640 1 -- 172.21.15.112:0/672739728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360074ca0 msgr2=0x7fa3601a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.855+0000 7fa366a00640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360074ca0 0x7fa3601a0b60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005ab0 tx=0x7fa34c003c10 comp rx=0 tx=0).stop 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.856+0000 7fa366a00640 1 -- 172.21.15.112:0/672739728 shutdown_connections 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.856+0000 7fa366a00640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fa3440a3da0 0x7fa3440a6260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.856+0000 7fa366a00640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa36010e630 0x7fa36019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.856+0000 7fa366a00640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa360074ca0 0x7fa3601a0b60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.856+0000 7fa366a00640 1 --2- 172.21.15.112:0/672739728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa360074380 0x7fa3601a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.856+0000 7fa366a00640 1 -- 172.21.15.112:0/672739728 >> 172.21.15.112:0/672739728 conn(0x7fa3600fd920 msgr2=0x7fa3600fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.856+0000 7fa366a00640 1 -- 172.21.15.112:0/672739728 shutdown_connections 2024-04-25T18:57:30.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:30.856+0000 7fa366a00640 1 -- 172.21.15.112:0/672739728 wait complete. 2024-04-25T18:57:30.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T18:57:31.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.078+0000 7fedf5800640 1 -- 172.21.15.112:0/1076001039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf0108580 msgr2=0x7fedf0108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:31.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.078+0000 7fedf5800640 1 --2- 172.21.15.112:0/1076001039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf0108580 0x7fedf0108a00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009900 tx=0x7fedd802a410 comp rx=0 tx=0).stop 2024-04-25T18:57:31.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.078+0000 7fedf5800640 1 -- 172.21.15.112:0/1076001039 shutdown_connections 2024-04-25T18:57:31.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.078+0000 7fedf5800640 1 --2- 172.21.15.112:0/1076001039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf0108f40 0x7fedf0115400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:31.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.078+0000 7fedf5800640 1 --2- 172.21.15.112:0/1076001039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf0108580 0x7fedf0108a00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:31.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.078+0000 7fedf5800640 1 --2- 172.21.15.112:0/1076001039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf010e640 0x7fedf010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:31.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.078+0000 7fedf5800640 1 -- 172.21.15.112:0/1076001039 >> 172.21.15.112:0/1076001039 conn(0x7fedf0075750 msgr2=0x7fedf0106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:31.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.079+0000 7fedf5800640 1 -- 172.21.15.112:0/1076001039 shutdown_connections 2024-04-25T18:57:31.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.079+0000 7fedf5800640 1 -- 172.21.15.112:0/1076001039 wait complete. 2024-04-25T18:57:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.079+0000 7fedf5800640 1 Processor -- start 2024-04-25T18:57:31.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fedf5800640 1 -- start start 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fedf5800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf0108580 0x7fedf01a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fedf5800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf0108f40 0x7fedf01a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fedf5800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf010e640 0x7fedf019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fedf5800640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0118800 con 0x7fedf0108580 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fedf5800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0118680 con 0x7fedf010e640 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fedf5800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0118980 con 0x7fedf0108f40 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fedeea00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf0108580 0x7fedf01a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fedeea00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf0108580 0x7fedf01a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44916/0 (socket says 172.21.15.112:44916) 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fedeea00640 1 -- 172.21.15.112:0/386959 learned_addr learned my addr 172.21.15.112:0/386959 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fede6000640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf0108f40 0x7fedf01a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.080+0000 7fedef400640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf010e640 0x7fedf019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.081+0000 7fedeea00640 1 -- 172.21.15.112:0/386959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf0108f40 msgr2=0x7fedf01a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.081+0000 7fedeea00640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf0108f40 0x7fedf01a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:31.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.081+0000 7fedeea00640 1 -- 172.21.15.112:0/386959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf010e640 msgr2=0x7fedf019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.081+0000 7fedeea00640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf010e640 0x7fedf019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.081+0000 7fedeea00640 1 -- 172.21.15.112:0/386959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf019f3e0 con 0x7fedf0108580 2024-04-25T18:57:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.081+0000 7fedeea00640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf0108580 0x7fedf01a4a60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fede000e9f0 tx=0x7fede000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.081+0000 7fede7e00640 1 -- 172.21.15.112:0/386959 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede000ccc0 con 0x7fedf0108580 2024-04-25T18:57:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.081+0000 7fede7e00640 1 -- 172.21.15.112:0/386959 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede00108e0 con 0x7fedf0108580 2024-04-25T18:57:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.081+0000 7fede7e00640 1 -- 172.21.15.112:0/386959 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0018880 con 0x7fedf0108580 2024-04-25T18:57:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.081+0000 7fedf5800640 1 -- 172.21.15.112:0/386959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf019f6d0 con 0x7fedf0108580 2024-04-25T18:57:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.081+0000 7fedf5800640 1 -- 172.21.15.112:0/386959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf01ab9e0 con 0x7fedf0108580 2024-04-25T18:57:31.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.083+0000 7fedf5800640 1 -- 172.21.15.112:0/386959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf0109ca0 con 0x7fedf0108580 2024-04-25T18:57:31.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.085+0000 7fede7e00640 1 -- 172.21.15.112:0/386959 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fede0010430 con 0x7fedf0108580 2024-04-25T18:57:31.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.086+0000 7fede7e00640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fedd40a3da0 0x7fedd40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:31.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.086+0000 7fede7e00640 1 -- 172.21.15.112:0/386959 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8571+0+0 (secure 0 0 0) 0x7fede00d0b90 con 0x7fedf0108580 2024-04-25T18:57:31.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.086+0000 7fede6000640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fedd40a3da0 0x7fedd40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:31.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.087+0000 7fede6000640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fedd40a3da0 0x7fedd40a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fedd8000c00 tx=0x7fedd8005e80 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:31.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.090+0000 7fede7e00640 1 -- 172.21.15.112:0/386959 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fede009d1a0 con 0x7fedf0108580 2024-04-25T18:57:31.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.336+0000 7fedf5800640 1 -- 172.21.15.112:0/386959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fedf0110990 con 0x7fedf0108580 2024-04-25T18:57:31.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.367+0000 7fede7e00640 1 -- 172.21.15.112:0/386959 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746071 (secure 0 0 0) 0x7fede009caa0 con 0x7fedf0108580 2024-04-25T18:57:31.381 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:57:31.381 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:57:31.381 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 26, 2024-04-25T18:57:31.381 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.381 INFO:teuthology.orchestra.run.smithi112.stdout: "active_gid": 14922, 2024-04-25T18:57:31.381 INFO:teuthology.orchestra.run.smithi112.stdout: "active_name": "x", 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addrs": { 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6800", 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2236031982 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6801", 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2236031982 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addr": "172.21.15.112:6801/2236031982", 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "active_change": "2024-04-25T18:47:24.619553+0000", 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "available": true, 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "standbys": [ 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 25034, 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "y", 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-04-25T18:57:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-04-25T18:57:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-04-25T18:57:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-04-25T18:57:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-04-25T18:57:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T18:57:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T18:57:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T18:57:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T18:57:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T18:57:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-04-25T18:57:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-04-25T18:57:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T18:57:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-04-25T18:57:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-04-25T18:57:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-04-25T18:57:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T18:57:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T18:57:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-04-25T18:57:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-04-25T18:57:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T18:57:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T18:57:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T18:57:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-04-25T18:57:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.535 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.536 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.537 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-04-25T18:57:31.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.540 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.541 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.542 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.543 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.544 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.545 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.546 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.547 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.549 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.549 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.549 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.549 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.549 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.549 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.550 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.551 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.553 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.554 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.555 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.556 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T18:57:31.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.558 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T18:57:31.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.560 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T18:57:31.561 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.562 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.563 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.564 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.565 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.566 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.567 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T18:57:31.569 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.569 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.569 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.569 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.569 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.569 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.569 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.569 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-04-25T18:57:31.569 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-04-25T18:57:31.569 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.570 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.570 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.570 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-04-25T18:57:31.570 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.570 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.570 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.570 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.570 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.570 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.570 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.571 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-04-25T18:57:31.572 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.574 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.575 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.576 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.577 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.578 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.579 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.580 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.581 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.582 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.583 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.584 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.587 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.589 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.590 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.591 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.592 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.593 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.594 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.595 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.596 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.597 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T18:57:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T18:57:31.599 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.600 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.602 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.603 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.604 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.607 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.608 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.610 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.611 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-04-25T18:57:31.613 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T18:57:31.614 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.615 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.616 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.618 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.619 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.620 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.621 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.622 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.623 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.624 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.626 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.628 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.629 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-04-25T18:57:31.630 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.631 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.632 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.634 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.635 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.638 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.640 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.642 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.646 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.647 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.648 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.649 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.650 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.652 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:31.653 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-04-25T18:57:31.654 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.655 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T18:57:31.656 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.657 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.658 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.659 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.661 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.662 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-04-25T18:57:31.663 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.664 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.665 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.666 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.667 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.668 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.670 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.671 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.672 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.673 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.674 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.675 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T18:57:31.676 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-04-25T18:57:31.678 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T18:57:31.679 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.680 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.681 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.682 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.684 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.685 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.686 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-04-25T18:57:31.687 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-04-25T18:57:31.687 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.687 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.687 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.687 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.687 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.687 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.687 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.687 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.687 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.688 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.690 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.691 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.693 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.694 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T18:57:31.695 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.697 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.698 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.699 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.700 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.701 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.704 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.706 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.707 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.708 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.709 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.711 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.712 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.713 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.714 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.715 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.716 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.720 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-04-25T18:57:31.721 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-04-25T18:57:31.722 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-04-25T18:57:31.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.724 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T18:57:31.725 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.726 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.727 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.730 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.731 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.732 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.733 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.734 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.735 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.736 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.737 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.738 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.740 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.741 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.742 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.743 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.744 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.746 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.747 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.748 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.749 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-04-25T18:57:31.750 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.751 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.752 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.753 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.754 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.755 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.756 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.757 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.758 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.759 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.761 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.762 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.763 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.764 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.765 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.766 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.767 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-04-25T18:57:31.768 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-04-25T18:57:31.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.770 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.771 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.772 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.773 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.774 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.775 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.776 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.777 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.778 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.779 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.781 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.782 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.783 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.785 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.786 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.788 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.789 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.790 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.791 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.793 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.794 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 25039, 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "z", 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.795 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.796 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.797 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.798 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.799 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.800 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.802 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-04-25T18:57:31.803 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.805 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.806 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.807 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.808 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.809 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.810 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T18:57:31.811 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.812 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-04-25T18:57:31.815 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.816 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.817 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.818 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.819 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.820 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.821 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.822 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-04-25T18:57:31.823 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.824 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:31.825 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.826 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-04-25T18:57:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.829 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.830 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.831 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-04-25T18:57:31.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.833 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.834 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-04-25T18:57:31.835 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-04-25T18:57:31.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-04-25T18:57:31.837 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-04-25T18:57:31.838 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-04-25T18:57:31.839 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.841 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.842 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.843 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.844 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-04-25T18:57:31.845 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-04-25T18:57:31.846 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.847 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.848 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.849 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.850 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T18:57:31.851 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.852 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.853 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.854 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.855 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.856 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.857 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.858 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.859 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.860 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.862 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.863 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.864 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.865 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.866 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.867 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T18:57:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T18:57:31.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.871 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.872 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.873 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T18:57:31.874 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.875 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.876 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.879 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-04-25T18:57:31.880 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.881 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.882 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.884 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.885 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.886 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.887 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.888 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.889 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.890 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.891 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.892 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.893 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.894 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T18:57:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.897 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.898 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T18:57:31.899 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T18:57:31.900 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T18:57:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.902 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.903 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.905 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.907 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.908 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.909 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.910 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.911 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.912 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.913 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.914 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T18:57:31.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.916 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.917 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.918 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.919 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.921 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.922 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.923 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.924 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.925 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T18:57:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T18:57:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T18:57:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T18:57:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-04-25T18:57:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T18:57:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-04-25T18:57:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T18:57:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-04-25T18:57:31.991 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.992 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-04-25T18:57:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T18:57:31.995 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.996 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:31.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:31.999 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.000 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.001 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.003 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.004 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.005 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.006 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.007 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.009 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.010 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.011 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-04-25T18:57:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.014 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.015 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.016 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.016 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.016 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.016 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.016 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.016 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.016 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.016 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.016 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.017 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.018 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.019 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.021 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.022 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.025 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.026 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.027 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.028 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.029 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.030 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.031 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.032 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.033 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.034 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.035 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.036 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.037 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.038 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.039 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.040 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.041 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.042 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.043 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.045 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.046 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.047 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-04-25T18:57:32.048 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.049 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.050 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.051 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.052 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.054 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.056 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.057 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.060 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.061 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.062 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.063 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.064 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.066 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.067 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.068 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.069 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.070 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.071 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.072 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.073 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.074 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-04-25T18:57:32.075 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.077 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.078 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.079 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.080 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.081 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.082 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-04-25T18:57:32.083 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.084 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.085 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.086 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-04-25T18:57:32.087 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T18:57:32.088 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.089 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.090 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.091 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.092 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.093 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.094 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.095 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.096 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.097 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.098 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.099 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.100 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.101 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.102 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.103 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.104 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.105 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.106 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.107 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.108 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.109 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.110 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.111 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.112 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.113 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.114 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.115 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.116 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.117 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.118 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.119 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.120 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.122 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-04-25T18:57:32.123 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-04-25T18:57:32.124 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-04-25T18:57:32.125 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-04-25T18:57:32.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.127 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.128 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.129 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.130 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.131 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-04-25T18:57:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-04-25T18:57:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-04-25T18:57:32.151 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.152 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.153 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.154 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.155 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.156 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.157 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.158 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.159 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.160 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.161 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.162 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.163 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.165 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.165 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.165 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.165 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.165 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.165 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-04-25T18:57:32.165 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-04-25T18:57:32.165 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.165 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.166 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.167 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-04-25T18:57:32.168 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.169 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.170 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.171 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.172 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.173 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.174 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.175 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.176 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.177 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.178 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.180 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.182 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.183 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.184 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.185 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.186 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.187 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.189 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-04-25T18:57:32.190 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.191 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T18:57:32.192 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.193 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.194 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.195 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.196 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.197 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.198 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.199 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-04-25T18:57:32.201 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-04-25T18:57:32.202 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.203 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.204 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "modules": [ 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard", 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "iostat", 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "nfs", 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "restful", 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "snap_schedule" 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.205 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.206 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.207 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.208 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.209 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.210 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-04-25T18:57:32.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-04-25T18:57:32.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.214 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.215 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.216 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.217 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.218 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-04-25T18:57:32.219 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-04-25T18:57:32.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:32.221 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:32.222 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.223 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:32.224 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.225 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.226 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.227 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.229 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.230 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.231 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T18:57:32.232 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-04-25T18:57:32.233 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.234 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:32.235 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:32.236 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.237 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-04-25T18:57:32.238 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-04-25T18:57:32.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-04-25T18:57:32.240 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-04-25T18:57:32.241 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-04-25T18:57:32.242 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:32.243 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-04-25T18:57:32.245 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.246 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-04-25T18:57:32.247 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.248 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.249 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.250 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.251 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-04-25T18:57:32.252 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.253 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.254 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.255 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.256 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.257 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T18:57:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T18:57:32.259 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.260 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.261 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.262 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-04-25T18:57:32.263 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.264 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.265 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.266 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.268 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-04-25T18:57:32.269 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-04-25T18:57:32.270 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-04-25T18:57:32.271 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-04-25T18:57:32.272 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.273 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.274 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.277 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.278 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.279 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.280 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.281 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.282 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T18:57:32.284 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.287 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.288 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.289 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.290 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-04-25T18:57:32.291 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-04-25T18:57:32.292 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-04-25T18:57:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-04-25T18:57:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.295 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.296 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.297 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.298 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.299 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.301 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.302 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.303 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.305 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T18:57:32.306 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.307 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.309 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.310 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T18:57:32.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.313 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.314 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.315 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.316 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.317 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.318 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T18:57:32.319 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.321 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.324 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.327 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.329 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.331 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T18:57:32.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T18:57:32.333 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T18:57:32.334 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.335 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.336 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.338 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T18:57:32.339 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.340 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.342 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.343 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.344 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.345 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.346 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.347 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.348 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.350 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.351 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.352 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T18:57:32.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.354 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.355 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.356 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.357 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.358 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.359 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.360 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.361 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.362 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-04-25T18:57:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T18:57:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-04-25T18:57:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-04-25T18:57:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:32.379 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.380 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-04-25T18:57:32.381 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.382 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.383 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.384 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.385 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T18:57:32.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-04-25T18:57:32.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.390 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-04-25T18:57:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.392 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.393 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.394 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.395 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.396 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-04-25T18:57:32.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-04-25T18:57:32.402 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.403 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.404 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.406 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-04-25T18:57:32.408 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.409 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.410 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.411 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.412 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T18:57:32.415 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.416 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.418 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.419 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.420 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-04-25T18:57:32.421 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.422 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.423 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.426 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.427 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-04-25T18:57:32.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T18:57:32.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.432 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.433 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.435 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.436 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.438 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.440 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.441 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.442 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.443 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.445 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.446 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.447 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.448 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.449 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.450 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.451 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.452 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.455 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.458 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.459 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.461 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.462 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.463 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.464 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.465 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.466 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.467 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.468 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.469 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.470 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.471 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.472 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.475 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.477 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.478 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.481 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.484 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.485 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.486 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.487 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T18:57:32.491 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.492 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-04-25T18:57:32.493 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-04-25T18:57:32.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.496 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.497 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.502 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.504 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.505 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.507 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.508 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.509 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.510 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T18:57:32.511 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T18:57:32.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.514 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.516 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.518 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.519 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.520 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.521 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.522 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.524 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.525 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.526 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.527 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.528 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.530 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.531 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.533 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.536 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.538 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-04-25T18:57:32.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.541 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.542 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.543 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.544 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.545 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.548 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.550 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.551 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.552 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.555 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-04-25T18:57:32.557 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:32.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.559 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.560 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.562 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.563 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.564 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.566 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.567 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.568 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.571 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.571 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.571 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.571 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.571 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.571 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-04-25T18:57:32.573 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-04-25T18:57:32.573 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.573 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.573 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.573 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T18:57:32.573 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.573 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.574 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.575 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.576 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.579 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.581 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.584 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.585 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T18:57:32.586 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.586 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.586 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.586 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.586 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.586 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.586 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.586 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-04-25T18:57:32.587 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.587 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.587 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.587 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.587 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.587 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.587 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.590 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.591 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.592 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.593 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.594 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.595 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.596 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.597 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-04-25T18:57:32.598 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-04-25T18:57:32.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T18:57:32.600 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.602 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.603 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.604 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.605 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.606 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T18:57:32.607 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T18:57:32.608 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T18:57:32.610 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T18:57:32.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: "services": { 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard": "https://172.21.15.112:8443/" 2024-04-25T18:57:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "always_on_modules": { 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus": [ 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific": [ 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-25T18:57:32.613 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy": [ 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-25T18:57:32.614 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "reef": [ 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-25T18:57:32.615 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "squid": [ 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.616 INFO:teuthology.orchestra.run.smithi112.stdout: "last_failure_osd_epoch": 95, 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: "active_clients": [ 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 238817619 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "libcephsqlite", 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1857208334 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 731504090 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:57:32.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 4089134861 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3702447220 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3311071430 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:32.620 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T18:57:32.621 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:57:32.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:31 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1702627940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T18:57:32.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:31 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/672739728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T18:57:32.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:32 smithi161 ceph-mon[32609]: pgmap v317: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T18:57:32.623 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:32 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/386959' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T18:57:32.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:31 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1702627940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T18:57:32.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:31 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/672739728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T18:57:32.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:32 smithi171 ceph-mon[33266]: pgmap v317: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T18:57:32.623 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:32 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/386959' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T18:57:32.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.376+0000 7fedf5800640 1 -- 172.21.15.112:0/386959 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fedd40a3da0 msgr2=0x7fedd40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:32.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.376+0000 7fedf5800640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fedd40a3da0 0x7fedd40a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fedd8000c00 tx=0x7fedd8005e80 comp rx=0 tx=0).stop 2024-04-25T18:57:32.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.376+0000 7fedf5800640 1 -- 172.21.15.112:0/386959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf0108580 msgr2=0x7fedf01a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:32.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.376+0000 7fedf5800640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf0108580 0x7fedf01a4a60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fede000e9f0 tx=0x7fede000edf0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.376+0000 7fedf5800640 1 -- 172.21.15.112:0/386959 shutdown_connections 2024-04-25T18:57:32.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.376+0000 7fedf5800640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fedd40a3da0 0x7fedd40a6260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.376+0000 7fedf5800640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf010e640 0x7fedf019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.376+0000 7fedf5800640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fedf0108f40 0x7fedf01a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.376+0000 7fedf5800640 1 --2- 172.21.15.112:0/386959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf0108580 0x7fedf01a4a60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.376+0000 7fedf5800640 1 -- 172.21.15.112:0/386959 >> 172.21.15.112:0/386959 conn(0x7fedf0075750 msgr2=0x7fedf0076930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.377+0000 7fedf5800640 1 -- 172.21.15.112:0/386959 shutdown_connections 2024-04-25T18:57:32.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:57:31.377+0000 7fedf5800640 1 -- 172.21.15.112:0/386959 wait complete. 2024-04-25T18:57:32.631 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T18:57:32.631 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T18:57:32.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:31 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1702627940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T18:57:32.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:31 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/672739728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T18:57:32.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:32 smithi112 ceph-mon[27864]: pgmap v317: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T18:57:32.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:32 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/386959' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T18:57:33.079 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:34.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:34 smithi161 ceph-mon[32609]: pgmap v318: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T18:57:34.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:34 smithi112 ceph-mon[27864]: pgmap v318: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T18:57:34.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:34 smithi171 ceph-mon[33266]: pgmap v318: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T18:57:35.530 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:57:35.530 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 5152, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4054, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 6170449, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 3515995, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 9, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 4, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.798985514, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.332821617, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.612578935, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.026956466, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4058, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.374332902, 2024-04-25T18:57:35.531 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000092245 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 8071, 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003108916, 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000385 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1700496, 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 165360 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 305, 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 504, 2024-04-25T18:57:35.532 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 86885, 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 695357, 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 21, 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 11, 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.084629666, 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.040301097, 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.026451634, 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.008989796, 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 497, 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.091209364, 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000183519 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:35.533 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 110, 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000029420, 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000267 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3904, 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 479104 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2245, 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4035, 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 834386, 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 2654564, 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 8, 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 6, 2024-04-25T18:57:35.534 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.645928117, 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.397273199, 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.353113975, 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.013397515, 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4033, 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.266754065, 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000066142 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4528, 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001786910, 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000394 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.535 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 118, 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.531481478, 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.012978656 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.536 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1021, 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.555041428, 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000543625 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 1426, 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 1302, 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1302, 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 2.630921773, 2024-04-25T18:57:35.537 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002020677 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 124, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 11, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 11, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 0, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 1, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 10, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 10, 2024-04-25T18:57:35.538 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 1406, 2024-04-25T18:57:35.539 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 1, 2024-04-25T18:57:35.539 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 2, 2024-04-25T18:57:35.539 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 3, 2024-04-25T18:57:35.539 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 2987, 2024-04-25T18:57:35.539 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 2154, 2024-04-25T18:57:35.539 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 121, 2024-04-25T18:57:35.539 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 65, 2024-04-25T18:57:35.539 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 8, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 1281, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 8, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 640, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 4, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 234, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 6, 2024-04-25T18:57:35.540 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 6, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 642, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T18:57:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 494, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 489, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 11, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 1296 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 20, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 461, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 259, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 266, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 20, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-25T18:57:35.542 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 640, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 640, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 2, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 19, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 8, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-25T18:57:35.543 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 1855, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 1855, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 3, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 3, 2024-04-25T18:57:35.544 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 2, 2024-04-25T18:57:35.545 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-25T18:57:35.545 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-04-25T18:57:35.545 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-04-25T18:57:35.545 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 8017199, 2024-04-25T18:57:35.545 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 0, 2024-04-25T18:57:35.545 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-25T18:57:35.545 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 39, 2024-04-25T18:57:35.545 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.149986899, 2024-04-25T18:57:35.545 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003845817 2024-04-25T18:57:35.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.545 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 0 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 15, 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 1365, 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 1350, 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 14, 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 238, 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 224, 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 11, 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 2058, 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 2047, 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 2, 2024-04-25T18:57:35.546 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 1286, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 1284, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 67028, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 223608 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 2074, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 3, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 1426, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 24, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 674, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.004526524, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004526524 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.547 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4, 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001086961, 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000271740 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.548 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 640, 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.490978787, 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000767154 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1, 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000145021, 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000145021 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.549 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.551 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-25T18:57:35.551 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.551 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.551 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.552 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 16, 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.053293038, 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003330814 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.553 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.554 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.555 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.557 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 640, 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 2.080891442, 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003251392 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:35.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 4, 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 2, 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 2, 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 1261, 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 630, 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 575, 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T18:57:35.559 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:57:35.560 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:57:35.561 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 280615, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 104, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T18:57:35.562 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 8224, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 171, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 63632, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 262, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T18:57:35.563 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 771, 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 3968129, 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 33, 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 771, 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 771, 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 4.032894999, 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005230732 2024-04-25T18:57:35.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 771, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 806 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 771, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 12, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 759, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 0, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 10, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 12, 2024-04-25T18:57:35.565 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 51, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 21, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 652, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 0, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 20, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-25T18:57:35.566 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-25T18:57:35.567 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 106, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 14, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 12, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 12, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-04-25T18:57:35.568 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 34, 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 0, 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 6 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 1, 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 21 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 10, 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 10, 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 640, 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 640, 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T18:57:35.570 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 7698, 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 6378735, 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 7698, 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 7698, 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 6378735, 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:35.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 771, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 4063489, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 771, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 4063489, 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:35.572 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 771, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 771, 2024-04-25T18:57:35.573 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 771, 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 771, 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 640, 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 95360, 2024-04-25T18:57:35.574 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 640, 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 12, 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 95360, 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x55b513124500": { 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:35.575 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 1855, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 3822895, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 1855, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 39, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 3822895, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:35.576 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:35.577 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:57:36.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:36 smithi161 ceph-mon[32609]: pgmap v319: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T18:57:36.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:36 smithi112 ceph-mon[27864]: pgmap v319: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T18:57:36.570 INFO:tasks.check_counter:Daemon mds.a mds.exported=8 2024-04-25T18:57:36.570 INFO:tasks.check_counter:Daemon mds.a mds.imported=8 2024-04-25T18:57:36.570 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=20 2024-04-25T18:57:36.570 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=461 2024-04-25T18:57:36.570 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-04-25T18:57:36.570 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-25T18:57:36.571 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T18:57:36.571 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-04-25T18:57:36.571 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T18:57:36.571 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T18:57:36.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:36 smithi171 ceph-mon[33266]: pgmap v319: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T18:57:37.009 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:38.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:38 smithi112 ceph-mon[27864]: pgmap v320: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T18:57:38.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:38 smithi161 ceph-mon[32609]: pgmap v320: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T18:57:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:38 smithi171 ceph-mon[33266]: pgmap v320: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T18:57:39.412 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.003863090, 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000982076, 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:57:39.413 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T18:57:39.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 254, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 243, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1599248, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 150000, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.109311384, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.026027144, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.106142591, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.011504657, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 242, 2024-04-25T18:57:39.415 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.024221736, 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000100089 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1599248, 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 150000 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:57:39.416 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 6, 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 188, 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3360, 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 189392, 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.039789038, 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.027299907, 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.006764958, 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 186, 2024-04-25T18:57:39.417 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.042052786, 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000226090 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 189392 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T18:57:39.418 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:57:39.419 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:57:39.420 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T18:57:39.421 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 149745, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 31, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T18:57:39.422 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T18:57:39.423 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 256, 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1563409, 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 256, 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 256, 2024-04-25T18:57:39.424 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1563409, 2024-04-25T18:57:39.425 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:39.425 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:39.425 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:39.425 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:39.425 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:39.425 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:39.425 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:57:40.182 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:39 smithi171 ceph-mon[33266]: pgmap v321: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T18:57:40.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:39 smithi161 ceph-mon[32609]: pgmap v321: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T18:57:40.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:39 smithi112 ceph-mon[27864]: pgmap v321: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T18:57:40.314 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T18:57:40.314 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T18:57:40.314 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T18:57:40.314 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T18:57:40.315 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T18:57:40.315 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T18:57:40.315 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T18:57:40.315 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-25T18:57:40.315 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T18:57:40.315 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T18:57:40.752 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:42.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:41 smithi112 ceph-mon[27864]: pgmap v322: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T18:57:42.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:41 smithi161 ceph-mon[32609]: pgmap v322: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T18:57:42.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:41 smithi171 ceph-mon[33266]: pgmap v322: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T18:57:43.012 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 544, 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 544, 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 29538039, 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 172178, 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 7, 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.186789908, 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.037270192, 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.104685132, 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.030160321, 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 543, 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.071931295, 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000132470 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:57:43.014 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 529, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 525, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 42657733, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 188029, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 4, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.237280984, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.041942308, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.160243918, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.026690194, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 524, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.060988781, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000116390 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.015 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 196384, 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 51232 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 178, 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 238, 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 25038825, 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 248056, 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-04-25T18:57:43.016 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.116911770, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.023606350, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.077314527, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.011153814, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 236, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.028529831, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000120889 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 191760 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.017 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1861, 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.179526337, 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000096467 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T18:57:43.018 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 0, 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-04-25T18:57:43.019 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 13272, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 7251, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 6021, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 2913, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 2, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-04-25T18:57:43.020 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 4, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 234, 2024-04-25T18:57:43.021 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 6, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T18:57:43.022 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-04-25T18:57:43.023 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-25T18:57:43.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 0, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 24677, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 0, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 3, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 17, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 168058607, 2024-04-25T18:57:43.025 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 172368324, 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 172368212, 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 24677 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 4659, 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 8584, 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 3925, 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 223, 2024-04-25T18:57:43.026 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 223, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 0, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 13272, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 13272, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 0, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 74612, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 215392 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 0, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 0, 2024-04-25T18:57:43.027 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 0, 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.028 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.029 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.030 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-25T18:57:43.031 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.032 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.034 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-25T18:57:43.034 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.034 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.034 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.035 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.036 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.037 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.038 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.039 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-25T18:57:43.040 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 2, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 2, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 0, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 0, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 1952, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.041 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:57:43.042 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:57:43.043 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 398192, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 4812, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T18:57:43.044 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 8224, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 171, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 20346829, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 167675, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T18:57:43.045 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 1147, 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 26, 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 4, 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 1147, 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1147, 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.655956892, 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001443728 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-25T18:57:43.046 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1147, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1148 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 1147, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 1147, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 0, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 611, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 0, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 533, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 0, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 0, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-25T18:57:43.047 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 2, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T18:57:43.048 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-25T18:57:43.049 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 106, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 8, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 9, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 9, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 0, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 2, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 0 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.050 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 0, 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-04-25T18:57:43.051 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 1247, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 97114289, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 1247, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1247, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 97114289, 2024-04-25T18:57:43.052 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 1147, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 96714461, 2024-04-25T18:57:43.053 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1147, 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 96714461, 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T18:57:43.054 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 1147, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 1147, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1147, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1147, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T18:57:43.055 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:43.056 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x557a5524e780": { 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:43.057 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:43.058 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:43.058 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:43.058 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:57:43.990 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T18:57:43.990 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T18:57:43.990 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-25T18:57:43.990 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-25T18:57:43.990 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-04-25T18:57:43.990 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-25T18:57:43.990 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T18:57:43.990 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-04-25T18:57:43.990 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T18:57:43.991 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T18:57:44.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:43 smithi161 ceph-mon[32609]: pgmap v323: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:44.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:43 smithi112 ceph-mon[27864]: pgmap v323: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:44.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:43 smithi171 ceph-mon[33266]: pgmap v323: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:44.434 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:46.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:45 smithi161 ceph-mon[32609]: pgmap v324: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:46.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:45 smithi112 ceph-mon[27864]: pgmap v324: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:46.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:45 smithi171 ceph-mon[33266]: pgmap v324: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:46.831 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T18:57:46.831 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:57:46.831 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-04-25T18:57:46.831 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-04-25T18:57:46.831 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-04-25T18:57:46.831 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.003701611, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000766189, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:46.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 189, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 320, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 189472, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.035763315, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.026735890, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.003347542, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:57:46.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 186, 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.029126072, 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000156591 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 189472 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:46.834 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 261, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 248, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1603104, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 152368, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.118486006, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.025400222, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.122059064, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.012888346, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 248, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.026752300, 2024-04-25T18:57:46.835 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000107872 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1603104, 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 152368 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T18:57:46.836 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:57:46.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T18:57:46.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148714, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 30, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T18:57:46.839 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T18:57:46.840 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 259, 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1563799, 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 259, 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 259, 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1563799, 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T18:57:46.841 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T18:57:47.863 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T18:57:47.863 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T18:57:47.863 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T18:57:47.863 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T18:57:47.863 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T18:57:47.863 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T18:57:47.863 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T18:57:47.863 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-25T18:57:47.863 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T18:57:47.864 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T18:57:48.114 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:57:48.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:47 smithi161 ceph-mon[32609]: pgmap v325: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:57:48.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:47 smithi112 ceph-mon[27864]: pgmap v325: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:57:48.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:47 smithi171 ceph-mon[33266]: pgmap v325: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:57:49.229 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:48 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:49.229 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:48 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.230 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:48 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.230 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:48 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.230 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:48 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:48 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:48 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:48 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:48 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:48 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:48 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:49.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:48 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:48 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:48 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:48 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:49.889 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-04-25T18:57:49.889 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 24734, 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 23893, 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 8121811, 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 68538019, 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 90, 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 1.771495567, 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.613303402, 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.483552393, 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.578230088, 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:49.890 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 23888, 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2.615112574, 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000109473 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1308, 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000218430, 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000166 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2156704, 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 278704 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 69716, 2024-04-25T18:57:49.891 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 122702, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 21814743, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 112686963, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 23, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 4.818207488, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 1.895011238, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 2.368321168, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.761613608, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 122703, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 5.576633740, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000045448 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:49.892 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 102533, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.018740504, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000182 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3520, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 566528 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 22724, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 23728, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 6658764, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 62692299, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 11, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 1.651857378, 2024-04-25T18:57:49.893 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.632612550, 2024-04-25T18:57:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.499814314, 2024-04-25T18:57:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.460291143, 2024-04-25T18:57:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 23726, 2024-04-25T18:57:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 4.068263152, 2024-04-25T18:57:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000171468 2024-04-25T18:57:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 7564, 2024-04-25T18:57:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001347043, 2024-04-25T18:57:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000178 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 42496, 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 2336 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 90147, 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 28.225307165, 2024-04-25T18:57:49.895 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000313103 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 7807, 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.768405093, 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000226515 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 20540, 2024-04-25T18:57:49.896 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 20531, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 20531, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 145.747411964, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.007098894 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 9, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 1194, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 22, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 9520, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 4967, 2024-04-25T18:57:49.897 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 4543, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 10, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 2420, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 648, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 3, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 3, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 4, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 22649, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 20949, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 3, 2024-04-25T18:57:49.898 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 9, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 644, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 9, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 1293, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-04-25T18:57:49.899 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 4, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 9557, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 18730, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 7863, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 15890, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 12814, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 424, 2024-04-25T18:57:49.900 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 13893 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 402, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 543, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 61, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 4528, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 4464, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 3, 2024-04-25T18:57:49.901 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 2, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 580, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 9, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 22, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-04-25T18:57:49.902 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 49002, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 47515, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 47515, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 1487, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 10319, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 97, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 94, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 94, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 3, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 1, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-04-25T18:57:49.903 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 14, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 168058607, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 172368324, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 25214, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 111.947014561, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.004439875 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 4755, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 9863, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 5108, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 234, 2024-04-25T18:57:49.904 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 263, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 29, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 9520, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 15065, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 5545, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 3, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 5565, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 5562, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 108964, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 231800 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 39023, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 693, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 20540, 2024-04-25T18:57:49.905 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 46, 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 1398, 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5098, 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 74.622437060, 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.014637590 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 107, 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.049538324, 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000462974 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.003154543, 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001051514 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1087, 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.419662855, 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000386074 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-04-25T18:57:49.907 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000379839, 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000379839 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.909 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-04-25T18:57:49.909 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.909 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.909 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.909 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-04-25T18:57:49.909 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.909 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.909 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.910 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-04-25T18:57:49.910 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 10, 2024-04-25T18:57:49.910 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.002165261, 2024-04-25T18:57:49.910 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000216526 2024-04-25T18:57:49.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.910 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-04-25T18:57:49.910 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 73, 2024-04-25T18:57:49.910 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 15.345785870, 2024-04-25T18:57:49.910 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.210216244 2024-04-25T18:57:49.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 10, 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.094296170, 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.009429617 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 420, 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.259896083, 2024-04-25T18:57:49.911 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000618800 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 453, 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.349735942, 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002979549 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 92, 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.187390557, 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002036853 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.912 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 67, 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.021695436, 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.045099931 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4, 2024-04-25T18:57:49.913 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.024489263, 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.006122315 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8682, 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 42.699995190, 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.004918221 2024-04-25T18:57:49.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8, 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.260635398, 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.032579424 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.915 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8, 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.022670532, 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002833816 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6, 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.004730556, 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000788426 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.916 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4402, 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 7.378753085, 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001676227 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-04-25T18:57:49.917 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 6, 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 4, 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 3984, 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 1992, 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 604, 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 120, 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 120, 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:57:49.918 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-04-25T18:57:49.919 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:57:49.920 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 971997, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 4862, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 88, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 1, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8224, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 171, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:57:49.921 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 18175021, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 185658, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 73234, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 179939514, 2024-04-25T18:57:49.922 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 32, 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 73234, 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 73234, 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1227.700767143, 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.016764081 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 73234, 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 84633 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 73234, 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 60, 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 73174, 2024-04-25T18:57:49.923 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 939, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 4491, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 60, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 25274, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 177, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 6, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 5100, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-04-25T18:57:49.924 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 8983, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 36717, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-04-25T18:57:49.925 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-04-25T18:57:49.926 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 106, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 14, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 2644, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 242 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-04-25T18:57:49.927 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 623 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 34, 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 10, 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 4493, 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 4464, 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-25T18:57:49.928 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 117170, 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 25978498, 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 117170, 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 117170, 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 25978498, 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:49.929 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 73234, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 180578074, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 73234, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 180578074, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.930 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 73234, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 73234, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 73234, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 73234, 2024-04-25T18:57:49.931 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 4464, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 638560, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 4464, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-25T18:57:49.932 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 60, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 638560, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55bd7226e500": { 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 49002, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 168174020, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 49002, 2024-04-25T18:57:49.933 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-25T18:57:49.934 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-25T18:57:49.934 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 25214, 2024-04-25T18:57:49.934 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 168174020, 2024-04-25T18:57:49.934 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:49.934 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:49.934 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:49.934 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:49.934 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:49.934 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:49.934 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-04-25T18:57:50.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:49 smithi161 ceph-mon[32609]: pgmap v326: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:50.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:49 smithi171 ceph-mon[33266]: pgmap v326: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:50.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:49 smithi112 ceph-mon[27864]: pgmap v326: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:50.636 INFO:tasks.check_counter:Daemon mds.b mds.exported=9 2024-04-25T18:57:50.636 INFO:tasks.check_counter:Daemon mds.b mds.imported=9 2024-04-25T18:57:50.636 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=402 2024-04-25T18:57:50.636 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-25T18:57:50.636 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T18:57:50.636 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T18:57:50.636 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T18:57:50.636 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-04-25T18:57:50.637 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T18:57:50.637 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T18:57:51.374 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.800 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:51 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:51 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:51.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:51.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:51 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:52 smithi161 ceph-mon[32609]: pgmap v327: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:57:52.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:52 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T18:57:52.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:52 smithi161 ceph-mon[32609]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T18:57:52.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:52 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:52 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:52 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:52 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:52 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:52 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:52 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:52 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:52 smithi171 ceph-mon[33266]: pgmap v327: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:57:52.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:52 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T18:57:52.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:52 smithi171 ceph-mon[33266]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T18:57:52.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:52 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:52 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:52 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:52 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:52 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:52 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:52 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.933 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:52 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:52 smithi112 ceph-mon[27864]: pgmap v327: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:57:52.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:52 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T18:57:52.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:52 smithi112 ceph-mon[27864]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T18:57:52.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:52 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:52 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:52 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:52 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:52 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:52 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:52 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:52 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:52.989 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 257, 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 246, 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1600064, 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 151920, 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.047263618, 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.013990526, 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.036033910, 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.005521982, 2024-04-25T18:57:52.990 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 245, 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.012660400, 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000051675 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1600064, 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 151920 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:52.991 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 6, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 190, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3360, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 189744, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.011812244, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.007794616, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001628272, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 188, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.039339240, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000209251 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:52.992 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 189744 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 0, 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2, 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 0, 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 0, 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-04-25T18:57:52.993 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.000182744, 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000122684, 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:57:52.995 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:57:52.996 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148670, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 30, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-04-25T18:57:52.997 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-04-25T18:57:52.998 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 259, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1563799, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 259, 2024-04-25T18:57:52.999 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-25T18:57:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-25T18:57:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 259, 2024-04-25T18:57:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1563799, 2024-04-25T18:57:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:53.000 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:53.000 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-04-25T18:57:53.674 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-25T18:57:53.674 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-25T18:57:53.674 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-04-25T18:57:53.674 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-04-25T18:57:53.674 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-25T18:57:53.674 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-25T18:57:53.674 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-25T18:57:53.674 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-04-25T18:57:53.675 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T18:57:53.675 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T18:57:53.931 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:57:53.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:53 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi112 to 176.4M 2024-04-25T18:57:53.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:53 smithi171 ceph-mon[33266]: Unable to set osd_memory_target on smithi112 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-04-25T18:57:53.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:53 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi112 to 176.4M 2024-04-25T18:57:53.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:53 smithi161 ceph-mon[32609]: Unable to set osd_memory_target on smithi112 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-04-25T18:57:53.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:53 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi112 to 176.4M 2024-04-25T18:57:53.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:53 smithi112 ceph-mon[27864]: Unable to set osd_memory_target on smithi112 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-04-25T18:57:54.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:54 smithi161 ceph-mon[32609]: pgmap v328: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:54.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:54 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:54.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:54 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:54.834 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:54 smithi171 ceph-mon[33266]: pgmap v328: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:54.834 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:54 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:54.834 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:54 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:54.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:54 smithi112 ceph-mon[27864]: pgmap v328: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:54.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:54 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:54.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:54 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:55.597 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-04-25T18:57:55.597 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:57:55.597 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 289, 2024-04-25T18:57:55.597 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 264, 2024-04-25T18:57:55.597 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1680898, 2024-04-25T18:57:55.597 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 154260, 2024-04-25T18:57:55.597 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-04-25T18:57:55.597 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-04-25T18:57:55.597 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.048927735, 2024-04-25T18:57:55.597 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.015004810, 2024-04-25T18:57:55.598 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.036603299, 2024-04-25T18:57:55.598 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.005785236, 2024-04-25T18:57:55.598 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:55.598 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 263, 2024-04-25T18:57:55.598 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.012912277, 2024-04-25T18:57:55.598 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000049096 2024-04-25T18:57:55.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.598 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:55.598 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.598 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.598 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1680064, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 153936 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 636, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 818, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 158362, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 679785, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.061877814, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.022184812, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.013134483, 2024-04-25T18:57:55.599 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.019658977, 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 816, 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.054522000, 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000066816 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 479440 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.600 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1723, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1725, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 572044, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 547972, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.156113069, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.055812811, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.038642289, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.047614662, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1723, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.061748179, 2024-04-25T18:57:55.601 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000035837 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-04-25T18:57:55.602 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4069, 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.106658504, 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000026212 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.603 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-04-25T18:57:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 5, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 5, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 5, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-04-25T18:57:55.605 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-04-25T18:57:55.606 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T18:57:55.607 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-04-25T18:57:55.608 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.609 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 33, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-04-25T18:57:55.610 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4251973, 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 4249368, 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 33 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 4, 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 27, 2024-04-25T18:57:55.611 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 23, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 2, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 14, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 12, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 5, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 29, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 24, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 51096, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215416 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-04-25T18:57:55.612 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-04-25T18:57:55.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-04-25T18:57:55.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.615 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-04-25T18:57:55.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.617 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.617 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-04-25T18:57:55.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.617 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.617 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-04-25T18:57:55.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-04-25T18:57:55.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-04-25T18:57:55.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-04-25T18:57:55.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-04-25T18:57:55.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.622 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-04-25T18:57:55.623 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.623 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.623 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.623 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-04-25T18:57:55.623 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-04-25T18:57:55.623 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-04-25T18:57:55.623 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.623 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.623 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.623 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 2190, 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-04-25T18:57:55.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-04-25T18:57:55.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-04-25T18:57:55.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 190177, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 53, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-04-25T18:57:55.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8224, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 171, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 17248, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 52, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-04-25T18:57:55.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 2354, 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 2354, 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2354, 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2.012955501, 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000855121 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2354, 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2355 2024-04-25T18:57:55.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 2354, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 2354, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 1481, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 871, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-04-25T18:57:55.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-04-25T18:57:55.631 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 106, 2024-04-25T18:57:55.632 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 14, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 5, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 5, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-04-25T18:57:55.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.634 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 2644, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 2155598, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 2644, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2644, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2155598, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-04-25T18:57:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-25T18:57:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-25T18:57:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-25T18:57:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2354, 2024-04-25T18:57:55.637 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 57674, 2024-04-25T18:57:55.637 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2354, 2024-04-25T18:57:55.637 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 57674, 2024-04-25T18:57:55.637 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:55.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.637 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.638 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:55.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.638 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-04-25T18:57:55.638 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:55.638 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-04-25T18:57:55.638 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:55.638 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-25T18:57:55.638 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-25T18:57:55.638 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:55.638 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:55.638 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2354, 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 2354, 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2354, 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2354, 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:55.639 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55d459ab86e0": { 2024-04-25T18:57:55.640 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:55.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:55.642 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:55.642 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-04-25T18:57:55.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:55 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:55 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:55 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:55 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:55 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:55 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:55 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:55 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:55 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:55.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:55 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:55.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:55 smithi171 ceph-mon[33266]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:55.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:55 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:55 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:55 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:55 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:55 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:55 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:55 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:55 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:55 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:55.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:55 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:55.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:55 smithi161 ceph-mon[32609]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:55.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:55 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:55 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:55 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:55 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:55 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:55 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:55 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:55 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:57:55.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:55 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:55.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:55 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:55.975 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:55 smithi112 ceph-mon[27864]: from='mgr.14922 ' entity='mgr.x' 2024-04-25T18:57:56.251 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T18:57:56.251 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T18:57:56.251 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T18:57:56.251 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-25T18:57:56.251 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-25T18:57:56.251 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T18:57:56.251 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T18:57:56.251 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-25T18:57:56.251 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T18:57:56.251 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T18:57:56.495 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:57:56.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:56 smithi171 ceph-mon[33266]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-25T18:57:56.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:56 smithi171 ceph-mon[33266]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T18:57:56.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:56 smithi171 ceph-mon[33266]: pgmap v329: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:56.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:56 smithi161 ceph-mon[32609]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-25T18:57:56.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:56 smithi161 ceph-mon[32609]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T18:57:56.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:56 smithi161 ceph-mon[32609]: pgmap v329: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:56.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:56 smithi112 ceph-mon[27864]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-04-25T18:57:56.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:56 smithi112 ceph-mon[27864]: Unable to set osd_memory_target on smithi161 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T18:57:56.974 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:56 smithi112 ceph-mon[27864]: pgmap v329: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:58.091 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-04-25T18:57:58.091 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:57:58.091 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 0, 2024-04-25T18:57:58.091 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2, 2024-04-25T18:57:58.091 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 0, 2024-04-25T18:57:58.091 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 0, 2024-04-25T18:57:58.091 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-04-25T18:57:58.091 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2024-04-25T18:57:58.091 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.001003275, 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000183425, 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:58.092 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 6, 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 194, 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3360, 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 190448, 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.012240581, 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.008180402, 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001655511, 2024-04-25T18:57:58.093 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 192, 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.031908083, 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000166187 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 190448 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:58.094 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 261, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 250, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1601152, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 154480, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.049642356, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.015049467, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.037796327, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.005932687, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 249, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.011628053, 2024-04-25T18:57:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000046699 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1601152, 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 154480 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-04-25T18:57:58.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:57:58.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:57:58.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148692, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 30, 2024-04-25T18:57:58.099 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:57:58.100 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 263, 2024-04-25T18:57:58.101 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1564319, 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 263, 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 263, 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1564319, 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-04-25T18:57:58.102 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-04-25T18:57:58.708 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T18:57:58.708 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T18:57:58.708 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-25T18:57:58.708 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-25T18:57:58.708 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T18:57:58.708 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T18:57:58.708 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T18:57:58.708 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-25T18:57:58.708 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T18:57:58.709 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T18:57:58.811 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:57:58 smithi112 ceph-mon[27864]: pgmap v330: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:57:58.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:57:58 smithi171 ceph-mon[33266]: pgmap v330: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:57:58.961 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:57:58.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:57:58 smithi161 ceph-mon[32609]: pgmap v330: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T18:58:00.647 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:00 smithi171 ceph-mon[33266]: pgmap v331: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:58:00.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:00 smithi112 ceph-mon[27864]: pgmap v331: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:58:00.781 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-25T18:58:00.781 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 2495, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2497, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 4528236, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 793417, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 6, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 2, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.205200111, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.068261473, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.052252360, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.069189050, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2495, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.087290371, 2024-04-25T18:58:00.782 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000034986 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 292, 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 267, 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1681164, 2024-04-25T18:58:00.783 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 155914, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 3, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 2, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.048465536, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.014518319, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.038165555, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.005869830, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 266, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.012930076, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000048609 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.784 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1680608, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 155216 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 7, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 192, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3628, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 485134, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 6, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.010734871, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.006757801, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001995387, 2024-04-25T18:58:00.785 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000034200, 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 190, 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.030648665, 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000161308 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 484816 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.786 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4233, 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.112015694, 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000026462 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-04-25T18:58:00.787 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.788 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 642, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 0, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 642, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 642, 2024-04-25T18:58:00.789 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 1, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-04-25T18:58:00.790 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-04-25T18:58:00.791 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-04-25T18:58:00.792 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T18:58:00.792 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T18:58:00.792 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-04-25T18:58:00.792 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-04-25T18:58:00.792 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T18:58:00.792 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-04-25T18:58:00.792 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T18:58:00.792 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T18:58:00.792 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T18:58:00.792 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-04-25T18:58:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-04-25T18:58:00.794 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 1855, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-04-25T18:58:00.795 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 3, 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 2, 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 8017199, 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 8014243, 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 1855 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.796 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 5, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 1297, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 1292, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 3, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 22, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 19, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 642, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 1935, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 1293, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 55236, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 215416 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.797 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.798 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.799 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.800 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.801 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.802 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.802 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-04-25T18:58:00.802 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.802 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.802 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.803 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.805 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.806 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.808 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 4, 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 2, 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 3, 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 2990, 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.809 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:58:00.810 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:58:00.811 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 182037, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 48, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-25T18:58:00.812 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 8224, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 171, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 470040, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2613, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.813 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 2498, 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 21, 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 3, 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 2498, 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2498, 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1.339973541, 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000536418 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2498, 2024-04-25T18:58:00.814 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 2499 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 2498, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 2498, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 1612, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 884, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-04-25T18:58:00.815 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 1, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-04-25T18:58:00.816 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-04-25T18:58:00.817 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 106, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 14, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 4, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 4, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-04-25T18:58:00.818 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-04-25T18:58:00.819 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 2790, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 5944078, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 2790, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2790, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 5944078, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.820 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 2498, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 3822900, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2498, 2024-04-25T18:58:00.821 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 3822900, 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:00.822 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 2498, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 2498, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2498, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 2498, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-25T18:58:00.823 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x55612b6d26e0": { 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-04-25T18:58:00.824 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:00.825 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-25T18:58:00.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:00 smithi161 ceph-mon[32609]: pgmap v331: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:58:01.556 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T18:58:01.557 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T18:58:01.557 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T18:58:01.557 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-25T18:58:01.557 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-25T18:58:01.557 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T18:58:01.557 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T18:58:01.557 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-25T18:58:01.557 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T18:58:01.557 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T18:58:01.798 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:58:02.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:02 smithi171 ceph-mon[33266]: pgmap v332: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T18:58:02.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:02 smithi161 ceph-mon[32609]: pgmap v332: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T18:58:02.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:02 smithi112 ceph-mon[27864]: pgmap v332: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T18:58:03.440 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-25T18:58:03.440 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:58:03.440 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 4, 2024-04-25T18:58:03.440 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 4, 2024-04-25T18:58:03.440 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3040, 2024-04-25T18:58:03.440 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 448, 2024-04-25T18:58:03.440 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-25T18:58:03.440 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-04-25T18:58:03.440 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.001323147, 2024-04-25T18:58:03.440 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000038542, 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000495567, 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3, 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000708925, 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000236308 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T18:58:03.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 259, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 248, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1600608, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 153200, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.047622891, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.014773417, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.035576771, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.005950788, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 247, 2024-04-25T18:58:03.442 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.012785472, 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000051763 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1600608, 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 153200 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 2, 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 189, 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 320, 2024-04-25T18:58:03.443 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 189472, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.011434866, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.007972512, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001145881, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 186, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.028988823, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000155853 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:03.444 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 189472 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:58:03.445 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:58:03.446 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:58:03.447 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 148714, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 30, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-25T18:58:03.448 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 261, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1564059, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 261, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T18:58:03.449 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T18:58:03.450 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 261, 2024-04-25T18:58:03.450 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1564059, 2024-04-25T18:58:03.450 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:03.450 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:03.450 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:03.450 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:03.450 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:03.450 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:03.450 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-25T18:58:04.111 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-04-25T18:58:04.112 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-04-25T18:58:04.112 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.f 2024-04-25T18:58:04.112 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.f 2024-04-25T18:58:04.112 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-25T18:58:04.112 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-25T18:58:04.112 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-25T18:58:04.112 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-04-25T18:58:04.112 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T18:58:04.112 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T18:58:04.352 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:58:04.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:04 smithi171 ceph-mon[33266]: pgmap v333: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T18:58:04.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:04 smithi161 ceph-mon[32609]: pgmap v333: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T18:58:04.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:04 smithi112 ceph-mon[27864]: pgmap v333: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T18:58:05.976 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-25T18:58:05.976 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:58:05.976 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 182, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 219, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 54267, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 160447, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 15, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.025516702, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.007009736, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.017634605, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.001204140, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 218, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.025379173, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000116418 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.977 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 280, 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000059942, 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000214 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 6752, 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 5600 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 78, 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 251, 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 34645, 2024-04-25T18:58:05.978 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 649588, 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 14, 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 7, 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.019236315, 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.009512806, 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.007316421, 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000754709, 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 248, 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.036539419, 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000147336 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 80, 2024-04-25T18:58:05.979 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000013496, 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000168 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 566144 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 4329, 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2499, 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 5319544, 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 692946, 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 6, 2024-04-25T18:58:05.980 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.236434067, 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.090413824, 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.186638362, 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.010374306, 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2498, 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.070807277, 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000028345 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6043, 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000873863, 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000144 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.981 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1693216, 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 160192 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 99, 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.009269768, 2024-04-25T18:58:05.982 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000093634 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 19, 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000552341, 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000029070 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 107, 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 13, 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-04-25T18:58:05.983 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 13, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.012214957, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000939612 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 94, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 11, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 20, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 2, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 8, 2024-04-25T18:58:05.984 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 10, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 13, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 130, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 3, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 406, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 248, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 92, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 57, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 1, 2024-04-25T18:58:05.985 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 12, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 1, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 4, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 4, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 234, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 6, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-04-25T18:58:05.986 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 5, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 6, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 8, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 4, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 2, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 10 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 2, 2024-04-25T18:58:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 500, 2024-04-25T18:58:05.988 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 278, 2024-04-25T18:58:05.988 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 299, 2024-04-25T18:58:05.988 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 2, 2024-04-25T18:58:05.988 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-04-25T18:58:05.988 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-04-25T18:58:05.988 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-04-25T18:58:05.988 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 4, 2024-04-25T18:58:05.988 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 4, 2024-04-25T18:58:05.988 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-04-25T18:58:05.988 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-04-25T18:58:05.988 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 2, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 19, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 1, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-04-25T18:58:05.989 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 33, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 33, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 2, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 2, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 2, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-04-25T18:58:05.990 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 4251973, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 0, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 20, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.039285642, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001964282 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 0 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 24, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 98, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 74, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 23, 2024-04-25T18:58:05.991 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 251, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 228, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 20, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 155, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 135, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 12, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 12, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 56436, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 223608 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 116, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 3, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 107, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 12, 2024-04-25T18:58:05.992 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 31, 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1, 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.003933298, 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.003933298 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1, 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000182251, 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000182251 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.993 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5, 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.001179588, 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000235917 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.994 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.995 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.996 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-04-25T18:58:05.996 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.996 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.996 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.996 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-04-25T18:58:05.996 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.996 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.996 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.996 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.997 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-04-25T18:58:05.997 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.997 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.997 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.997 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.997 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-04-25T18:58:05.997 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.997 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.997 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2, 2024-04-25T18:58:05.998 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000816058, 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000408029 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-04-25T18:58:05.999 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.000 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.001 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.002 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4, 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.006103762, 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001525940 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.003 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 1, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 2, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 1, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 1, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 8, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 8, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 537, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-25T18:58:06.004 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:58:06.005 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-25T18:58:06.006 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 272247, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 96, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 8224, 2024-04-25T18:58:06.007 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 171, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 103765, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 408, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 81, 2024-04-25T18:58:06.008 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 71045, 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 33, 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 81, 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 81, 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.509989847, 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.006296170 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 81, 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 112 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 81, 2024-04-25T18:58:06.009 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 1, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 80, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 0, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 10, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 1, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 21, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 13, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 16, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-04-25T18:58:06.010 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 20, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-04-25T18:58:06.011 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-04-25T18:58:06.012 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 106, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 14, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 10, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 10, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 26, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 0, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 5 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 1, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-04-25T18:58:06.013 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 21 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 4, 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 4, 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 4, 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 4, 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:06.014 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 4585, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 4975865, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 4585, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 4585, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 4975865, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.015 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 81, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 71641, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 81, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 71641, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.016 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 81, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 81, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 81, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 81, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.017 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 4, 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 596, 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 4, 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T18:58:06.018 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 1, 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 596, 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x5601b012a500": { 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 33, 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 57669, 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:06.019 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 33, 2024-04-25T18:58:06.020 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T18:58:06.020 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T18:58:06.020 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 20, 2024-04-25T18:58:06.020 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 57669, 2024-04-25T18:58:06.020 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:06.020 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:06.020 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:06.020 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:06.020 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:06.020 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:06.020 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-25T18:58:06.843 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:06 smithi171 ceph-mon[33266]: pgmap v334: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T18:58:06.845 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-04-25T18:58:06.845 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-04-25T18:58:06.845 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=2 2024-04-25T18:58:06.845 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=500 2024-04-25T18:58:06.845 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T18:58:06.845 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T18:58:06.845 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T18:58:06.845 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-25T18:58:06.845 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T18:58:06.846 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T18:58:06.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:06 smithi161 ceph-mon[32609]: pgmap v334: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T18:58:06.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:06 smithi112 ceph-mon[27864]: pgmap v334: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T18:58:07.244 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:58:07.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:07 smithi171 ceph-mon[33266]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T18:58:07.634 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:07 smithi171 ceph-mon[33266]: pgmap v336: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T18:58:07.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:07 smithi161 ceph-mon[32609]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T18:58:07.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:07 smithi161 ceph-mon[32609]: pgmap v336: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T18:58:07.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:07 smithi112 ceph-mon[27864]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T18:58:07.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:07 smithi112 ceph-mon[27864]: pgmap v336: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T18:58:09.362 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-25T18:58:09.362 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T18:58:09.362 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 270, 2024-04-25T18:58:09.362 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 257, 2024-04-25T18:58:09.362 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1605616, 2024-04-25T18:58:09.362 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 157648, 2024-04-25T18:58:09.362 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-25T18:58:09.362 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-04-25T18:58:09.362 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.051139711, 2024-04-25T18:58:09.362 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.015802338, 2024-04-25T18:58:09.362 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.035594817, 2024-04-25T18:58:09.363 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.006599496, 2024-04-25T18:58:09.363 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:09.363 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 257, 2024-04-25T18:58:09.363 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.013148479, 2024-04-25T18:58:09.363 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000051161 2024-04-25T18:58:09.363 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:09.363 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:09.363 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1605616, 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 157648 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 3, 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 196, 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 480, 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 269072, 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 6, 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-04-25T18:58:09.364 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.011955125, 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.007857546, 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001691255, 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 193, 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.036966939, 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000191538 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:09.365 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 269072 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 0, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 0, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 0, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.000293884, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000231390, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T18:58:09.366 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-04-25T18:58:09.367 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T18:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-25T18:58:09.369 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 148794, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 32, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-25T18:58:09.370 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T18:58:09.371 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 269, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1565060, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 269, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 269, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1565060, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T18:58:09.372 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:09.373 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T18:58:09.373 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-25T18:58:10.014 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T18:58:10.014 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T18:58:10.014 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-25T18:58:10.015 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-25T18:58:10.015 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T18:58:10.015 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T18:58:10.015 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T18:58:10.015 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-25T18:58:10.015 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T18:58:10.033 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T18:58:10.034 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:58:10.034 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T18:58:10.092 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:58:10.092 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T18:58:10.147 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-25T18:58:10.217 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:09 smithi112 ceph-mon[27864]: pgmap v337: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T18:58:10.221 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:09 smithi161 ceph-mon[32609]: pgmap v337: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T18:58:10.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:09 smithi171 ceph-mon[33266]: pgmap v337: 97 pgs: 97 active+clean; 136 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T18:58:10.580 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:12.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:11 smithi112 ceph-mon[27864]: pgmap v338: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-25T18:58:12.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:11 smithi171 ceph-mon[33266]: pgmap v338: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-25T18:58:12.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:11 smithi161 ceph-mon[32609]: pgmap v338: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-25T18:58:13.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.104+0000 7f7531400700 1 -- 172.21.15.112:0/2956250399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f752c10e3b0 msgr2=0x7f752c115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:13.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.104+0000 7f7531400700 1 --2- 172.21.15.112:0/2956250399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f752c10e3b0 0x7f752c115f80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f752400b0d0 tx=0x7f752400b3e0 comp rx=0 tx=0).stop 2024-04-25T18:58:13.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.104+0000 7f7531400700 1 -- 172.21.15.112:0/2956250399 shutdown_connections 2024-04-25T18:58:13.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.104+0000 7f7531400700 1 --2- 172.21.15.112:0/2956250399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f752c10e3b0 0x7f752c115f80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:13.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.104+0000 7f7531400700 1 --2- 172.21.15.112:0/2956250399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f752c1092d0 0x7f752c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:13.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.104+0000 7f7531400700 1 --2- 172.21.15.112:0/2956250399 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f752c109c80 0x7f752c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:13.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.104+0000 7f7531400700 1 -- 172.21.15.112:0/2956250399 >> 172.21.15.112:0/2956250399 conn(0x7f752c076210 msgr2=0x7f752c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.104+0000 7f7531400700 1 -- 172.21.15.112:0/2956250399 shutdown_connections 2024-04-25T18:58:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.104+0000 7f7531400700 1 -- 172.21.15.112:0/2956250399 wait complete. 2024-04-25T18:58:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.105+0000 7f7531400700 1 Processor -- start 2024-04-25T18:58:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.105+0000 7f7531400700 1 -- start start 2024-04-25T18:58:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.106+0000 7f7531400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f752c1092d0 0x7f752c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.106+0000 7f7531400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f752c109c80 0x7f752c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.106+0000 7f7531400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f752c10e3b0 0x7f752c1a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.106+0000 7f7531400700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1192a0 con 0x7f752c109c80 2024-04-25T18:58:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.106+0000 7f7531400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f752c119120 con 0x7f752c1092d0 2024-04-25T18:58:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.106+0000 7f7531400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f752c119420 con 0x7f752c10e3b0 2024-04-25T18:58:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.106+0000 7f7529e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f752c109c80 0x7f752c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.106+0000 7f7529e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f752c109c80 0x7f752c19fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51894/0 (socket says 172.21.15.112:51894) 2024-04-25T18:58:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.106+0000 7f7529e00700 1 -- 172.21.15.112:0/2535636941 learned_addr learned my addr 172.21.15.112:0/2535636941 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:58:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.106+0000 7f752a800700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f752c1092d0 0x7f752c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.106+0000 7f752b200700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f752c10e3b0 0x7f752c1a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.107+0000 7f752b200700 1 -- 172.21.15.112:0/2535636941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f752c1092d0 msgr2=0x7f752c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.107+0000 7f752b200700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f752c1092d0 0x7f752c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.107+0000 7f752b200700 1 -- 172.21.15.112:0/2535636941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f752c109c80 msgr2=0x7f752c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.107+0000 7f752b200700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f752c109c80 0x7f752c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.107+0000 7f752b200700 1 -- 172.21.15.112:0/2535636941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c1a46e0 con 0x7f752c10e3b0 2024-04-25T18:58:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.107+0000 7f7529e00700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f752c109c80 0x7f752c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.107+0000 7f752b200700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f752c10e3b0 0x7f752c1a3fa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f752400b770 tx=0x7f7524007980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.107+0000 7f7523400700 1 -- 172.21.15.112:0/2535636941 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752401f030 con 0x7f752c10e3b0 2024-04-25T18:58:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.107+0000 7f7523400700 1 -- 172.21.15.112:0/2535636941 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7524005050 con 0x7f752c10e3b0 2024-04-25T18:58:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.107+0000 7f7531400700 1 -- 172.21.15.112:0/2535636941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c1a4970 con 0x7f752c10e3b0 2024-04-25T18:58:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.107+0000 7f7523400700 1 -- 172.21.15.112:0/2535636941 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752401b690 con 0x7f752c10e3b0 2024-04-25T18:58:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.108+0000 7f7531400700 1 -- 172.21.15.112:0/2535636941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c1ac250 con 0x7f752c10e3b0 2024-04-25T18:58:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.109+0000 7f7531400700 1 -- 172.21.15.112:0/2535636941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752c10a7b0 con 0x7f752c10e3b0 2024-04-25T18:58:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.111+0000 7f7523400700 1 -- 172.21.15.112:0/2535636941 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f7524005200 con 0x7f752c10e3b0 2024-04-25T18:58:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.112+0000 7f7523400700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f75100a3e30 0x7f75100a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.112+0000 7f7523400700 1 -- 172.21.15.112:0/2535636941 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8615+0+0 (secure 0 0 0) 0x7f752400f020 con 0x7f752c10e3b0 2024-04-25T18:58:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.112+0000 7f752a800700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f75100a3e30 0x7f75100a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.113+0000 7f752a800700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f75100a3e30 0x7f75100a62f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f751c009bf0 tx=0x7f751c009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:13.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.115+0000 7f7523400700 1 -- 172.21.15.112:0/2535636941 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7524028020 con 0x7f752c10e3b0 2024-04-25T18:58:13.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.355+0000 7f7531400700 1 -- 172.21.15.112:0/2535636941 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f752c02df80 con 0x7f752c10e3b0 2024-04-25T18:58:13.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.356+0000 7f7523400700 1 -- 172.21.15.112:0/2535636941 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v107) ==== 81+0+954 (secure 0 0 0) 0x7f752409df90 con 0x7f752c10e3b0 2024-04-25T18:58:13.358 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2665612708 2024-04-26T18:47:24.619297+0000 2024-04-25T18:58:13.358 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/698167759 2024-04-26T18:47:24.619297+0000 2024-04-25T18:58:13.358 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/722306877 2024-04-26T18:47:24.619297+0000 2024-04-25T18:58:13.358 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/733062511 2024-04-26T18:34:54.948120+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/733062511 2024-04-26T18:34:54.948120+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/181862246 2024-04-26T18:34:54.948120+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3393454453 2024-04-26T18:34:20.605775+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/605196912 2024-04-26T18:47:24.619297+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3195850527 2024-04-26T18:34:20.605775+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2936464788 2024-04-26T18:47:24.619297+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/3468460974 2024-04-26T18:34:20.605775+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3392048544 2024-04-26T18:34:20.605775+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/605196912 2024-04-26T18:47:24.619297+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1329430933 2024-04-26T18:34:54.948120+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/735578423 2024-04-26T18:34:54.948120+0000 2024-04-25T18:58:13.359 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/3468460974 2024-04-26T18:34:20.605775+0000 2024-04-25T18:58:13.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 -- 172.21.15.112:0/2535636941 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f75100a3e30 msgr2=0x7f75100a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:13.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f75100a3e30 0x7f75100a62f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f751c009bf0 tx=0x7f751c009260 comp rx=0 tx=0).stop 2024-04-25T18:58:13.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 -- 172.21.15.112:0/2535636941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f752c10e3b0 msgr2=0x7f752c1a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:13.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f752c10e3b0 0x7f752c1a3fa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f752400b770 tx=0x7f7524007980 comp rx=0 tx=0).stop 2024-04-25T18:58:13.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 -- 172.21.15.112:0/2535636941 shutdown_connections 2024-04-25T18:58:13.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f75100a3e30 0x7f75100a62f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:13.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f752c10e3b0 0x7f752c1a3fa0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:13.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f752c1092d0 0x7f752c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:13.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 --2- 172.21.15.112:0/2535636941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f752c109c80 0x7f752c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:13.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 -- 172.21.15.112:0/2535636941 >> 172.21.15.112:0/2535636941 conn(0x7f752c076210 msgr2=0x7f752c076750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:13.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 -- 172.21.15.112:0/2535636941 shutdown_connections 2024-04-25T18:58:13.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:13.359+0000 7f7531400700 1 -- 172.21.15.112:0/2535636941 wait complete. 2024-04-25T18:58:13.363 INFO:teuthology.orchestra.run.smithi112.stderr:listed 16 entries 2024-04-25T18:58:14.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:13 smithi112 ceph-mon[27864]: pgmap v339: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-25T18:58:14.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:13 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2535636941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T18:58:14.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:13 smithi112 ceph-mon[27864]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T18:58:14.407 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T18:58:14.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:58:14.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T18:58:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:13 smithi171 ceph-mon[33266]: pgmap v339: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-25T18:58:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:13 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2535636941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T18:58:14.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:13 smithi171 ceph-mon[33266]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T18:58:14.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:13 smithi161 ceph-mon[32609]: pgmap v339: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-25T18:58:14.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:13 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2535636941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T18:58:14.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:13 smithi161 ceph-mon[32609]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T18:58:14.528 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T18:58:14.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:58:14.528 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T18:58:14.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:58:14.607 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T18:58:14.651 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T18:58:14.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:58:14.651 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T18:58:14.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:58:14.756 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T18:58:14.867 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T18:58:14.867 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T18:58:14.867 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T18:58:14.910 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/0ae575455491f20b954f632c1395f2aa1e72be569b82bd7d1f3a15adfbc08b9b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/BLEPWPTZIPXJR2U5WKG4HLSCHV:/var/lib/containers/storage/overlay/l/FXZ2Y7MVJLZKYO7TEZ3WJX2K57,upperdir=/var/lib/containers/storage/overlay/0ae575455491f20b954f632c1395f2aa1e72be569b82bd7d1f3a15adfbc08b9b/diff,workdir=/var/lib/containers/storage/overlay/0ae575455491f20b954f632c1395f2aa1e72be569b82bd7d1f3a15adfbc08b9b/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a37b8eeaf27c0e8a5b1f25da662511a5e0b5759c73a191eecdf480a5e2f0b1a0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/BLEPWPTZIPXJR2U5WKG4HLSCHV:/var/lib/containers/storage/overlay/l/FXZ2Y7MVJLZKYO7TEZ3WJX2K57,upperdir=/var/lib/containers/storage/overlay/a37b8eeaf27c0e8a5b1f25da662511a5e0b5759c73a191eecdf480a5e2f0b1a0/diff,workdir=/var/lib/containers/storage/overlay/a37b8eeaf27c0e8a5b1f25da662511a5e0b5759c73a191eecdf480a5e2f0b1a0/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/3efdb276c20c31cd09aa051fd08b56acad8589129d1313caa1d2136df2097780/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/BLEPWPTZIPXJR2U5WKG4HLSCHV:/var/lib/containers/storage/overlay/l/FXZ2Y7MVJLZKYO7TEZ3WJX2K57,upperdir=/var/lib/containers/storage/overlay/3efdb276c20c31cd09aa051fd08b56acad8589129d1313caa1d2136df2097780/diff,workdir=/var/lib/containers/storage/overlay/3efdb276c20c31cd09aa051fd08b56acad8589129d1313caa1d2136df2097780/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ebcc8455fa28f4b8ecd9ab0da44a117c6c1905c35e381e70118e28858dd6a1e0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/BLEPWPTZIPXJR2U5WKG4HLSCHV:/var/lib/containers/storage/overlay/l/FXZ2Y7MVJLZKYO7TEZ3WJX2K57,upperdir=/var/lib/containers/storage/overlay/ebcc8455fa28f4b8ecd9ab0da44a117c6c1905c35e381e70118e28858dd6a1e0/diff,workdir=/var/lib/containers/storage/overlay/ebcc8455fa28f4b8ecd9ab0da44a117c6c1905c35e381e70118e28858dd6a1e0/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5d39b5fbdb2bad121204043592a6874402a79822dd8a22f31ef401001ed895cf/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/BLEPWPTZIPXJR2U5WKG4HLSCHV:/var/lib/containers/storage/overlay/l/FXZ2Y7MVJLZKYO7TEZ3WJX2K57,upperdir=/var/lib/containers/storage/overlay/5d39b5fbdb2bad121204043592a6874402a79822dd8a22f31ef401001ed895cf/diff,workdir=/var/lib/containers/storage/overlay/5d39b5fbdb2bad121204043592a6874402a79822dd8a22f31ef401001ed895cf/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/425ed32ada13830735d3d589c730e8676dd71d66965927fad4f29f7183be2ae5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/BLEPWPTZIPXJR2U5WKG4HLSCHV:/var/lib/containers/storage/overlay/l/FXZ2Y7MVJLZKYO7TEZ3WJX2K57,upperdir=/var/lib/containers/storage/overlay/425ed32ada13830735d3d589c730e8676dd71d66965927fad4f29f7183be2ae5/diff,workdir=/var/lib/containers/storage/overlay/425ed32ada13830735d3d589c730e8676dd71d66965927fad4f29f7183be2ae5/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e4d559691fe3c02497e3abb7b87a743f78eb39382445b6904dba2b0a0e4ee8ec/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/BLEPWPTZIPXJR2U5WKG4HLSCHV:/var/lib/containers/storage/overlay/l/FXZ2Y7MVJLZKYO7TEZ3WJX2K57,upperdir=/var/lib/containers/storage/overlay/e4d559691fe3c02497e3abb7b87a743f78eb39382445b6904dba2b0a0e4ee8ec/diff,workdir=/var/lib/containers/storage/overlay/e4d559691fe3c02497e3abb7b87a743f78eb39382445b6904dba2b0a0e4ee8ec/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3affccbf7da1bb9fdd9fe2af5798e6946e20224aeb602af6829199a0a6e252f0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/BLEPWPTZIPXJR2U5WKG4HLSCHV:/var/lib/containers/storage/overlay/l/FXZ2Y7MVJLZKYO7TEZ3WJX2K57,upperdir=/var/lib/containers/storage/overlay/3affccbf7da1bb9fdd9fe2af5798e6946e20224aeb602af6829199a0a6e252f0/diff,workdir=/var/lib/containers/storage/overlay/3affccbf7da1bb9fdd9fe2af5798e6946e20224aeb602af6829199a0a6e252f0/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/81700da60ce04bca39c64c1d4c1dc8ff4d44e4f966b5d195d51c19427ec5df0d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/BLEPWPTZIPXJR2U5WKG4HLSCHV:/var/lib/containers/storage/overlay/l/FXZ2Y7MVJLZKYO7TEZ3WJX2K57,upperdir=/var/lib/containers/storage/overlay/81700da60ce04bca39c64c1d4c1dc8ff4d44e4f966b5d195d51c19427ec5df0d/diff,workdir=/var/lib/containers/storage/overlay/81700da60ce04bca39c64c1d4c1dc8ff4d44e4f966b5d195d51c19427ec5df0d/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 overlay /var/lib/containers/storage/overlay/27f3239c3e7bc0c756caba723a76e9b08a176ca3063f7991705fe88314ed2a07/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/BLEPWPTZIPXJR2U5WKG4HLSCHV:/var/lib/containers/storage/overlay/l/FXZ2Y7MVJLZKYO7TEZ3WJX2K57,upperdir=/var/lib/containers/storage/overlay/27f3239c3e7bc0c756caba723a76e9b08a176ca3063f7991705fe88314ed2a07/diff,workdir=/var/lib/containers/storage/overlay/27f3239c3e7bc0c756caba723a76e9b08a176ca3063f7991705fe88314ed2a07/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 2024-04-25T18:58:14.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:58:14.910 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T18:58:15.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:58:15.005 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T18:58:15.079 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-04-25T18:58:15.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:58:15.083 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T18:58:15.101 INFO:tasks.cephadm:Teardown begin 2024-04-25T18:58:15.101 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T18:58:15.101 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:58:15.551 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:16.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:15 smithi112 ceph-mon[27864]: pgmap v341: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 122 B/s wr, 3 op/s 2024-04-25T18:58:16.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:15 smithi171 ceph-mon[33266]: pgmap v341: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 122 B/s wr, 3 op/s 2024-04-25T18:58:16.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:15 smithi161 ceph-mon[32609]: pgmap v341: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 122 B/s wr, 3 op/s 2024-04-25T18:58:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.958+0000 7f8c58800700 1 -- 172.21.15.112:0/1351340484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c50108a30 msgr2=0x7f8c50108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.958+0000 7f8c58800700 1 --2- 172.21.15.112:0/1351340484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c50108a30 0x7f8c50108e10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009990 tx=0x7f8c44009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.958+0000 7f8c58800700 1 -- 172.21.15.112:0/1351340484 shutdown_connections 2024-04-25T18:58:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.959+0000 7f8c58800700 1 --2- 172.21.15.112:0/1351340484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c5006b0f0 0x7f8c5010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.959+0000 7f8c58800700 1 --2- 172.21.15.112:0/1351340484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c50108a30 0x7f8c50108e10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.959+0000 7f8c58800700 1 --2- 172.21.15.112:0/1351340484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c5006a750 0x7f8c5006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.959+0000 7f8c58800700 1 -- 172.21.15.112:0/1351340484 >> 172.21.15.112:0/1351340484 conn(0x7f8c500788f0 msgr2=0x7f8c50078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.959+0000 7f8c58800700 1 -- 172.21.15.112:0/1351340484 shutdown_connections 2024-04-25T18:58:17.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.959+0000 7f8c58800700 1 -- 172.21.15.112:0/1351340484 wait complete. 2024-04-25T18:58:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.960+0000 7f8c58800700 1 Processor -- start 2024-04-25T18:58:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.960+0000 7f8c58800700 1 -- start start 2024-04-25T18:58:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.960+0000 7f8c58800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c5006a750 0x7f8c50101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.960+0000 7f8c58800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c5006b0f0 0x7f8c50101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.960+0000 7f8c58800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c50108a30 0x7f8c50102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.960+0000 7f8c58800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50112c20 con 0x7f8c50108a30 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.960+0000 7f8c58800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50112aa0 con 0x7f8c5006a750 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.960+0000 7f8c58800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50112da0 con 0x7f8c5006b0f0 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.960+0000 7f8c55400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c5006b0f0 0x7f8c50101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.960+0000 7f8c56800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c50108a30 0x7f8c50102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.961+0000 7f8c55e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c5006a750 0x7f8c50101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.961+0000 7f8c55400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c5006b0f0 0x7f8c50101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:59390/0 (socket says 172.21.15.112:59390) 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.961+0000 7f8c55400700 1 -- 172.21.15.112:0/3309693594 learned_addr learned my addr 172.21.15.112:0/3309693594 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.961+0000 7f8c55400700 1 -- 172.21.15.112:0/3309693594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c5006a750 msgr2=0x7f8c50101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.961+0000 7f8c55400700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c5006a750 0x7f8c50101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.961+0000 7f8c55400700 1 -- 172.21.15.112:0/3309693594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c50108a30 msgr2=0x7f8c50102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.961+0000 7f8c55400700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c50108a30 0x7f8c50102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.961+0000 7f8c55400700 1 -- 172.21.15.112:0/3309693594 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c501a7630 con 0x7f8c5006b0f0 2024-04-25T18:58:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.961+0000 7f8c56800700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c50108a30 0x7f8c50102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.961+0000 7f8c55400700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c5006b0f0 0x7f8c50101f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000e9a0 tx=0x7f8c4000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.962+0000 7f8c4aa00700 1 -- 172.21.15.112:0/3309693594 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4000ca10 con 0x7f8c5006b0f0 2024-04-25T18:58:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.962+0000 7f8c58800700 1 -- 172.21.15.112:0/3309693594 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c501a7920 con 0x7f8c5006b0f0 2024-04-25T18:58:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.962+0000 7f8c4aa00700 1 -- 172.21.15.112:0/3309693594 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c4000cbb0 con 0x7f8c5006b0f0 2024-04-25T18:58:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.962+0000 7f8c4aa00700 1 -- 172.21.15.112:0/3309693594 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40018680 con 0x7f8c5006b0f0 2024-04-25T18:58:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.962+0000 7f8c58800700 1 -- 172.21.15.112:0/3309693594 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c501a7eb0 con 0x7f8c5006b0f0 2024-04-25T18:58:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.963+0000 7f8c58800700 1 -- 172.21.15.112:0/3309693594 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c50069730 con 0x7f8c5006b0f0 2024-04-25T18:58:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.965+0000 7f8c4aa00700 1 -- 172.21.15.112:0/3309693594 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f8c40010430 con 0x7f8c5006b0f0 2024-04-25T18:58:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.966+0000 7f8c4aa00700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8c380a3e30 0x7f8c380a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.966+0000 7f8c4aa00700 1 -- 172.21.15.112:0/3309693594 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8571+0+0 (secure 0 0 0) 0x7f8c400d0990 con 0x7f8c5006b0f0 2024-04-25T18:58:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.967+0000 7f8c55e00700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8c380a3e30 0x7f8c380a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:17.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.967+0000 7f8c55e00700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8c380a3e30 0x7f8c380a62f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006010 tx=0x7f8c4400b260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:17.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:17.971+0000 7f8c4aa00700 1 -- 172.21.15.112:0/3309693594 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c4009cfa0 con 0x7f8c5006b0f0 2024-04-25T18:58:18.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:17 smithi112 ceph-mon[27864]: pgmap v342: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 818 B/s wr, 7 op/s 2024-04-25T18:58:18.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.246+0000 7f8c58800700 1 -- 172.21.15.112:0/3309693594 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c5002df80 con 0x7f8c5006b0f0 2024-04-25T18:58:18.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.248+0000 7f8c4aa00700 1 -- 172.21.15.112:0/3309693594 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11895 (secure 0 0 0) 0x7f8c4009c8a0 con 0x7f8c5006b0f0 2024-04-25T18:58:18.251 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:58:18.251 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:56:28.660237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25739":{"gid":25739,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3065773788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3065773788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3065773788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24493,24511,24512]},"id":1}]} 2024-04-25T18:58:18.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 -- 172.21.15.112:0/3309693594 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8c380a3e30 msgr2=0x7f8c380a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:18.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8c380a3e30 0x7f8c380a62f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006010 tx=0x7f8c4400b260 comp rx=0 tx=0).stop 2024-04-25T18:58:18.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 -- 172.21.15.112:0/3309693594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c5006b0f0 msgr2=0x7f8c50101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:18.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c5006b0f0 0x7f8c50101f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000e9a0 tx=0x7f8c4000ecb0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 -- 172.21.15.112:0/3309693594 shutdown_connections 2024-04-25T18:58:18.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f8c380a3e30 0x7f8c380a62f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c50108a30 0x7f8c50102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c5006a750 0x7f8c50101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 --2- 172.21.15.112:0/3309693594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c5006b0f0 0x7f8c50101f50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 -- 172.21.15.112:0/3309693594 >> 172.21.15.112:0/3309693594 conn(0x7f8c500788f0 msgr2=0x7f8c500ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 -- 172.21.15.112:0/3309693594 shutdown_connections 2024-04-25T18:58:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:18.252+0000 7f8c58800700 1 -- 172.21.15.112:0/3309693594 wait complete. 2024-04-25T18:58:18.255 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 38 2024-04-25T18:58:18.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:17 smithi171 ceph-mon[33266]: pgmap v342: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 818 B/s wr, 7 op/s 2024-04-25T18:58:18.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:17 smithi161 ceph-mon[32609]: pgmap v342: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 818 B/s wr, 7 op/s 2024-04-25T18:58:19.127 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:58:19.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:18 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3309693594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:19.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:18 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3309693594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:19.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:18 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3309693594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:19.558 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:20.373 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:19 smithi161 ceph-mon[32609]: pgmap v343: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 818 B/s wr, 7 op/s 2024-04-25T18:58:20.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:19 smithi171 ceph-mon[33266]: pgmap v343: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 818 B/s wr, 7 op/s 2024-04-25T18:58:20.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:19 smithi112 ceph-mon[27864]: pgmap v343: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 818 B/s wr, 7 op/s 2024-04-25T18:58:21.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.894+0000 7f88b8e00700 1 -- 172.21.15.112:0/1906678039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88b4108f80 msgr2=0x7f88b4109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:21.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.894+0000 7f88b8e00700 1 --2- 172.21.15.112:0/1906678039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88b4108f80 0x7f88b4109400 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f889c009990 tx=0x7f889c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.894+0000 7f88b8e00700 1 -- 172.21.15.112:0/1906678039 shutdown_connections 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.894+0000 7f88b8e00700 1 --2- 172.21.15.112:0/1906678039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88b4109940 0x7f88b4115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.895+0000 7f88b8e00700 1 --2- 172.21.15.112:0/1906678039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b410f030 0x7f88b410f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.895+0000 7f88b8e00700 1 --2- 172.21.15.112:0/1906678039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88b4108f80 0x7f88b4109400 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.895+0000 7f88b8e00700 1 -- 172.21.15.112:0/1906678039 >> 172.21.15.112:0/1906678039 conn(0x7f88b40787d0 msgr2=0x7f88b4107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.895+0000 7f88b8e00700 1 -- 172.21.15.112:0/1906678039 shutdown_connections 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.895+0000 7f88b8e00700 1 -- 172.21.15.112:0/1906678039 wait complete. 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.896+0000 7f88b8e00700 1 Processor -- start 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.896+0000 7f88b8e00700 1 -- start start 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b8e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b4108f80 0x7f88b419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b8e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88b4109940 0x7f88b419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:21.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b8e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88b410f030 0x7f88b41a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b8e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4119120 con 0x7f88b4109940 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b8e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4118fa0 con 0x7f88b4108f80 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b8e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41192a0 con 0x7f88b410f030 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b3400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b4108f80 0x7f88b419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b3400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b4108f80 0x7f88b419f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.112:56246/0 (socket says 172.21.15.112:56246) 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88aaa00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88b4109940 0x7f88b419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b3400700 1 -- 172.21.15.112:0/3155895775 learned_addr learned my addr 172.21.15.112:0/3155895775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b3e00700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88b410f030 0x7f88b41a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b3400700 1 -- 172.21.15.112:0/3155895775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88b410f030 msgr2=0x7f88b41a3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b3400700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88b410f030 0x7f88b41a3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b3400700 1 -- 172.21.15.112:0/3155895775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88b4109940 msgr2=0x7f88b419fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b3400700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88b4109940 0x7f88b419fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:21.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.897+0000 7f88b3400700 1 -- 172.21.15.112:0/3155895775 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b41a4580 con 0x7f88b4108f80 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.898+0000 7f88b3400700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b4108f80 0x7f88b419f570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c370 tx=0x7f88a400c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.898+0000 7f88b0c00700 1 -- 172.21.15.112:0/3155895775 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a400cea0 con 0x7f88b4108f80 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.898+0000 7f88b8e00700 1 -- 172.21.15.112:0/3155895775 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b41a4810 con 0x7f88b4108f80 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.898+0000 7f88b0c00700 1 -- 172.21.15.112:0/3155895775 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a40090d0 con 0x7f88b4108f80 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.898+0000 7f88b0c00700 1 -- 172.21.15.112:0/3155895775 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4004770 con 0x7f88b4108f80 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.898+0000 7f88b8e00700 1 -- 172.21.15.112:0/3155895775 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b41ac0f0 con 0x7f88b4108f80 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.899+0000 7f88b8e00700 1 -- 172.21.15.112:0/3155895775 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b4069730 con 0x7f88b4108f80 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.901+0000 7f88b0c00700 1 -- 172.21.15.112:0/3155895775 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f88a401a020 con 0x7f88b4108f80 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.903+0000 7f88b0c00700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88980a3e30 0x7f88980a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.903+0000 7f88b0c00700 1 -- 172.21.15.112:0/3155895775 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8571+0+0 (secure 0 0 0) 0x7f88a40d04e0 con 0x7f88b4108f80 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.903+0000 7f88aaa00700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88980a3e30 0x7f88980a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.904+0000 7f88aaa00700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88980a3e30 0x7f88980a62f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f889c0098e0 tx=0x7f889c005960 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:21.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:21.905+0000 7f88b0c00700 1 -- 172.21.15.112:0/3155895775 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88a4010040 con 0x7f88b4108f80 2024-04-25T18:58:22.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.183+0000 7f88b8e00700 1 -- 172.21.15.112:0/3155895775 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88b402df80 con 0x7f88b4108f80 2024-04-25T18:58:22.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.184+0000 7f88b0c00700 1 -- 172.21.15.112:0/3155895775 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11895 (secure 0 0 0) 0x7f88a409caf0 con 0x7f88b4108f80 2024-04-25T18:58:22.187 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:58:22.187 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1467985525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1467985525},{"type":"v1","addr":"172.21.15.171:6833","nonce":1467985525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3214695167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3214695167},{"type":"v1","addr":"172.21.15.161:6833","nonce":3214695167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/139408672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":139408672},{"type":"v1","addr":"172.21.15.112:6835","nonce":139408672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/2558651587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":2558651587},{"type":"v1","addr":"172.21.15.112:6837","nonce":2558651587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3751012536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3751012536},{"type":"v1","addr":"172.21.15.161:6835","nonce":3751012536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2423338358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2423338358},{"type":"v1","addr":"172.21.15.171:6835","nonce":2423338358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:43:19.347467+0000","modified":"2024-04-25T18:56:28.660237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24511,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4007863322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4007863322},{"type":"v1","addr":"172.21.15.161:6837","nonce":4007863322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/3522712423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3522712423},{"type":"v1","addr":"172.21.15.171:6837","nonce":3522712423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.171:6839/1642228003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1642228003},{"type":"v1","addr":"172.21.15.171:6839","nonce":1642228003}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.112:6841/1028859850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":1028859850},{"type":"v1","addr":"172.21.15.112:6841","nonce":1028859850}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.161:6839/2315334000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2315334000},{"type":"v1","addr":"172.21.15.161:6839","nonce":2315334000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25739":{"gid":25739,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/3065773788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3065773788},{"type":"v1","addr":"172.21.15.112:6839","nonce":3065773788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24493,24511,24512]},"id":1}]} 2024-04-25T18:58:22.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.187+0000 7f88b8e00700 1 -- 172.21.15.112:0/3155895775 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88980a3e30 msgr2=0x7f88980a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:22.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.187+0000 7f88b8e00700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88980a3e30 0x7f88980a62f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f889c0098e0 tx=0x7f889c005960 comp rx=0 tx=0).stop 2024-04-25T18:58:22.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.187+0000 7f88b8e00700 1 -- 172.21.15.112:0/3155895775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b4108f80 msgr2=0x7f88b419f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:22.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.187+0000 7f88b8e00700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b4108f80 0x7f88b419f570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c370 tx=0x7f88a400c680 comp rx=0 tx=0).stop 2024-04-25T18:58:22.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.187+0000 7f88b8e00700 1 -- 172.21.15.112:0/3155895775 shutdown_connections 2024-04-25T18:58:22.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.187+0000 7f88b8e00700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f88980a3e30 0x7f88980a62f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.187+0000 7f88b8e00700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f88b410f030 0x7f88b41a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.187+0000 7f88b8e00700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b4108f80 0x7f88b419f570 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.187+0000 7f88b8e00700 1 --2- 172.21.15.112:0/3155895775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88b4109940 0x7f88b419fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.187+0000 7f88b8e00700 1 -- 172.21.15.112:0/3155895775 >> 172.21.15.112:0/3155895775 conn(0x7f88b40787d0 msgr2=0x7f88b4079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:22.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.188+0000 7f88b8e00700 1 -- 172.21.15.112:0/3155895775 shutdown_connections 2024-04-25T18:58:22.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:22.188+0000 7f88b8e00700 1 -- 172.21.15.112:0/3155895775 wait complete. 2024-04-25T18:58:22.191 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 38 2024-04-25T18:58:22.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:22 smithi112 ceph-mon[27864]: pgmap v344: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-25T18:58:22.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:22 smithi171 ceph-mon[33266]: pgmap v344: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-25T18:58:22.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:22 smithi161 ceph-mon[32609]: pgmap v344: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-25T18:58:23.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:58:23.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:23 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/3155895775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:23.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:23 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/3155895775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:23.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:23 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/3155895775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:23.615 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:24.019 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:24 smithi112 ceph-mon[27864]: pgmap v345: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-25T18:58:24.333 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:24 smithi171 ceph-mon[33266]: pgmap v345: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-25T18:58:24.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:24 smithi161 ceph-mon[32609]: pgmap v345: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-25T18:58:25.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.935+0000 7f2c44800700 1 -- 172.21.15.112:0/3070268615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c3c10e3b0 msgr2=0x7f2c3c115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:25.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.935+0000 7f2c44800700 1 --2- 172.21.15.112:0/3070268615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c3c10e3b0 0x7f2c3c115f80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800b0d0 tx=0x7f2c3800b3e0 comp rx=0 tx=0).stop 2024-04-25T18:58:25.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.935+0000 7f2c44800700 1 -- 172.21.15.112:0/3070268615 shutdown_connections 2024-04-25T18:58:25.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.935+0000 7f2c44800700 1 --2- 172.21.15.112:0/3070268615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c3c10e3b0 0x7f2c3c115f80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:25.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.935+0000 7f2c44800700 1 --2- 172.21.15.112:0/3070268615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c3c1092d0 0x7f2c3c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:25.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.935+0000 7f2c44800700 1 --2- 172.21.15.112:0/3070268615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c3c109c80 0x7f2c3c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:25.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.935+0000 7f2c44800700 1 -- 172.21.15.112:0/3070268615 >> 172.21.15.112:0/3070268615 conn(0x7f2c3c076210 msgr2=0x7f2c3c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:25.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.935+0000 7f2c44800700 1 -- 172.21.15.112:0/3070268615 shutdown_connections 2024-04-25T18:58:25.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.936+0000 7f2c44800700 1 -- 172.21.15.112:0/3070268615 wait complete. 2024-04-25T18:58:25.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.937+0000 7f2c44800700 1 Processor -- start 2024-04-25T18:58:25.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.937+0000 7f2c44800700 1 -- start start 2024-04-25T18:58:25.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.937+0000 7f2c44800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c3c1092d0 0x7f2c3c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:25.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.937+0000 7f2c44800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c3c109c80 0x7f2c3c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:25.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.937+0000 7f2c44800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c3c10e3b0 0x7f2c3c1a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:25.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.937+0000 7f2c44800700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1192a0 con 0x7f2c3c1092d0 2024-04-25T18:58:25.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.937+0000 7f2c44800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c119120 con 0x7f2c3c10e3b0 2024-04-25T18:58:25.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.937+0000 7f2c44800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c119420 con 0x7f2c3c109c80 2024-04-25T18:58:25.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.938+0000 7f2c41400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c3c109c80 0x7f2c3c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:25.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.938+0000 7f2c41e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c3c1092d0 0x7f2c3c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:25.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.938+0000 7f2c41400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c3c109c80 0x7f2c3c19fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.112:36490/0 (socket says 172.21.15.112:36490) 2024-04-25T18:58:25.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.938+0000 7f2c41400700 1 -- 172.21.15.112:0/2136981522 learned_addr learned my addr 172.21.15.112:0/2136981522 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:58:25.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.938+0000 7f2c42800700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c3c10e3b0 0x7f2c3c1a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:25.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.938+0000 7f2c41400700 1 -- 172.21.15.112:0/2136981522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c3c10e3b0 msgr2=0x7f2c3c1a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:25.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.938+0000 7f2c41400700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c3c10e3b0 0x7f2c3c1a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:25.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.938+0000 7f2c41400700 1 -- 172.21.15.112:0/2136981522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c3c1092d0 msgr2=0x7f2c3c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:25.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.938+0000 7f2c41400700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c3c1092d0 0x7f2c3c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:25.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.938+0000 7f2c41400700 1 -- 172.21.15.112:0/2136981522 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c1a4740 con 0x7f2c3c109c80 2024-04-25T18:58:25.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.939+0000 7f2c41400700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c3c109c80 0x7f2c3c19fc70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00e970 tx=0x7f2c2c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:25.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.939+0000 7f2c36a00700 1 -- 172.21.15.112:0/2136981522 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c004a30 con 0x7f2c3c109c80 2024-04-25T18:58:25.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.939+0000 7f2c36a00700 1 -- 172.21.15.112:0/2136981522 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2c00cc90 con 0x7f2c3c109c80 2024-04-25T18:58:25.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.939+0000 7f2c44800700 1 -- 172.21.15.112:0/2136981522 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c1a4a30 con 0x7f2c3c109c80 2024-04-25T18:58:25.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.939+0000 7f2c36a00700 1 -- 172.21.15.112:0/2136981522 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c005610 con 0x7f2c3c109c80 2024-04-25T18:58:25.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.939+0000 7f2c44800700 1 -- 172.21.15.112:0/2136981522 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c1ac2b0 con 0x7f2c3c109c80 2024-04-25T18:58:25.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.940+0000 7f2c44800700 1 -- 172.21.15.112:0/2136981522 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3c069730 con 0x7f2c3c109c80 2024-04-25T18:58:25.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.942+0000 7f2c36a00700 1 -- 172.21.15.112:0/2136981522 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f2c2c00c7e0 con 0x7f2c3c109c80 2024-04-25T18:58:25.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.943+0000 7f2c36a00700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a3db0 0x7f2c240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:25.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.944+0000 7f2c36a00700 1 -- 172.21.15.112:0/2136981522 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8571+0+0 (secure 0 0 0) 0x7f2c2c0d0ba0 con 0x7f2c3c109c80 2024-04-25T18:58:25.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.944+0000 7f2c41e00700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a3db0 0x7f2c240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:25.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.944+0000 7f2c41e00700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a3db0 0x7f2c240a6270 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009c60 tx=0x7f2c30009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:25.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:25.945+0000 7f2c36a00700 1 -- 172.21.15.112:0/2136981522 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c2c053ff0 con 0x7f2c3c109c80 2024-04-25T18:58:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.173+0000 7f2c44800700 1 -- 172.21.15.112:0/2136981522 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c3c02df80 con 0x7f2c3c109c80 2024-04-25T18:58:26.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.175+0000 7f2c36a00700 1 -- 172.21.15.112:0/2136981522 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+23182 (secure 0 0 0) 0x7f2c2c09d1b0 con 0x7f2c3c109c80 2024-04-25T18:58:26.177 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T18:58:26.178 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":108,"fsid":"2fe18a52-0332-11ef-bc93-c7b262605968","created":"2024-04-25T18:33:37.379089+0000","modified":"2024-04-25T18:58:13.885249+0000","last_up_change":"2024-04-25T18:42:00.961646+0000","last_in_change":"2024-04-25T18:41:40.647162+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:37:45.106199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:43:11.017975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:43:15.116917+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efb4bc01-e29b-4402-8a7a-3387d1731ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6803","nonce":232254535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6805","nonce":232254535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6809","nonce":232254535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":232254535},{"type":"v1","addr":"172.21.15.112:6807","nonce":232254535}]},"public_addr":"172.21.15.112:6803/232254535","cluster_addr":"172.21.15.112:6805/232254535","heartbeat_back_addr":"172.21.15.112:6809/232254535","heartbeat_front_addr":"172.21.15.112:6807/232254535","state":["exists","up"]},{"osd":1,"uuid":"65a83fa2-96a6-4743-8c17-21531f5e885e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6801","nonce":664224744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6803","nonce":664224744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6807","nonce":664224744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":664224744},{"type":"v1","addr":"172.21.15.161:6805","nonce":664224744}]},"public_addr":"172.21.15.161:6801/664224744","cluster_addr":"172.21.15.161:6803/664224744","heartbeat_back_addr":"172.21.15.161:6807/664224744","heartbeat_front_addr":"172.21.15.161:6805/664224744","state":["exists","up"]},{"osd":2,"uuid":"7562ec4c-c126-48ec-a51c-56fa461a2e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6801","nonce":345172073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6803","nonce":345172073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6807","nonce":345172073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":345172073},{"type":"v1","addr":"172.21.15.171:6805","nonce":345172073}]},"public_addr":"172.21.15.171:6801/345172073","cluster_addr":"172.21.15.171:6803/345172073","heartbeat_back_addr":"172.21.15.171:6807/345172073","heartbeat_front_addr":"172.21.15.171:6805/345172073","state":["exists","up"]},{"osd":3,"uuid":"d7aae8cd-fec7-46d9-8773-25eaaca3cc78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6811","nonce":1472407156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6813","nonce":1472407156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6817","nonce":1472407156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1472407156},{"type":"v1","addr":"172.21.15.112:6815","nonce":1472407156}]},"public_addr":"172.21.15.112:6811/1472407156","cluster_addr":"172.21.15.112:6813/1472407156","heartbeat_back_addr":"172.21.15.112:6817/1472407156","heartbeat_front_addr":"172.21.15.112:6815/1472407156","state":["exists","up"]},{"osd":4,"uuid":"b8bf57e7-09d0-4b1b-ba86-e39f3a4970e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6809","nonce":373204306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6811","nonce":373204306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6815","nonce":373204306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":373204306},{"type":"v1","addr":"172.21.15.161:6813","nonce":373204306}]},"public_addr":"172.21.15.161:6809/373204306","cluster_addr":"172.21.15.161:6811/373204306","heartbeat_back_addr":"172.21.15.161:6815/373204306","heartbeat_front_addr":"172.21.15.161:6813/373204306","state":["exists","up"]},{"osd":5,"uuid":"56124d12-bc1f-4cba-8430-938248851776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6809","nonce":2643512313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6811","nonce":2643512313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6815","nonce":2643512313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2643512313},{"type":"v1","addr":"172.21.15.171:6813","nonce":2643512313}]},"public_addr":"172.21.15.171:6809/2643512313","cluster_addr":"172.21.15.171:6811/2643512313","heartbeat_back_addr":"172.21.15.171:6815/2643512313","heartbeat_front_addr":"172.21.15.171:6813/2643512313","state":["exists","up"]},{"osd":6,"uuid":"9563a9d3-352b-4686-975c-282ec167248b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6819","nonce":1933654933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6821","nonce":1933654933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6825","nonce":1933654933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1933654933},{"type":"v1","addr":"172.21.15.112:6823","nonce":1933654933}]},"public_addr":"172.21.15.112:6819/1933654933","cluster_addr":"172.21.15.112:6821/1933654933","heartbeat_back_addr":"172.21.15.112:6825/1933654933","heartbeat_front_addr":"172.21.15.112:6823/1933654933","state":["exists","up"]},{"osd":7,"uuid":"f2d06e86-6aac-40f4-956e-18fe0b0e3543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6817","nonce":3060201138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6819","nonce":3060201138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6823","nonce":3060201138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":3060201138},{"type":"v1","addr":"172.21.15.161:6821","nonce":3060201138}]},"public_addr":"172.21.15.161:6817/3060201138","cluster_addr":"172.21.15.161:6819/3060201138","heartbeat_back_addr":"172.21.15.161:6823/3060201138","heartbeat_front_addr":"172.21.15.161:6821/3060201138","state":["exists","up"]},{"osd":8,"uuid":"f006e45f-2bb9-45d0-9020-be7c33c6a825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6817","nonce":835756571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6819","nonce":835756571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6823","nonce":835756571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":835756571},{"type":"v1","addr":"172.21.15.171:6821","nonce":835756571}]},"public_addr":"172.21.15.171:6817/835756571","cluster_addr":"172.21.15.171:6819/835756571","heartbeat_back_addr":"172.21.15.171:6823/835756571","heartbeat_front_addr":"172.21.15.171:6821/835756571","state":["exists","up"]},{"osd":9,"uuid":"c5a96491-4151-4633-b3bc-2c214fe51c43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6827","nonce":349758528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6829","nonce":349758528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6833","nonce":349758528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":349758528},{"type":"v1","addr":"172.21.15.112:6831","nonce":349758528}]},"public_addr":"172.21.15.112:6827/349758528","cluster_addr":"172.21.15.112:6829/349758528","heartbeat_back_addr":"172.21.15.112:6833/349758528","heartbeat_front_addr":"172.21.15.112:6831/349758528","state":["exists","up"]},{"osd":10,"uuid":"244eda3e-25a0-490e-879f-c1e2726c1be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6825","nonce":3013292765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6827","nonce":3013292765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6831","nonce":3013292765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3013292765},{"type":"v1","addr":"172.21.15.161:6829","nonce":3013292765}]},"public_addr":"172.21.15.161:6825/3013292765","cluster_addr":"172.21.15.161:6827/3013292765","heartbeat_back_addr":"172.21.15.161:6831/3013292765","heartbeat_front_addr":"172.21.15.161:6829/3013292765","state":["exists","up"]},{"osd":11,"uuid":"356e5c54-bbc3-45ee-bda2-85164ce3cb92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6825","nonce":1010336458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6827","nonce":1010336458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6831","nonce":1010336458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1010336458},{"type":"v1","addr":"172.21.15.171:6829","nonce":1010336458}]},"public_addr":"172.21.15.171:6825/1010336458","cluster_addr":"172.21.15.171:6827/1010336458","heartbeat_back_addr":"172.21.15.171:6831/1010336458","heartbeat_front_addr":"172.21.15.171:6829/1010336458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:36:51.024650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:17.026638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:37:41.239726+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:15.148968+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:38:41.029118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:06.225858+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:39:41.639444+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:07.753400+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:40:32.292085+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:07.159926+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:33.749782+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:41:58.594447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/2665612708":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/698167759":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/722306877":"2024-04-26T18:47:24.619297+0000","172.21.15.112:6801/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/733062511":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/181862246":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/3393454453":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6800/605196912":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/3195850527":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/2936464788":"2024-04-26T18:47:24.619297+0000","172.21.15.112:6801/3468460974":"2024-04-26T18:34:20.605775+0000","172.21.15.112:0/3392048544":"2024-04-26T18:34:20.605775+0000","172.21.15.112:6801/605196912":"2024-04-26T18:47:24.619297+0000","172.21.15.112:0/1329430933":"2024-04-26T18:34:54.948120+0000","172.21.15.112:0/735578423":"2024-04-26T18:34:54.948120+0000","172.21.15.112:6800/3468460974":"2024-04-26T18:34:20.605775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:58:26.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 -- 172.21.15.112:0/2136981522 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a3db0 msgr2=0x7f2c240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:26.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a3db0 0x7f2c240a6270 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009c60 tx=0x7f2c30009260 comp rx=0 tx=0).stop 2024-04-25T18:58:26.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 -- 172.21.15.112:0/2136981522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c3c109c80 msgr2=0x7f2c3c19fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:26.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c3c109c80 0x7f2c3c19fc70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00e970 tx=0x7f2c2c00ec80 comp rx=0 tx=0).stop 2024-04-25T18:58:26.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 -- 172.21.15.112:0/2136981522 shutdown_connections 2024-04-25T18:58:26.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f2c240a3db0 0x7f2c240a6270 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c3c10e3b0 0x7f2c3c1a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c3c1092d0 0x7f2c3c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 --2- 172.21.15.112:0/2136981522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c3c109c80 0x7f2c3c19fc70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 -- 172.21.15.112:0/2136981522 >> 172.21.15.112:0/2136981522 conn(0x7f2c3c076210 msgr2=0x7f2c3c076790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 -- 172.21.15.112:0/2136981522 shutdown_connections 2024-04-25T18:58:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:26.178+0000 7f2c44800700 1 -- 172.21.15.112:0/2136981522 wait complete. 2024-04-25T18:58:26.434 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:26 smithi171 ceph-mon[33266]: pgmap v346: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 650 B/s wr, 6 op/s 2024-04-25T18:58:26.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:26 smithi161 ceph-mon[32609]: pgmap v346: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 650 B/s wr, 6 op/s 2024-04-25T18:58:26.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:26 smithi112 ceph-mon[27864]: pgmap v346: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 650 B/s wr, 6 op/s 2024-04-25T18:58:27.128 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs fail cephfs 2024-04-25T18:58:27.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:27 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/2136981522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:27.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:27 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/2136981522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:27.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:27 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/2136981522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:27.585 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:28.410 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:28 smithi112 ceph-mon[27864]: pgmap v347: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T18:58:28.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:28 smithi171 ceph-mon[33266]: pgmap v347: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T18:58:28.474 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:28 smithi161 ceph-mon[32609]: pgmap v347: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T18:58:29.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.955+0000 7fdad4e00700 1 -- 172.21.15.112:0/3458244729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdad0101ad0 msgr2=0x7fdad01117c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:29.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.955+0000 7fdad4e00700 1 --2- 172.21.15.112:0/3458244729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdad0101ad0 0x7fdad01117c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdac800b0a0 tx=0x7fdac800b3b0 comp rx=0 tx=0).stop 2024-04-25T18:58:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.955+0000 7fdad4e00700 1 -- 172.21.15.112:0/3458244729 shutdown_connections 2024-04-25T18:58:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.955+0000 7fdad4e00700 1 --2- 172.21.15.112:0/3458244729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdad0101ad0 0x7fdad01117c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.955+0000 7fdad4e00700 1 --2- 172.21.15.112:0/3458244729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdad0103950 0x7fdad0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.955+0000 7fdad4e00700 1 --2- 172.21.15.112:0/3458244729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0101030 0x7fdad01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.955+0000 7fdad4e00700 1 -- 172.21.15.112:0/3458244729 >> 172.21.15.112:0/3458244729 conn(0x7fdad0077fb0 msgr2=0x7fdad00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.955+0000 7fdad4e00700 1 -- 172.21.15.112:0/3458244729 shutdown_connections 2024-04-25T18:58:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.955+0000 7fdad4e00700 1 -- 172.21.15.112:0/3458244729 wait complete. 2024-04-25T18:58:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.956+0000 7fdad4e00700 1 Processor -- start 2024-04-25T18:58:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdad4e00700 1 -- start start 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdad4e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdad0101030 0x7fdad019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdad4e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0101ad0 0x7fdad019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdad4e00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdad0103950 0x7fdad019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdad4e00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0114e00 con 0x7fdad0103950 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdad4e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0114c80 con 0x7fdad0101ad0 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdad4e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0114f80 con 0x7fdad0101030 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdace800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdad0103950 0x7fdad019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdace800700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdad0103950 0x7fdad019fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33378/0 (socket says 172.21.15.112:33378) 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdacd400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0101ad0 0x7fdad019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdace800700 1 -- 172.21.15.112:0/229449741 learned_addr learned my addr 172.21.15.112:0/229449741 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.957+0000 7fdacde00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdad0101030 0x7fdad019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.958+0000 7fdace800700 1 -- 172.21.15.112:0/229449741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdad0101030 msgr2=0x7fdad019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.958+0000 7fdace800700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdad0101030 0x7fdad019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.958+0000 7fdace800700 1 -- 172.21.15.112:0/229449741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0101ad0 msgr2=0x7fdad019b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.958+0000 7fdace800700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0101ad0 0x7fdad019b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.958+0000 7fdace800700 1 -- 172.21.15.112:0/229449741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad01a02a0 con 0x7fdad0103950 2024-04-25T18:58:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.958+0000 7fdacd400700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0101ad0 0x7fdad019b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.958+0000 7fdace800700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdad0103950 0x7fdad019fb60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdac800b780 tx=0x7fdac8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.958+0000 7fdac6a00700 1 -- 172.21.15.112:0/229449741 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8021070 con 0x7fdad0103950 2024-04-25T18:58:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.959+0000 7fdac6a00700 1 -- 172.21.15.112:0/229449741 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac8007e30 con 0x7fdad0103950 2024-04-25T18:58:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.959+0000 7fdad4e00700 1 -- 172.21.15.112:0/229449741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad01a0530 con 0x7fdad0103950 2024-04-25T18:58:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.959+0000 7fdac6a00700 1 -- 172.21.15.112:0/229449741 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8004700 con 0x7fdad0103950 2024-04-25T18:58:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.959+0000 7fdad4e00700 1 -- 172.21.15.112:0/229449741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad01a7e10 con 0x7fdad0103950 2024-04-25T18:58:29.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.960+0000 7fdad4e00700 1 -- 172.21.15.112:0/229449741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad0102040 con 0x7fdad0103950 2024-04-25T18:58:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.962+0000 7fdac6a00700 1 -- 172.21.15.112:0/229449741 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fdac8007980 con 0x7fdad0103950 2024-04-25T18:58:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.963+0000 7fdac6a00700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdab40a3db0 0x7fdab40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.963+0000 7fdac6a00700 1 -- 172.21.15.112:0/229449741 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8571+0+0 (secure 0 0 0) 0x7fdac800f020 con 0x7fdad0103950 2024-04-25T18:58:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.963+0000 7fdacde00700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdab40a3db0 0x7fdab40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.964+0000 7fdacde00700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdab40a3db0 0x7fdab40a6270 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009be0 tx=0x7fdac0009260 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:29.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:29.968+0000 7fdac6a00700 1 -- 172.21.15.112:0/229449741 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdac802a020 con 0x7fdad0103950 2024-04-25T18:58:30.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:30 smithi112 ceph-mon[27864]: pgmap v348: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:58:30.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:30 smithi112 ceph-mon[27864]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:58:30.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.242+0000 7fdad4e00700 1 -- 172.21.15.112:0/229449741 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdad002df80 con 0x7fdad0103950 2024-04-25T18:58:30.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.248+0000 7fdac6a00700 1 -- 172.21.15.112:0/229449741 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) ==== 161+0+0 (secure 0 0 0) 0x7fdac809e300 con 0x7fdad0103950 2024-04-25T18:58:30.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.251+0000 7fdad4e00700 1 -- 172.21.15.112:0/229449741 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdab40a3db0 msgr2=0x7fdab40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:30.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.251+0000 7fdad4e00700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdab40a3db0 0x7fdab40a6270 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009be0 tx=0x7fdac0009260 comp rx=0 tx=0).stop 2024-04-25T18:58:30.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.251+0000 7fdad4e00700 1 -- 172.21.15.112:0/229449741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdad0103950 msgr2=0x7fdad019fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:30.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.251+0000 7fdad4e00700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdad0103950 0x7fdad019fb60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdac800b780 tx=0x7fdac8004d00 comp rx=0 tx=0).stop 2024-04-25T18:58:30.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.252+0000 7fdad4e00700 1 -- 172.21.15.112:0/229449741 shutdown_connections 2024-04-25T18:58:30.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.252+0000 7fdad4e00700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7fdab40a3db0 0x7fdab40a6270 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.252+0000 7fdad4e00700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdad0103950 0x7fdad019fb60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.252+0000 7fdad4e00700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdad0101030 0x7fdad019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.252+0000 7fdad4e00700 1 --2- 172.21.15.112:0/229449741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0101ad0 0x7fdad019b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.252+0000 7fdad4e00700 1 -- 172.21.15.112:0/229449741 >> 172.21.15.112:0/229449741 conn(0x7fdad0077fb0 msgr2=0x7fdad00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:30.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.252+0000 7fdad4e00700 1 -- 172.21.15.112:0/229449741 shutdown_connections 2024-04-25T18:58:30.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:30.252+0000 7fdad4e00700 1 -- 172.21.15.112:0/229449741 wait complete. 2024-04-25T18:58:30.255 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T18:58:30.299 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:30 smithi161 ceph-mon[32609]: pgmap v348: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:58:30.299 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:30 smithi161 ceph-mon[32609]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:58:30.299 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:58:30 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h[62511]: ignoring --setuser ceph since I am not root 2024-04-25T18:58:30.299 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:58:30 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h[62511]: ignoring --setgroup ceph since I am not root 2024-04-25T18:58:30.300 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:58:30 smithi161 ceph-mds[62515]: -- 172.21.15.161:0/2372074684 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bbd4431a40 con 0x55bbd4405000 2024-04-25T18:58:30.300 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 18:58:30 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h[62511]: starting mds.h at 2024-04-25T18:58:30.300 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:58:30 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b[62830]: ignoring --setuser ceph since I am not root 2024-04-25T18:58:30.300 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:58:30 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b[62830]: ignoring --setgroup ceph since I am not root 2024-04-25T18:58:30.300 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:58:30 smithi161 ceph-mds[62834]: -- 172.21.15.161:0/3674261693 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b96b0afa40 con 0x55b96b082800 2024-04-25T18:58:30.300 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 18:58:30 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b[62830]: starting mds.b at 2024-04-25T18:58:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:30 smithi171 ceph-mon[33266]: pgmap v348: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:58:30.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:30 smithi171 ceph-mon[33266]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:58:30.433 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:58:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c[63670]: ignoring --setuser ceph since I am not root 2024-04-25T18:58:30.433 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:58:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c[63670]: ignoring --setgroup ceph since I am not root 2024-04-25T18:58:30.433 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:58:30 smithi171 ceph-mds[63674]: -- 172.21.15.171:0/3201537145 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572b8ac5a40 con 0x5572b8a98800 2024-04-25T18:58:30.433 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 18:58:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c[63670]: starting mds.c at 2024-04-25T18:58:30.433 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:58:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i[63990]: ignoring --setuser ceph since I am not root 2024-04-25T18:58:30.433 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:58:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i[63990]: ignoring --setgroup ceph since I am not root 2024-04-25T18:58:30.433 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:58:30 smithi171 ceph-mds[63994]: -- 172.21.15.171:0/124352932 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a662b89a40 con 0x55a662b5c800 2024-04-25T18:58:30.433 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 18:58:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i[63990]: starting mds.i at 2024-04-25T18:58:30.676 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:30 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a[27859]: 2024-04-25T18:58:30.244+0000 7f288f800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T18:58:30.676 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:58:30 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a[67160]: ignoring --setuser ceph since I am not root 2024-04-25T18:58:30.676 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:58:30 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a[67160]: ignoring --setgroup ceph since I am not root 2024-04-25T18:58:30.676 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:58:30 smithi112 ceph-mds[67164]: -- 172.21.15.112:0/3561883183 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d8162fa40 con 0x559d81602800 2024-04-25T18:58:30.676 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 18:58:30 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a[67160]: starting mds.a at 2024-04-25T18:58:30.677 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:58:30 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: ignoring --setuser ceph since I am not root 2024-04-25T18:58:30.677 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:58:30 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: ignoring --setgroup ceph since I am not root 2024-04-25T18:58:30.677 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:58:30 smithi112 ceph-mds[86791]: -- 172.21.15.112:0/459527957 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603fb487a40 con 0x5603fb45b000 2024-04-25T18:58:30.677 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 18:58:30 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: starting mds.g at 2024-04-25T18:58:31.159 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T18:58:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:31 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/229449741' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T18:58:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:31 smithi171 ceph-mon[33266]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T18:58:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:31 smithi171 ceph-mon[33266]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T18:58:31.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:31 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/229449741' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T18:58:31.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:31 smithi171 ceph-mon[33266]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T18:58:31.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:31 smithi171 ceph-mon[33266]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T18:58:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:31 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/229449741' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T18:58:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:31 smithi161 ceph-mon[32609]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T18:58:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:31 smithi161 ceph-mon[32609]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T18:58:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:31 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/229449741' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T18:58:31.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:31 smithi161 ceph-mon[32609]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T18:58:31.472 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:31 smithi161 ceph-mon[32609]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T18:58:31.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:31 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/229449741' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T18:58:31.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:31 smithi112 ceph-mon[27864]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T18:58:31.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:31 smithi112 ceph-mon[27864]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T18:58:31.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:31 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/229449741' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T18:58:31.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:31 smithi112 ceph-mon[27864]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T18:58:31.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:31 smithi112 ceph-mon[27864]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T18:58:31.626 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:32.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: pgmap v350: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.171:6836/1061182773,v1:172.21.15.171:6838/1061182773] up:boot 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.171:6837/3651314583,v1:172.21.15.171:6839/3651314583] up:boot 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.161:6836/1160546971,v1:172.21.15.161:6837/1160546971] up:boot 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.161:6838/3843262904,v1:172.21.15.161:6839/3843262904] up:boot 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.112:6838/1718259564,v1:172.21.15.112:6839/1718259564] up:boot 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: mds.? [v2:172.21.15.112:6840/3066898428,v1:172.21.15.112:6841/3066898428] up:boot 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:58:32.475 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:32 smithi112 ceph-mon[27864]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:58:32.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: pgmap v350: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.171:6836/1061182773,v1:172.21.15.171:6838/1061182773] up:boot 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.171:6837/3651314583,v1:172.21.15.171:6839/3651314583] up:boot 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.161:6836/1160546971,v1:172.21.15.161:6837/1160546971] up:boot 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.161:6838/3843262904,v1:172.21.15.161:6839/3843262904] up:boot 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.112:6838/1718259564,v1:172.21.15.112:6839/1718259564] up:boot 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: mds.? [v2:172.21.15.112:6840/3066898428,v1:172.21.15.112:6841/3066898428] up:boot 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:58:32.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:32 smithi171 ceph-mon[33266]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:58:32.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: pgmap v350: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.171:6836/1061182773,v1:172.21.15.171:6838/1061182773] up:boot 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.171:6837/3651314583,v1:172.21.15.171:6839/3651314583] up:boot 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.161:6836/1160546971,v1:172.21.15.161:6837/1160546971] up:boot 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.161:6838/3843262904,v1:172.21.15.161:6839/3843262904] up:boot 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.112:6838/1718259564,v1:172.21.15.112:6839/1718259564] up:boot 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: mds.? [v2:172.21.15.112:6840/3066898428,v1:172.21.15.112:6841/3066898428] up:boot 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:58:32.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:32 smithi161 ceph-mon[32609]: from='mgr.14922 172.21.15.112:0/3630561857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:58:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.169+0000 7f17d2000700 1 -- 172.21.15.112:0/335023613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17cc1092d0 msgr2=0x7f17cc1096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.169+0000 7f17d2000700 1 --2- 172.21.15.112:0/335023613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17cc1092d0 0x7f17cc1096b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0099c0 tx=0x7f17bc009cd0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.169+0000 7f17d2000700 1 -- 172.21.15.112:0/335023613 shutdown_connections 2024-04-25T18:58:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.169+0000 7f17d2000700 1 --2- 172.21.15.112:0/335023613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17cc10e3b0 0x7f17cc115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.169+0000 7f17d2000700 1 --2- 172.21.15.112:0/335023613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17cc1092d0 0x7f17cc1096b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.169+0000 7f17d2000700 1 --2- 172.21.15.112:0/335023613 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17cc109c80 0x7f17cc10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.169+0000 7f17d2000700 1 -- 172.21.15.112:0/335023613 >> 172.21.15.112:0/335023613 conn(0x7f17cc076210 msgr2=0x7f17cc107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.169+0000 7f17d2000700 1 -- 172.21.15.112:0/335023613 shutdown_connections 2024-04-25T18:58:34.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.169+0000 7f17d2000700 1 -- 172.21.15.112:0/335023613 wait complete. 2024-04-25T18:58:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.170+0000 7f17d2000700 1 Processor -- start 2024-04-25T18:58:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.170+0000 7f17d2000700 1 -- start start 2024-04-25T18:58:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.171+0000 7f17d2000700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17cc1092d0 0x7f17cc1a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.171+0000 7f17d2000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17cc109c80 0x7f17cc1a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.171+0000 7f17d2000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17cc10e3b0 0x7f17cc19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.171+0000 7f17d2000700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1192b0 con 0x7f17cc1092d0 2024-04-25T18:58:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.171+0000 7f17d2000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc119130 con 0x7f17cc109c80 2024-04-25T18:58:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.171+0000 7f17d2000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc119430 con 0x7f17cc10e3b0 2024-04-25T18:58:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.171+0000 7f17cb400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17cc1092d0 0x7f17cc1a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:34.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.171+0000 7f17cb400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17cc1092d0 0x7f17cc1a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33424/0 (socket says 172.21.15.112:33424) 2024-04-25T18:58:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.171+0000 7f17cb400700 1 -- 172.21.15.112:0/1143746532 learned_addr learned my addr 172.21.15.112:0/1143746532 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:58:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.171+0000 7f17cbe00700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17cc10e3b0 0x7f17cc19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.171+0000 7f17caa00700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17cc109c80 0x7f17cc1a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.172+0000 7f17caa00700 1 -- 172.21.15.112:0/1143746532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17cc10e3b0 msgr2=0x7f17cc19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.172+0000 7f17caa00700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17cc10e3b0 0x7f17cc19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.172+0000 7f17caa00700 1 -- 172.21.15.112:0/1143746532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17cc1092d0 msgr2=0x7f17cc1a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.172+0000 7f17caa00700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17cc1092d0 0x7f17cc1a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.172+0000 7f17caa00700 1 -- 172.21.15.112:0/1143746532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc19fd90 con 0x7f17cc109c80 2024-04-25T18:58:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.172+0000 7f17cb400700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17cc1092d0 0x7f17cc1a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.172+0000 7f17cbe00700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17cc10e3b0 0x7f17cc19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:34.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.172+0000 7f17caa00700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17cc109c80 0x7f17cc1a5960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e9c0 tx=0x7f17b800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:34.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.172+0000 7f17c3e00700 1 -- 172.21.15.112:0/1143746532 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8004220 con 0x7f17cc109c80 2024-04-25T18:58:34.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.172+0000 7f17d2000700 1 -- 172.21.15.112:0/1143746532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc1a0080 con 0x7f17cc109c80 2024-04-25T18:58:34.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.173+0000 7f17c3e00700 1 -- 172.21.15.112:0/1143746532 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17b80043c0 con 0x7f17cc109c80 2024-04-25T18:58:34.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.173+0000 7f17c3e00700 1 -- 172.21.15.112:0/1143746532 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b800ce20 con 0x7f17cc109c80 2024-04-25T18:58:34.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.173+0000 7f17d2000700 1 -- 172.21.15.112:0/1143746532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc1ac2f0 con 0x7f17cc109c80 2024-04-25T18:58:34.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.174+0000 7f17d2000700 1 -- 172.21.15.112:0/1143746532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc069730 con 0x7f17cc109c80 2024-04-25T18:58:34.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.176+0000 7f17c3e00700 1 -- 172.21.15.112:0/1143746532 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f17b80050e0 con 0x7f17cc109c80 2024-04-25T18:58:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.177+0000 7f17c3e00700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17b00a3e30 0x7f17b00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.177+0000 7f17c3e00700 1 -- 172.21.15.112:0/1143746532 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8785+0+0 (secure 0 0 0) 0x7f17b80d0940 con 0x7f17cc109c80 2024-04-25T18:58:34.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.177+0000 7f17cb400700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17b00a3e30 0x7f17b00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:34.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.178+0000 7f17cb400700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17b00a3e30 0x7f17b00a62f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009990 tx=0x7f17bc005ab0 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:34.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.180+0000 7f17c3e00700 1 -- 172.21.15.112:0/1143746532 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17b809ce70 con 0x7f17cc109c80 2024-04-25T18:58:34.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:34.455+0000 7f17d2000700 1 -- 172.21.15.112:0/1143746532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f17cc1a0b10 con 0x7f17cc109c80 2024-04-25T18:58:34.457 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:34 smithi112 ceph-mon[27864]: pgmap v351: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:58:34.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:34 smithi171 ceph-mon[33266]: pgmap v351: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:58:34.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:34 smithi161 ceph-mon[32609]: pgmap v351: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:58:35.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.278+0000 7f17c3e00700 1 -- 172.21.15.112:0/1143746532 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7f17b809c770 con 0x7f17cc109c80 2024-04-25T18:58:35.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 -- 172.21.15.112:0/1143746532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17b00a3e30 msgr2=0x7f17b00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:35.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17b00a3e30 0x7f17b00a62f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009990 tx=0x7f17bc005ab0 comp rx=0 tx=0).stop 2024-04-25T18:58:35.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 -- 172.21.15.112:0/1143746532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17cc109c80 msgr2=0x7f17cc1a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:35.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17cc109c80 0x7f17cc1a5960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e9c0 tx=0x7f17b800ecd0 comp rx=0 tx=0).stop 2024-04-25T18:58:35.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 -- 172.21.15.112:0/1143746532 shutdown_connections 2024-04-25T18:58:35.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f17b00a3e30 0x7f17b00a62f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:35.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17cc10e3b0 0x7f17cc19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:35.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17cc1092d0 0x7f17cc1a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:35.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 --2- 172.21.15.112:0/1143746532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17cc109c80 0x7f17cc1a5960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:35.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 -- 172.21.15.112:0/1143746532 >> 172.21.15.112:0/1143746532 conn(0x7f17cc076210 msgr2=0x7f17cc076870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:35.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 -- 172.21.15.112:0/1143746532 shutdown_connections 2024-04-25T18:58:35.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:35.282+0000 7f17d2000700 1 -- 172.21.15.112:0/1143746532 wait complete. 2024-04-25T18:58:35.674 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:35 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/1143746532' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:58:35.674 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:35 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:58:35.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:35 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/1143746532' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:58:35.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:35 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:58:35.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:35 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/1143746532' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:58:35.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:35 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:58:36.149 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T18:58:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:36 smithi171 ceph-mon[33266]: pgmap v352: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:58:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:36 smithi171 ceph-mon[33266]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T18:58:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:36 smithi171 ceph-mon[33266]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T18:58:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:36 smithi171 ceph-mon[33266]: Cluster is now healthy 2024-04-25T18:58:36.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:36 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:58:36.433 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:36 smithi171 ceph-mon[33266]: fsmap 12 up:standby 2024-04-25T18:58:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:36 smithi112 ceph-mon[27864]: pgmap v352: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:58:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:36 smithi112 ceph-mon[27864]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T18:58:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:36 smithi112 ceph-mon[27864]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T18:58:36.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:36 smithi112 ceph-mon[27864]: Cluster is now healthy 2024-04-25T18:58:36.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:36 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:58:36.474 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:36 smithi112 ceph-mon[27864]: fsmap 12 up:standby 2024-04-25T18:58:36.615 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:36.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:36 smithi161 ceph-mon[32609]: pgmap v352: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:58:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:36 smithi161 ceph-mon[32609]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T18:58:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:36 smithi161 ceph-mon[32609]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T18:58:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:36 smithi161 ceph-mon[32609]: Cluster is now healthy 2024-04-25T18:58:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:36 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:58:36.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:36 smithi161 ceph-mon[32609]: fsmap 12 up:standby 2024-04-25T18:58:38.388 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:38 smithi112 ceph-mon[27864]: pgmap v353: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T18:58:38.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:38 smithi171 ceph-mon[33266]: pgmap v353: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T18:58:38.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:38 smithi161 ceph-mon[32609]: pgmap v353: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T18:58:39.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.139+0000 7f5077c00700 1 -- 172.21.15.112:0/2000194729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5070074c70 msgr2=0x7f5070075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:39.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.139+0000 7f5077c00700 1 --2- 172.21.15.112:0/2000194729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5070074c70 0x7f5070075050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5064009990 tx=0x7f5064009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.140+0000 7f5077c00700 1 -- 172.21.15.112:0/2000194729 shutdown_connections 2024-04-25T18:58:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.140+0000 7f5077c00700 1 --2- 172.21.15.112:0/2000194729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507010f440 0x7f5070111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.140+0000 7f5077c00700 1 --2- 172.21.15.112:0/2000194729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5070074c70 0x7f5070075050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.140+0000 7f5077c00700 1 --2- 172.21.15.112:0/2000194729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5070075590 0x7f507010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.140+0000 7f5077c00700 1 -- 172.21.15.112:0/2000194729 >> 172.21.15.112:0/2000194729 conn(0x7f5070079ba0 msgr2=0x7f5070079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.140+0000 7f5077c00700 1 -- 172.21.15.112:0/2000194729 shutdown_connections 2024-04-25T18:58:39.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.140+0000 7f5077c00700 1 -- 172.21.15.112:0/2000194729 wait complete. 2024-04-25T18:58:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.141+0000 7f5077c00700 1 Processor -- start 2024-04-25T18:58:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.141+0000 7f5077c00700 1 -- start start 2024-04-25T18:58:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.141+0000 7f5077c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5070074c70 0x7f507019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.141+0000 7f5077c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5070075590 0x7f507019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5077c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507010f440 0x7f507019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5077c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5070114e70 con 0x7f507010f440 2024-04-25T18:58:39.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5077c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5070114cf0 con 0x7f5070075590 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5077c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5070114ff0 con 0x7f5070074c70 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5075c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507010f440 0x7f507019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5075c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507010f440 0x7f507019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33448/0 (socket says 172.21.15.112:33448) 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5075c00700 1 -- 172.21.15.112:0/4087191488 learned_addr learned my addr 172.21.15.112:0/4087191488 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5075200700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5070074c70 0x7f507019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f506fe00700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5070075590 0x7f507019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5075c00700 1 -- 172.21.15.112:0/4087191488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5070074c70 msgr2=0x7f507019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5075c00700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5070074c70 0x7f507019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5075c00700 1 -- 172.21.15.112:0/4087191488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5070075590 msgr2=0x7f507019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5075c00700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5070075590 0x7f507019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f5075c00700 1 -- 172.21.15.112:0/4087191488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50701a0320 con 0x7f507010f440 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.142+0000 7f506fe00700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5070075590 0x7f507019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.143+0000 7f5075c00700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507010f440 0x7f507019fbe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f506800c110 tx=0x7f506800c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:39.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.143+0000 7f506d600700 1 -- 172.21.15.112:0/4087191488 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50680092e0 con 0x7f507010f440 2024-04-25T18:58:39.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.143+0000 7f506d600700 1 -- 172.21.15.112:0/4087191488 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5068003a00 con 0x7f507010f440 2024-04-25T18:58:39.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.143+0000 7f506d600700 1 -- 172.21.15.112:0/4087191488 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50680046c0 con 0x7f507010f440 2024-04-25T18:58:39.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.143+0000 7f5077c00700 1 -- 172.21.15.112:0/4087191488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50701a05b0 con 0x7f507010f440 2024-04-25T18:58:39.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.143+0000 7f5077c00700 1 -- 172.21.15.112:0/4087191488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50701a7e90 con 0x7f507010f440 2024-04-25T18:58:39.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.144+0000 7f5077c00700 1 -- 172.21.15.112:0/4087191488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5070069730 con 0x7f507010f440 2024-04-25T18:58:39.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.147+0000 7f506d600700 1 -- 172.21.15.112:0/4087191488 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f506801a020 con 0x7f507010f440 2024-04-25T18:58:39.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.148+0000 7f506d600700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f50580a3db0 0x7f50580a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:39.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.149+0000 7f506d600700 1 -- 172.21.15.112:0/4087191488 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8785+0+0 (secure 0 0 0) 0x7f50680d0340 con 0x7f507010f440 2024-04-25T18:58:39.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.149+0000 7f5075200700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f50580a3db0 0x7f50580a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:39.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.150+0000 7f5075200700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f50580a3db0 0x7f50580a6270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f506400b2e0 tx=0x7f5064005fd0 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:39.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.154+0000 7f506d600700 1 -- 172.21.15.112:0/4087191488 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5068010040 con 0x7f507010f440 2024-04-25T18:58:39.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:39.382+0000 7f5077c00700 1 -- 172.21.15.112:0/4087191488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f507002df80 con 0x7f507010f440 2024-04-25T18:58:40.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.287+0000 7f506d600700 1 -- 172.21.15.112:0/4087191488 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) ==== 181+0+0 (secure 0 0 0) 0x7f506809c870 con 0x7f507010f440 2024-04-25T18:58:40.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.292+0000 7f5077c00700 1 -- 172.21.15.112:0/4087191488 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f50580a3db0 msgr2=0x7f50580a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:40.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.292+0000 7f5077c00700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f50580a3db0 0x7f50580a6270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f506400b2e0 tx=0x7f5064005fd0 comp rx=0 tx=0).stop 2024-04-25T18:58:40.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.292+0000 7f5077c00700 1 -- 172.21.15.112:0/4087191488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507010f440 msgr2=0x7f507019fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:40.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.292+0000 7f5077c00700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507010f440 0x7f507019fbe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f506800c110 tx=0x7f506800c420 comp rx=0 tx=0).stop 2024-04-25T18:58:40.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.293+0000 7f5077c00700 1 -- 172.21.15.112:0/4087191488 shutdown_connections 2024-04-25T18:58:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.293+0000 7f5077c00700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7f50580a3db0 0x7f50580a6270 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f506400b2e0 tx=0x7f5064005fd0 comp rx=0 tx=0).stop 2024-04-25T18:58:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.293+0000 7f5077c00700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507010f440 0x7f507019fbe0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.293+0000 7f5077c00700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5070074c70 0x7f507019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.293+0000 7f5077c00700 1 --2- 172.21.15.112:0/4087191488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5070075590 0x7f507019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.293+0000 7f5077c00700 1 -- 172.21.15.112:0/4087191488 >> 172.21.15.112:0/4087191488 conn(0x7f5070079ba0 msgr2=0x7f50700ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.294+0000 7f5077c00700 1 -- 172.21.15.112:0/4087191488 shutdown_connections 2024-04-25T18:58:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:40.294+0000 7f5077c00700 1 -- 172.21.15.112:0/4087191488 wait complete. 2024-04-25T18:58:40.297 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-04-25T18:58:40.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:40 smithi171 ceph-mon[33266]: pgmap v354: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T18:58:40.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:40 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/4087191488' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T18:58:40.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:40 smithi161 ceph-mon[32609]: pgmap v354: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T18:58:40.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:40 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/4087191488' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T18:58:40.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:40 smithi112 ceph-mon[27864]: pgmap v354: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T18:58:40.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:40 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/4087191488' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T18:58:41.189 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T18:58:41.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:41 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/4087191488' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T18:58:41.473 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:41 smithi112 ceph-mon[27864]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T18:58:41.639 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:41.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:41 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/4087191488' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T18:58:41.683 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:41 smithi171 ceph-mon[33266]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T18:58:41.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:41 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/4087191488' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T18:58:41.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:41 smithi161 ceph-mon[32609]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T18:58:42.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:42 smithi171 ceph-mon[33266]: pgmap v356: 65 pgs: 65 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:42.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:42 smithi161 ceph-mon[32609]: pgmap v356: 65 pgs: 65 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:42.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:42 smithi112 ceph-mon[27864]: pgmap v356: 65 pgs: 65 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.874+0000 7ff255c00700 1 -- 172.21.15.112:0/2667440380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff250105780 msgr2=0x7ff2501097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.874+0000 7ff255c00700 1 --2- 172.21.15.112:0/2667440380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff250105780 0x7ff2501097d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0099c0 tx=0x7ff23c009cd0 comp rx=0 tx=0).stop 2024-04-25T18:58:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.874+0000 7ff255c00700 1 -- 172.21.15.112:0/2667440380 shutdown_connections 2024-04-25T18:58:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.874+0000 7ff255c00700 1 --2- 172.21.15.112:0/2667440380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff250109eb0 0x7ff250111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.874+0000 7ff255c00700 1 --2- 172.21.15.112:0/2667440380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff250104dd0 0x7ff2501051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.874+0000 7ff255c00700 1 --2- 172.21.15.112:0/2667440380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff250105780 0x7ff2501097d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.874+0000 7ff255c00700 1 -- 172.21.15.112:0/2667440380 >> 172.21.15.112:0/2667440380 conn(0x7ff250100660 msgr2=0x7ff250102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.874+0000 7ff255c00700 1 -- 172.21.15.112:0/2667440380 shutdown_connections 2024-04-25T18:58:43.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.875+0000 7ff255c00700 1 -- 172.21.15.112:0/2667440380 wait complete. 2024-04-25T18:58:43.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.875+0000 7ff255c00700 1 Processor -- start 2024-04-25T18:58:43.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.876+0000 7ff255c00700 1 -- start start 2024-04-25T18:58:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.876+0000 7ff255c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff250104dd0 0x7ff25019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.876+0000 7ff255c00700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff250105780 0x7ff25019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.876+0000 7ff255c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff250109eb0 0x7ff25019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.876+0000 7ff255c00700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff250114d30 con 0x7ff250105780 2024-04-25T18:58:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.876+0000 7ff255c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff250114bb0 con 0x7ff250104dd0 2024-04-25T18:58:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.876+0000 7ff255c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff250114eb0 con 0x7ff250109eb0 2024-04-25T18:58:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.876+0000 7ff24f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff250105780 0x7ff25019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.876+0000 7ff24fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff250104dd0 0x7ff25019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.877+0000 7ff24f400700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff250105780 0x7ff25019b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54628/0 (socket says 172.21.15.112:54628) 2024-04-25T18:58:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.877+0000 7ff24f400700 1 -- 172.21.15.112:0/529255490 learned_addr learned my addr 172.21.15.112:0/529255490 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T18:58:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.877+0000 7ff255200700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff250109eb0 0x7ff25019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:43.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.877+0000 7ff24fe00700 1 -- 172.21.15.112:0/529255490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff250109eb0 msgr2=0x7ff25019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:43.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.877+0000 7ff24fe00700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff250109eb0 0x7ff25019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:43.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.877+0000 7ff24fe00700 1 -- 172.21.15.112:0/529255490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff250105780 msgr2=0x7ff25019b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:43.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.877+0000 7ff24fe00700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff250105780 0x7ff25019b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:43.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.877+0000 7ff24fe00700 1 -- 172.21.15.112:0/529255490 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2501a0240 con 0x7ff250104dd0 2024-04-25T18:58:43.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.877+0000 7ff255200700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff250109eb0 0x7ff25019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:58:43.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.877+0000 7ff24fe00700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff250104dd0 0x7ff25019b280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff24000e9f0 tx=0x7ff24000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:43.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.878+0000 7ff24cc00700 1 -- 172.21.15.112:0/529255490 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24000cbd0 con 0x7ff250104dd0 2024-04-25T18:58:43.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.878+0000 7ff24cc00700 1 -- 172.21.15.112:0/529255490 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24000cd70 con 0x7ff250104dd0 2024-04-25T18:58:43.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.878+0000 7ff24cc00700 1 -- 172.21.15.112:0/529255490 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240018680 con 0x7ff250104dd0 2024-04-25T18:58:43.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.878+0000 7ff255c00700 1 -- 172.21.15.112:0/529255490 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2501a0530 con 0x7ff250104dd0 2024-04-25T18:58:43.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.878+0000 7ff255c00700 1 -- 172.21.15.112:0/529255490 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2501a7db0 con 0x7ff250104dd0 2024-04-25T18:58:43.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.879+0000 7ff255c00700 1 -- 172.21.15.112:0/529255490 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff250069730 con 0x7ff250104dd0 2024-04-25T18:58:43.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.881+0000 7ff24cc00700 1 -- 172.21.15.112:0/529255490 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7ff240010430 con 0x7ff250104dd0 2024-04-25T18:58:43.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.882+0000 7ff24cc00700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff2340a3db0 0x7ff2340a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:43.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.882+0000 7ff24cc00700 1 -- 172.21.15.112:0/529255490 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8202+0+0 (secure 0 0 0) 0x7ff2400d0900 con 0x7ff250104dd0 2024-04-25T18:58:43.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.883+0000 7ff24f400700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff2340a3db0 0x7ff2340a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:43.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.883+0000 7ff24f400700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff2340a3db0 0x7ff2340a6270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009990 tx=0x7ff23c005960 comp rx=0 tx=0).ready entity=mgr.14922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:43.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:43.885+0000 7ff24cc00700 1 -- 172.21.15.112:0/529255490 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff24009d080 con 0x7ff250104dd0 2024-04-25T18:58:44.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.110+0000 7ff255c00700 1 -- 172.21.15.112:0/529255490 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff25002df80 con 0x7ff250104dd0 2024-04-25T18:58:44.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.309+0000 7ff24cc00700 1 -- 172.21.15.112:0/529255490 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) ==== 169+0+0 (secure 0 0 0) 0x7ff24009c980 con 0x7ff250104dd0 2024-04-25T18:58:44.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.315+0000 7ff255c00700 1 -- 172.21.15.112:0/529255490 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff2340a3db0 msgr2=0x7ff2340a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:44.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.315+0000 7ff255c00700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff2340a3db0 0x7ff2340a6270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009990 tx=0x7ff23c005960 comp rx=0 tx=0).stop 2024-04-25T18:58:44.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.315+0000 7ff255c00700 1 -- 172.21.15.112:0/529255490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff250104dd0 msgr2=0x7ff25019b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:44.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.315+0000 7ff255c00700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff250104dd0 0x7ff25019b280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff24000e9f0 tx=0x7ff24000ed00 comp rx=0 tx=0).stop 2024-04-25T18:58:44.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.315+0000 7ff255c00700 1 -- 172.21.15.112:0/529255490 shutdown_connections 2024-04-25T18:58:44.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.315+0000 7ff255c00700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.112:6800/2236031982,v1:172.21.15.112:6801/2236031982] conn(0x7ff2340a3db0 0x7ff2340a6270 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:44.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.315+0000 7ff255c00700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff250109eb0 0x7ff25019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:44.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.315+0000 7ff255c00700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff250104dd0 0x7ff25019b280 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:44.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.315+0000 7ff255c00700 1 --2- 172.21.15.112:0/529255490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff250105780 0x7ff25019b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:44.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.315+0000 7ff255c00700 1 -- 172.21.15.112:0/529255490 >> 172.21.15.112:0/529255490 conn(0x7ff250100660 msgr2=0x7ff2501029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:44.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.316+0000 7ff255c00700 1 -- 172.21.15.112:0/529255490 shutdown_connections 2024-04-25T18:58:44.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T18:58:44.316+0000 7ff255c00700 1 -- 172.21.15.112:0/529255490 wait complete. 2024-04-25T18:58:44.323 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-04-25T18:58:44.371 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:44 smithi112 ceph-mon[27864]: pgmap v357: 65 pgs: 65 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:44.371 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:44 smithi112 ceph-mon[27864]: from='client.? 172.21.15.112:0/529255490' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T18:58:44.371 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:44 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T18:58:44.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:44 smithi171 ceph-mon[33266]: pgmap v357: 65 pgs: 65 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:44.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:44 smithi171 ceph-mon[33266]: from='client.? 172.21.15.112:0/529255490' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T18:58:44.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:44 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T18:58:44.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:44 smithi161 ceph-mon[32609]: pgmap v357: 65 pgs: 65 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:58:44.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:44 smithi161 ceph-mon[32609]: from='client.? 172.21.15.112:0/529255490' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T18:58:44.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:44 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T18:58:45.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:58:45.274 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:58:45.303 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:58:45.339 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T18:58:45.339 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T18:58:45.382 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T18:58:45.398 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T18:58:45.415 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T18:58:45.415 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T18:58:45.415 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.a 2024-04-25T18:58:45.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:45 smithi171 ceph-mon[33266]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T18:58:45.682 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:45 smithi171 ceph-mon[33266]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T18:58:45.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:45 smithi161 ceph-mon[32609]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T18:58:45.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:45 smithi161 ceph-mon[32609]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T18:58:45.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:45 smithi112 ceph-mon[27864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T18:58:45.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:45 smithi112 ceph-mon[27864]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T18:58:45.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:45 smithi112 systemd[1]: Stopping Ceph mon.a for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:58:46.379 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:45 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a[27859]: 2024-04-25T18:58:45.984+0000 7f2896600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:58:46.379 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:45 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a[27859]: 2024-04-25T18:58:45.984+0000 7f2896600700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T18:58:46.379 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:46 smithi112 podman[113000]: 2024-04-25 18:58:46.037137507 +0000 UTC m=+0.226138704 container died 8e659435d39e1dc00ddec175623d7d37a80a8826781e84c5c9f9cb4d651ae6e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:58:46.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:46 smithi112 podman[113000]: 2024-04-25 18:58:46.529465188 +0000 UTC m=+0.718466384 container cleanup 8e659435d39e1dc00ddec175623d7d37a80a8826781e84c5c9f9cb4d651ae6e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T18:58:46.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:46 smithi112 bash[113000]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a 2024-04-25T18:58:47.154 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 18:58:46 smithi112 podman[113013]: 2024-04-25 18:58:46.722394833 +0000 UTC m=+0.673533193 container remove 8e659435d39e1dc00ddec175623d7d37a80a8826781e84c5c9f9cb4d651ae6e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T18:58:47.198 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T18:58:47.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:58:47.319 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T18:58:47.319 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T18:58:47.319 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.b 2024-04-25T18:58:47.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:47 smithi161 systemd[1]: Stopping Ceph mon.b for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:58:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:47 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-b[32603]: 2024-04-25T18:58:47.630+0000 7fb930c21700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:58:47.971 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:47 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-b[32603]: 2024-04-25T18:58:47.630+0000 7fb930c21700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T18:58:47.972 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:47 smithi161 podman[68847]: 2024-04-25 18:58:47.876222285 +0000 UTC m=+0.358759203 container died c70d9eb4feb45b110cce1f9d035dfac8da3c5fb0df027fb2e3ba15c6c329b921 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T18:58:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:48 smithi161 podman[68847]: 2024-04-25 18:58:48.264143098 +0000 UTC m=+0.746680014 container cleanup c70d9eb4feb45b110cce1f9d035dfac8da3c5fb0df027fb2e3ba15c6c329b921 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T18:58:48.471 INFO:journalctl@ceph.mon.b.smithi161.stdout:Apr 25 18:58:48 smithi161 bash[68847]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-b 2024-04-25T18:58:48.603 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T18:58:48.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:58:48.646 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T18:58:48.646 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T18:58:48.646 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.c 2024-04-25T18:58:48.932 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:48 smithi171 systemd[1]: Stopping Ceph mon.c for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:58:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:48 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-c[33262]: 2024-04-25T18:58:48.963+0000 7f70e8a58700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:58:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:48 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-c[33262]: 2024-04-25T18:58:48.963+0000 7f70e8a58700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T18:58:49.432 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:49 smithi171 podman[70008]: 2024-04-25 18:58:49.198923284 +0000 UTC m=+0.352827782 container died c5d70d8d788b976837fd4bf53d574e3ab2f2792ac8c6a01dfa19b7797beac6ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T18:58:49.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:49 smithi171 podman[70008]: 2024-04-25 18:58:49.588859662 +0000 UTC m=+0.742764167 container cleanup c5d70d8d788b976837fd4bf53d574e3ab2f2792ac8c6a01dfa19b7797beac6ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T18:58:49.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 18:58:49 smithi171 bash[70008]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mon-c 2024-04-25T18:58:49.961 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T18:58:50.006 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:58:50.007 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T18:58:50.007 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T18:58:50.007 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.x 2024-04-25T18:58:50.473 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:58:50 smithi112 systemd[1]: Stopping Ceph mgr.x for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:58:50.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:58:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:58:50.639+0000 7fca5b400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:58:50.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:58:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x[28103]: 2024-04-25T18:58:50.639+0000 7fca5b400700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T18:58:50.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:58:50 smithi112 podman[113121]: 2024-04-25 18:58:50.733653465 +0000 UTC m=+0.286767096 container died 0472f2a3d65ad6c8c9257de2417ec854a5f73f4697bad68948e80671963f49fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-25T18:58:51.474 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:58:51 smithi112 podman[113121]: 2024-04-25 18:58:51.284698517 +0000 UTC m=+0.837812178 container cleanup 0472f2a3d65ad6c8c9257de2417ec854a5f73f4697bad68948e80671963f49fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:58:51.474 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:58:51 smithi112 bash[113121]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x 2024-04-25T18:58:51.974 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 18:58:51 smithi112 podman[113134]: 2024-04-25 18:58:51.611457216 +0000 UTC m=+0.862479070 container remove 0472f2a3d65ad6c8c9257de2417ec854a5f73f4697bad68948e80671963f49fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:58:52.091 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T18:58:52.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:58:52.196 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T18:58:52.196 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T18:58:52.196 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.y 2024-04-25T18:58:52.523 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:58:52 smithi161 systemd[1]: Stopping Ceph mgr.y for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:58:52.795 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:58:52 smithi161 podman[68984]: 2024-04-25 18:58:52.52236337 +0000 UTC m=+0.138318042 container died 37d97ce2f9f65e0ac8ab8f17c2a52a5c8d4dcd18d274342703c6b06059f8bccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T18:58:53.054 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:58:52 smithi161 podman[68984]: 2024-04-25 18:58:52.868312896 +0000 UTC m=+0.484267575 container cleanup 37d97ce2f9f65e0ac8ab8f17c2a52a5c8d4dcd18d274342703c6b06059f8bccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:58:53.054 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:58:52 smithi161 bash[68984]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y 2024-04-25T18:58:53.288 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T18:58:53.307 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:58:53 smithi161 podman[68999]: 2024-04-25 18:58:53.053539837 +0000 UTC m=+0.526862034 container remove 37d97ce2f9f65e0ac8ab8f17c2a52a5c8d4dcd18d274342703c6b06059f8bccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T18:58:53.308 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:58:53 smithi161 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-25T18:58:53.308 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:58:53 smithi161 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-25T18:58:53.308 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:58:53 smithi161 systemd[1]: Stopped Ceph mgr.y for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T18:58:53.308 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Apr 25 18:58:53 smithi161 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.y.service: Consumed 20.800s CPU time. 2024-04-25T18:58:53.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:58:53.321 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T18:58:53.321 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T18:58:53.321 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.z 2024-04-25T18:58:53.639 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:58:53 smithi171 systemd[1]: Stopping Ceph mgr.z for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:58:53.928 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:58:53 smithi171 podman[70131]: 2024-04-25 18:58:53.638932683 +0000 UTC m=+0.127382626 container died 1dd5024bb2eba322c3e22796ac4c36e119692a2a8eb4a662bbbc8946c0335b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:58:54.182 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:58:54 smithi171 podman[70131]: 2024-04-25 18:58:54.009923218 +0000 UTC m=+0.498373161 container cleanup 1dd5024bb2eba322c3e22796ac4c36e119692a2a8eb4a662bbbc8946c0335b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T18:58:54.182 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:58:54 smithi171 bash[70131]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z 2024-04-25T18:58:54.562 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:58:54 smithi171 podman[70146]: 2024-04-25 18:58:54.369608536 +0000 UTC m=+0.723776950 container remove 1dd5024bb2eba322c3e22796ac4c36e119692a2a8eb4a662bbbc8946c0335b3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:58:54.563 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 18:58:54 smithi171 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-25T18:58:54.578 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T18:58:54.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:58:54.611 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T18:58:54.611 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T18:58:54.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.0 2024-04-25T18:58:54.974 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:58:54 smithi112 systemd[1]: Stopping Ceph osd.0 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:58:55.474 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:58:55 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0[40293]: 2024-04-25T18:58:55.196+0000 7f5d51e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:58:55.474 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:58:55 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0[40293]: 2024-04-25T18:58:55.196+0000 7f5d51e00700 -1 osd.0 111 *** Got signal Terminated *** 2024-04-25T18:58:55.474 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:58:55 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0[40293]: 2024-04-25T18:58:55.196+0000 7f5d51e00700 -1 osd.0 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T18:59:00.974 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:59:00 smithi112 podman[113241]: 2024-04-25 18:59:00.663588277 +0000 UTC m=+5.612472841 container died 8be2293b0c13bd5b0e2812094d8a6b3a6c00d6284fae9d3620eca99aeac7f69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T18:59:01.519 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:59:01 smithi112 podman[113241]: 2024-04-25 18:59:01.241262188 +0000 UTC m=+6.190146759 container cleanup 8be2293b0c13bd5b0e2812094d8a6b3a6c00d6284fae9d3620eca99aeac7f69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:59:01.520 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:59:01 smithi112 bash[113241]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0 2024-04-25T18:59:01.974 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:59:01 smithi112 podman[113260]: 2024-04-25 18:59:01.517441552 +0000 UTC m=+0.833701111 container remove 8be2293b0c13bd5b0e2812094d8a6b3a6c00d6284fae9d3620eca99aeac7f69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T18:59:02.724 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:59:02 smithi112 podman[113346]: 2024-04-25 18:59:02.358459714 +0000 UTC m=+0.036477810 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:59:02.724 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:59:02 smithi112 podman[113346]: 2024-04-25 18:59:02.485063113 +0000 UTC m=+0.163081239 container create de9e5a0b9c33b2dad459ad3e75cd78e94c0954ec612f8c5768ce90211cdc5fd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T18:59:03.217 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:59:02 smithi112 podman[113346]: 2024-04-25 18:59:02.764434217 +0000 UTC m=+0.442452347 container init de9e5a0b9c33b2dad459ad3e75cd78e94c0954ec612f8c5768ce90211cdc5fd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T18:59:03.217 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:59:02 smithi112 podman[113346]: 2024-04-25 18:59:02.776251702 +0000 UTC m=+0.454269802 container start de9e5a0b9c33b2dad459ad3e75cd78e94c0954ec612f8c5768ce90211cdc5fd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T18:59:03.217 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:59:02 smithi112 podman[113346]: 2024-04-25 18:59:02.871026951 +0000 UTC m=+0.549045065 container attach de9e5a0b9c33b2dad459ad3e75cd78e94c0954ec612f8c5768ce90211cdc5fd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T18:59:03.474 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 18:59:03 smithi112 podman[113346]: 2024-04-25 18:59:03.216244316 +0000 UTC m=+0.894262449 container died de9e5a0b9c33b2dad459ad3e75cd78e94c0954ec612f8c5768ce90211cdc5fd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T18:59:04.064 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T18:59:04.164 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:59:04.165 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T18:59:04.165 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T18:59:04.165 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.1 2024-04-25T18:59:04.468 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:04 smithi161 systemd[1]: Stopping Ceph osd.1 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:59:04.721 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:04 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1[37195]: 2024-04-25T18:59:04.466+0000 7f504762f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:59:04.721 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:04 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1[37195]: 2024-04-25T18:59:04.466+0000 7f504762f700 -1 osd.1 111 *** Got signal Terminated *** 2024-04-25T18:59:04.721 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:04 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1[37195]: 2024-04-25T18:59:04.466+0000 7f504762f700 -1 osd.1 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T18:59:10.275 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:09 smithi161 podman[69112]: 2024-04-25 18:59:09.999368139 +0000 UTC m=+5.646553367 container died fd7d5c6e960297d5b198106b50a1224cf433bfbc584f3d96c5bbe6dbe43f86ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:59:10.721 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:10 smithi161 podman[69112]: 2024-04-25 18:59:10.274474102 +0000 UTC m=+5.921659322 container cleanup fd7d5c6e960297d5b198106b50a1224cf433bfbc584f3d96c5bbe6dbe43f86ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-25T18:59:10.721 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:10 smithi161 bash[69112]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1 2024-04-25T18:59:10.721 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:10 smithi161 podman[69129]: 2024-04-25 18:59:10.517277768 +0000 UTC m=+0.513417711 container remove fd7d5c6e960297d5b198106b50a1224cf433bfbc584f3d96c5bbe6dbe43f86ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:59:11.096 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:10 smithi161 podman[69219]: 2024-04-25 18:59:10.746979909 +0000 UTC m=+0.017918416 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:59:11.096 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:10 smithi161 podman[69219]: 2024-04-25 18:59:10.851483771 +0000 UTC m=+0.122422263 container create 0022fd4f5fffa2c165c4af213217a02ac3e1b567cba7656bf3b5bc366f581b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:59:11.096 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:11 smithi161 podman[69219]: 2024-04-25 18:59:11.095534313 +0000 UTC m=+0.366472805 container init 0022fd4f5fffa2c165c4af213217a02ac3e1b567cba7656bf3b5bc366f581b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T18:59:11.396 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:11 smithi161 podman[69219]: 2024-04-25 18:59:11.098866411 +0000 UTC m=+0.369804904 container start 0022fd4f5fffa2c165c4af213217a02ac3e1b567cba7656bf3b5bc366f581b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T18:59:11.397 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:11 smithi161 podman[69219]: 2024-04-25 18:59:11.179090003 +0000 UTC m=+0.450028511 container attach 0022fd4f5fffa2c165c4af213217a02ac3e1b567cba7656bf3b5bc366f581b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T18:59:11.721 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:11 smithi161 podman[69219]: 2024-04-25 18:59:11.396109079 +0000 UTC m=+0.667047572 container died 0022fd4f5fffa2c165c4af213217a02ac3e1b567cba7656bf3b5bc366f581b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T18:59:12.001 INFO:journalctl@ceph.osd.1.smithi161.stdout:Apr 25 18:59:11 smithi161 podman[69245]: 2024-04-25 18:59:11.982192204 +0000 UTC m=+0.580123507 container remove 0022fd4f5fffa2c165c4af213217a02ac3e1b567cba7656bf3b5bc366f581b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T18:59:12.007 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T18:59:12.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:59:12.044 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T18:59:12.044 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T18:59:12.045 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.2 2024-04-25T18:59:12.340 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:12 smithi171 systemd[1]: Stopping Ceph osd.2 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:59:12.682 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:12 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2[38410]: 2024-04-25T18:59:12.337+0000 7f6505a80700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:59:12.683 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:12 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2[38410]: 2024-04-25T18:59:12.337+0000 7f6505a80700 -1 osd.2 111 *** Got signal Terminated *** 2024-04-25T18:59:12.683 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:12 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2[38410]: 2024-04-25T18:59:12.337+0000 7f6505a80700 -1 osd.2 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T18:59:18.081 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:17 smithi171 podman[70269]: 2024-04-25 18:59:17.792517044 +0000 UTC m=+5.560236822 container died d4e856c3f03899452253cd8e3bbb55ef057c046d8cd27ba87f7ba3fb40ad35b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T18:59:18.414 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:18 smithi171 podman[70269]: 2024-04-25 18:59:18.188042408 +0000 UTC m=+5.955762196 container cleanup d4e856c3f03899452253cd8e3bbb55ef057c046d8cd27ba87f7ba3fb40ad35b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T18:59:18.414 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:18 smithi171 bash[70269]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2 2024-04-25T18:59:18.682 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:18 smithi171 podman[70288]: 2024-04-25 18:59:18.413886951 +0000 UTC m=+0.614050267 container remove d4e856c3f03899452253cd8e3bbb55ef057c046d8cd27ba87f7ba3fb40ad35b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:59:19.068 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:18 smithi171 podman[70380]: 2024-04-25 18:59:18.650673452 +0000 UTC m=+0.017812918 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:59:19.068 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:18 smithi171 podman[70380]: 2024-04-25 18:59:18.748127139 +0000 UTC m=+0.115266595 container create bed8204187426e93b9e22a0abcd465d03e61a7fe98c403cbe06aac8ae56d621e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:59:19.068 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:18 smithi171 podman[70380]: 2024-04-25 18:59:18.984194901 +0000 UTC m=+0.351334357 container init bed8204187426e93b9e22a0abcd465d03e61a7fe98c403cbe06aac8ae56d621e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T18:59:19.068 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:18 smithi171 podman[70380]: 2024-04-25 18:59:18.987444025 +0000 UTC m=+0.354583483 container start bed8204187426e93b9e22a0abcd465d03e61a7fe98c403cbe06aac8ae56d621e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T18:59:19.068 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:19 smithi171 podman[70380]: 2024-04-25 18:59:19.066994024 +0000 UTC m=+0.434133483 container attach bed8204187426e93b9e22a0abcd465d03e61a7fe98c403cbe06aac8ae56d621e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T18:59:19.342 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 18:59:19 smithi171 podman[70380]: 2024-04-25 18:59:19.290151812 +0000 UTC m=+0.657291265 container died bed8204187426e93b9e22a0abcd465d03e61a7fe98c403cbe06aac8ae56d621e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:59:19.914 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T18:59:19.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:59:19.951 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T18:59:19.951 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T18:59:19.951 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.3 2024-04-25T18:59:20.474 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:20 smithi112 systemd[1]: Stopping Ceph osd.3 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:59:20.974 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:20 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3[47225]: 2024-04-25T18:59:20.484+0000 7ff505400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:59:20.974 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:20 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3[47225]: 2024-04-25T18:59:20.484+0000 7ff505400700 -1 osd.3 111 *** Got signal Terminated *** 2024-04-25T18:59:20.974 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:20 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3[47225]: 2024-04-25T18:59:20.484+0000 7ff505400700 -1 osd.3 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T18:59:21.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:20.711+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:21.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:21 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:21.665+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:22.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:21.732+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:22.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:21.903+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:22.682 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:22.390+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:22.939 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:22 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3[47225]: 2024-04-25T18:59:22.863+0000 7ff4fbc00700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.453547+0000 front 2024-04-25T18:58:57.452988+0000 (oldest deadline 2024-04-25T18:59:22.753120+0000) 2024-04-25T18:59:22.940 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:22 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:22.626+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:22.971 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:22.908+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:23.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:22.703+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:23.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:22 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:22.937+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:23.682 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:23.428+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:23.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:23.669+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:23.907 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:23 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:23.626+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:24.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:23.920+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:24.221 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:23.848+0000 7f7b7fa3c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:58.834561+0000 front 2024-04-25T18:58:58.834769+0000 (oldest deadline 2024-04-25T18:59:23.434848+0000) 2024-04-25T18:59:24.223 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:23 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3[47225]: 2024-04-25T18:59:23.904+0000 7ff4fbc00700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.453547+0000 front 2024-04-25T18:58:57.452988+0000 (oldest deadline 2024-04-25T18:59:22.753120+0000) 2024-04-25T18:59:24.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:23 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:23.986+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:24.682 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:24 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:24.418+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:24.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:24 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:24.656+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:24.975 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3[47225]: 2024-04-25T18:59:24.871+0000 7ff4fbc00700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.453547+0000 front 2024-04-25T18:58:57.452988+0000 (oldest deadline 2024-04-25T18:59:22.753120+0000) 2024-04-25T18:59:24.975 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:24.651+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:25.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:24 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:24.955+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:25.222 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:24 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:24.803+0000 7f7b7fa3c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:58.834561+0000 front 2024-04-25T18:58:58.834769+0000 (oldest deadline 2024-04-25T18:59:23.434848+0000) 2024-04-25T18:59:25.473 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:24 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:24.982+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:25.678 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:25 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:25.401+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:25.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:25 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:25.676+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:25.973 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:25 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:25.663+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:26.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:25 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:25.955+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:26.222 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:25 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:25.810+0000 7f7b7fa3c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:58.834561+0000 front 2024-04-25T18:58:58.834769+0000 (oldest deadline 2024-04-25T18:59:23.434848+0000) 2024-04-25T18:59:26.342 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:26 smithi112 podman[113461]: 2024-04-25 18:59:26.007633566 +0000 UTC m=+5.657527196 container died b302163a34c8c96c32e84bbaf99a37dcbd75b5a3cac8cd22e6caa5df1c4e9c09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T18:59:26.342 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:25 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:25.981+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:26.648 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:26 smithi112 podman[113461]: 2024-04-25 18:59:26.460796286 +0000 UTC m=+6.110689909 container cleanup b302163a34c8c96c32e84bbaf99a37dcbd75b5a3cac8cd22e6caa5df1c4e9c09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:59:26.648 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:26 smithi112 bash[113461]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3 2024-04-25T18:59:26.682 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:26 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:26.378+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:26.950 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:26 smithi112 podman[113480]: 2024-04-25 18:59:26.753865864 +0000 UTC m=+0.736566216 container remove b302163a34c8c96c32e84bbaf99a37dcbd75b5a3cac8cd22e6caa5df1c4e9c09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:59:26.951 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:26.646+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:27.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:26 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:26.711+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:27.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:26 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:26.951+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:27.222 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:26 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:26.793+0000 7f7b7fa3c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:58.834561+0000 front 2024-04-25T18:58:58.834769+0000 (oldest deadline 2024-04-25T18:59:23.434848+0000) 2024-04-25T18:59:27.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:26 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:26.947+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:27.682 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:27.403+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:27.937 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:27 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:27.624+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:27.937 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:27 smithi112 podman[113570]: 2024-04-25 18:59:27.511048354 +0000 UTC m=+0.036981248 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:59:27.937 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:27 smithi112 podman[113570]: 2024-04-25 18:59:27.646531975 +0000 UTC m=+0.172464894 container create 3545e7a1ae65285a227516b0645de9c828ec19daec35cbb89f917c9840ec414b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T18:59:28.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:27.705+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:28.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:27 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:27.981+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:28.221 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:27 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:27.807+0000 7f7b7fa3c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:58.834561+0000 front 2024-04-25T18:58:58.834769+0000 (oldest deadline 2024-04-25T18:59:23.434848+0000) 2024-04-25T18:59:28.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:27 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:27.936+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:28.224 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:27 smithi112 podman[113570]: 2024-04-25 18:59:27.935073307 +0000 UTC m=+0.461006273 container init 3545e7a1ae65285a227516b0645de9c828ec19daec35cbb89f917c9840ec414b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T18:59:28.224 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:27 smithi112 podman[113570]: 2024-04-25 18:59:27.947455911 +0000 UTC m=+0.473388863 container start 3545e7a1ae65285a227516b0645de9c828ec19daec35cbb89f917c9840ec414b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T18:59:28.224 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:28 smithi112 podman[113570]: 2024-04-25 18:59:28.040517298 +0000 UTC m=+0.566450247 container attach 3545e7a1ae65285a227516b0645de9c828ec19daec35cbb89f917c9840ec414b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:59:28.654 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 18:59:28 smithi112 podman[113570]: 2024-04-25 18:59:28.396630582 +0000 UTC m=+0.922563535 container died 3545e7a1ae65285a227516b0645de9c828ec19daec35cbb89f917c9840ec414b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T18:59:28.682 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:28 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:28.360+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:28.931 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:28 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:28.651+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:29.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:28 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:28.725+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:29.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:28 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:28.947+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:29.221 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:28 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:28.836+0000 7f7b7fa3c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:58.834561+0000 front 2024-04-25T18:58:58.834769+0000 (oldest deadline 2024-04-25T18:59:23.434848+0000) 2024-04-25T18:59:29.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:28 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:28.929+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:29.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T18:59:29.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:59:29.358 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T18:59:29.358 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T18:59:29.358 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.4 2024-04-25T18:59:29.652 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:29 smithi161 systemd[1]: Stopping Ceph osd.4 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:59:29.682 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:29 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:29.381+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:29.914 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:29 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:29.650+0000 7f7b87450700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:59:29.914 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:29 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:29.650+0000 7f7b87450700 -1 osd.4 111 *** Got signal Terminated *** 2024-04-25T18:59:29.914 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:29 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:29.650+0000 7f7b87450700 -1 osd.4 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T18:59:29.914 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:29 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:29.795+0000 7f7b7fa3c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:58.834561+0000 front 2024-04-25T18:58:58.834769+0000 (oldest deadline 2024-04-25T18:59:23.434848+0000) 2024-04-25T18:59:29.973 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:29 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:29.694+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:30.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:29 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:29.742+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:30.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:29 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:29.912+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:30.474 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:29 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:29.971+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:30.682 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:30.417+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:30.928 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:30 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:30.719+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:31.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:30.739+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:31.221 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:30 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:30.823+0000 7f7b7fa3c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:58.834561+0000 front 2024-04-25T18:58:58.834769+0000 (oldest deadline 2024-04-25T18:59:23.434848+0000) 2024-04-25T18:59:31.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:30 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:30.901+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:31.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:30 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:30.926+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:31.743 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:31 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:31.434+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:31.974 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:31 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:31.906+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:31.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:31 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:31.720+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:32.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:31 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:31.741+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:32.221 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:31 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:31.812+0000 7f7b7fa3c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:58.834561+0000 front 2024-04-25T18:58:58.834769+0000 (oldest deadline 2024-04-25T18:59:23.434848+0000) 2024-04-25T18:59:32.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:31 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:31.921+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:32.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:31 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:31.921+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:32.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:31 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:31.959+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:32.730 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:32 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:32.438+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:32.730 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:32 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:32.438+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.233053+0000 front 2024-04-25T18:59:07.233157+0000 (oldest deadline 2024-04-25T18:59:31.933096+0000) 2024-04-25T18:59:32.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:32 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:32.704+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:32.974 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:32 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:32.907+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:32.974 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:32 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:32.907+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:33.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:32 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:32.728+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:33.221 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:32 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:32.853+0000 7f7b7fa3c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:58.834561+0000 front 2024-04-25T18:58:58.834769+0000 (oldest deadline 2024-04-25T18:59:23.434848+0000) 2024-04-25T18:59:33.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:32 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:32.946+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:33.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:32 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:32.874+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:33.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:32 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:32.874+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:33.682 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:33 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:33.422+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:33.682 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:33 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:33.422+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.233053+0000 front 2024-04-25T18:59:07.233157+0000 (oldest deadline 2024-04-25T18:59:31.933096+0000) 2024-04-25T18:59:33.934 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:33 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:33.669+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:34.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:33 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:33.713+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:34.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:33 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:33.713+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:34.221 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:33 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4[43730]: 2024-04-25T18:59:33.861+0000 7f7b7fa3c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:58.834561+0000 front 2024-04-25T18:58:58.834769+0000 (oldest deadline 2024-04-25T18:59:23.434848+0000) 2024-04-25T18:59:34.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:33 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:33.936+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:34.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:33 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:33.861+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:34.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:33 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:33.861+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:34.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:33 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:33.932+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:34.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:33 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:33.932+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:34.678 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:34 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:34.385+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:34.678 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:34 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:34.385+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.233053+0000 front 2024-04-25T18:59:07.233157+0000 (oldest deadline 2024-04-25T18:59:31.933096+0000) 2024-04-25T18:59:34.928 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:34 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:34.673+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:34.928 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:34 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:34.673+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:34.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:34 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:34.676+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:34.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:34 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:34.676+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:35.221 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:34 smithi161 podman[69336]: 2024-04-25 18:59:34.991418893 +0000 UTC m=+5.442588381 container died 641e3ac3233343ba2e8287f7b50bcd74bb424bfa71fae4af6b13de1ad9ebae9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:59:35.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:34 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:34.934+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:35.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:34 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:34.887+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:35.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:34 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:34.887+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:35.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:34 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:34.926+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:35.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:34 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:34.926+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:35.674 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:35 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:35.392+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:35.675 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:35 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:35.392+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.233053+0000 front 2024-04-25T18:59:07.233157+0000 (oldest deadline 2024-04-25T18:59:31.933096+0000) 2024-04-25T18:59:35.721 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:35 smithi161 podman[69336]: 2024-04-25 18:59:35.308945038 +0000 UTC m=+5.760114531 container cleanup 641e3ac3233343ba2e8287f7b50bcd74bb424bfa71fae4af6b13de1ad9ebae9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T18:59:35.721 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:35 smithi161 bash[69336]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4 2024-04-25T18:59:35.721 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:35 smithi161 podman[69353]: 2024-04-25 18:59:35.476735283 +0000 UTC m=+0.483776648 container remove 641e3ac3233343ba2e8287f7b50bcd74bb424bfa71fae4af6b13de1ad9ebae9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T18:59:35.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:35 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:35.711+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:35.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:35 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:35.711+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:35.957 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:35 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:35.686+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:35.957 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:35 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:35.686+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:36.172 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:35 smithi161 podman[69448]: 2024-04-25 18:59:35.90243998 +0000 UTC m=+0.095043782 container create d1ab716d276f51a18cb78a6f06a4689768b5dba8d79037c07b5320c27f33ac0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T18:59:36.172 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:35 smithi161 podman[69448]: 2024-04-25 18:59:35.825505298 +0000 UTC m=+0.018109068 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:59:36.172 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:36 smithi161 podman[69448]: 2024-04-25 18:59:36.096171651 +0000 UTC m=+0.288775408 container init d1ab716d276f51a18cb78a6f06a4689768b5dba8d79037c07b5320c27f33ac0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T18:59:36.172 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:36 smithi161 podman[69448]: 2024-04-25 18:59:36.099235755 +0000 UTC m=+0.291839513 container start d1ab716d276f51a18cb78a6f06a4689768b5dba8d79037c07b5320c27f33ac0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:59:36.173 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:35 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:35.890+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:36.173 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:35 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:35.882+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:36.173 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:35 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:35.882+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:36.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:35 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:35.955+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:36.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:35 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:35.955+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:36.471 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:36 smithi161 podman[69448]: 2024-04-25 18:59:36.171081122 +0000 UTC m=+0.363684888 container attach d1ab716d276f51a18cb78a6f06a4689768b5dba8d79037c07b5320c27f33ac0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:59:36.472 INFO:journalctl@ceph.osd.4.smithi161.stdout:Apr 25 18:59:36 smithi161 podman[69448]: 2024-04-25 18:59:36.392406103 +0000 UTC m=+0.585009864 container died d1ab716d276f51a18cb78a6f06a4689768b5dba8d79037c07b5320c27f33ac0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T18:59:36.671 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:36 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:36.376+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:36.672 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:36 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:36.410+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:36.672 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:36 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:36.410+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.233053+0000 front 2024-04-25T18:59:07.233157+0000 (oldest deadline 2024-04-25T18:59:31.933096+0000) 2024-04-25T18:59:36.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:36 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:36.669+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:36.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:36 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:36.669+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:36.950 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:36 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:36.935+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:36.950 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:36 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:36.843+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:36.950 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:36 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:36.844+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:36.969 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T18:59:36.970 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:36 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:36.670+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:36.970 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:36 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:36.670+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:37.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:59:37.007 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T18:59:37.007 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T18:59:37.008 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.5 2024-04-25T18:59:37.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:36 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:36.968+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:37.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:36 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:36.968+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:37.340 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:37 smithi171 systemd[1]: Stopping Ceph osd.5 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:59:37.637 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:37 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:37.403+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:37.638 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:37 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:37.339+0000 7fcbc0965700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:59:37.638 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:37 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:37.339+0000 7fcbc0965700 -1 osd.5 111 *** Got signal Terminated *** 2024-04-25T18:59:37.638 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:37 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:37.339+0000 7fcbc0965700 -1 osd.5 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T18:59:37.638 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:37 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:37.458+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:37.638 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:37 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:37.458+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.233053+0000 front 2024-04-25T18:59:07.233157+0000 (oldest deadline 2024-04-25T18:59:31.933096+0000) 2024-04-25T18:59:37.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:37 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:37.635+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:37.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:37 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:37.635+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:37.940 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:37 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:37.640+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:37.940 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:37 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:37.640+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:38.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:37 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:37.939+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:38.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:37 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:37.797+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:38.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:37 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:37.797+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:38.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:37 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:37.938+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:38.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:37 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:37.938+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:38.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:37 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:37.938+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:38.658 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:38 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:38.375+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:38.659 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:38 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:38.442+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:38.659 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:38 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:38.442+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.233053+0000 front 2024-04-25T18:59:07.233157+0000 (oldest deadline 2024-04-25T18:59:31.933096+0000) 2024-04-25T18:59:38.914 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:38 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:38.605+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:38.914 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:38 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:38.605+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:38.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:38 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:38.656+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:38.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:38 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:38.656+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:38.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:38 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:38.656+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:39.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:38 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:38.956+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:39.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:38 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:38.956+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:39.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:38 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:38.844+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:39.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:38 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:38.844+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:39.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:38 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:38.912+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:39.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:38 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:38.912+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:39.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:38 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:38.912+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:39.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:39 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:39.414+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:39.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:39 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:39.614+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:39.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:39 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:39.614+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:39.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:39 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:39.614+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:39.683 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:39 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:39.398+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:39.683 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:39 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:39.398+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.233053+0000 front 2024-04-25T18:59:07.233157+0000 (oldest deadline 2024-04-25T18:59:31.933096+0000) 2024-04-25T18:59:39.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:39 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:39.586+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:39.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:39 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:39.586+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:39.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:39 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:39.586+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:40.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:39 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:39.851+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:40.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:39 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:39.851+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:40.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:39 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:39.928+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:40.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:39 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:39.928+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:40.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:39 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:39.909+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:40.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:39 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:39.909+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:40.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:39 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:39.909+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:40.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:40 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:40.386+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:40.683 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:40 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:40.424+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:40.683 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:40 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:40.424+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.233053+0000 front 2024-04-25T18:59:07.233157+0000 (oldest deadline 2024-04-25T18:59:31.933096+0000) 2024-04-25T18:59:40.683 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:40 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:40.424+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.834280+0000 front 2024-04-25T18:59:14.834312+0000 (oldest deadline 2024-04-25T18:59:40.035026+0000) 2024-04-25T18:59:40.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:40 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:40.590+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:40.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:40 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:40.590+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:40.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:40 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:40.590+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:40.898 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:40 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:40.584+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:40.898 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:40 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:40.584+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:40.898 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:40 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:40.584+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:41.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:40 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:40.936+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:41.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:40 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:40.936+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:41.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:40 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:40.847+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:41.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:40 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:40.847+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:41.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:40 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:40.895+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:41.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:40 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:40.895+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:41.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:40 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:40.895+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:41.604 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:41 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:41.339+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:41.604 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:41 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:41.440+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.430567+0000 front 2024-04-25T18:58:54.430842+0000 (oldest deadline 2024-04-25T18:59:22.031154+0000) 2024-04-25T18:59:41.604 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:41 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:41.440+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.233053+0000 front 2024-04-25T18:59:07.233157+0000 (oldest deadline 2024-04-25T18:59:31.933096+0000) 2024-04-25T18:59:41.604 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:41 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T18:59:41.440+0000 7fcbbcf59700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.834280+0000 front 2024-04-25T18:59:14.834312+0000 (oldest deadline 2024-04-25T18:59:40.035026+0000) 2024-04-25T18:59:41.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:41 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:41.602+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:41.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:41 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:41.602+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:41.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:41 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:41.602+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:41.945 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:41 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:41.594+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:41.945 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:41 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:41.594+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:41.945 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:41 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:41.594+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:42.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:41 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:41.891+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:42.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:41 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:41.891+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:42.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:41 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:41.881+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:42.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:41 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:41.881+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:42.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:41 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:41.881+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:42.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:41 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:41.943+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:42.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:41 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:41.943+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:42.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:41 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:41.943+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:42.633 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:42 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:42.352+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:42.634 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:42 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:42.352+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:42.932 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:42 smithi171 podman[70498]: 2024-04-25 18:59:42.707343131 +0000 UTC m=+5.508086276 container died 501f1344dd21b94ece3c084069be79d5a370806c5ffe6c022685dcea2c72af57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:59:42.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:42 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:42.632+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:42.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:42 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:42.632+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:42.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:42 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:42.632+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:42.964 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:42 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:42.558+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:42.964 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:42 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:42.558+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:42.964 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:42 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:42.558+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:43.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:42 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:42.889+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:43.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:42 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:42.889+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:43.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:42 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:42.872+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:43.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:42 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:42.872+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:43.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:42 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:42.872+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:43.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:42 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:42.962+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:43.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:42 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:42.962+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:43.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:42 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:42.962+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:43.282 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:43 smithi171 podman[70498]: 2024-04-25 18:59:43.038966205 +0000 UTC m=+5.839709354 container cleanup 501f1344dd21b94ece3c084069be79d5a370806c5ffe6c022685dcea2c72af57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:59:43.282 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:43 smithi171 bash[70498]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5 2024-04-25T18:59:43.628 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:43 smithi171 podman[70516]: 2024-04-25 18:59:43.281643482 +0000 UTC m=+0.573400156 container remove 501f1344dd21b94ece3c084069be79d5a370806c5ffe6c022685dcea2c72af57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-04-25T18:59:43.628 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:43 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:43.389+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:43.629 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:43 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:43.389+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:43.932 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:43 smithi171 podman[70605]: 2024-04-25 18:59:43.607375576 +0000 UTC m=+0.018159569 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:59:43.932 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:43 smithi171 podman[70605]: 2024-04-25 18:59:43.716029875 +0000 UTC m=+0.126813864 container create 39599ee116e3d776fe9d531fa57f3bda04807ba749aa73c07b1cfa874587ff73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T18:59:43.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:43 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:43.626+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:43.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:43 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:43.626+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:43.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:43 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:43.626+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:43.956 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:43 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:43.607+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:43.956 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:43 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:43.607+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:43.956 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:43 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:43.607+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:44.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:43 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:43.931+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:44.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:43 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:43.931+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:44.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:43 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:43.846+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:44.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:43 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:43.846+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:44.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:43 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:43.846+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:44.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:43 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:43.954+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:44.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:43 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:43.954+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:44.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:43 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:43.954+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:44.244 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:43 smithi171 podman[70605]: 2024-04-25 18:59:43.952391708 +0000 UTC m=+0.363175699 container init 39599ee116e3d776fe9d531fa57f3bda04807ba749aa73c07b1cfa874587ff73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T18:59:44.244 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:43 smithi171 podman[70605]: 2024-04-25 18:59:43.955570034 +0000 UTC m=+0.366354031 container start 39599ee116e3d776fe9d531fa57f3bda04807ba749aa73c07b1cfa874587ff73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T18:59:44.244 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:44 smithi171 podman[70605]: 2024-04-25 18:59:44.018249035 +0000 UTC m=+0.429033054 container attach 39599ee116e3d776fe9d531fa57f3bda04807ba749aa73c07b1cfa874587ff73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T18:59:44.580 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:44 smithi171 podman[70605]: 2024-04-25 18:59:44.243634379 +0000 UTC m=+0.654418370 container died 39599ee116e3d776fe9d531fa57f3bda04807ba749aa73c07b1cfa874587ff73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-25T18:59:44.581 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:44 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:44.363+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:44.581 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:44 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:44.363+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:44.829 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:44 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:44.827+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:44.829 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:44 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:44.827+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:44.834 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 18:59:44 smithi171 podman[70631]: 2024-04-25 18:59:44.813355128 +0000 UTC m=+0.564100967 container remove 39599ee116e3d776fe9d531fa57f3bda04807ba749aa73c07b1cfa874587ff73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:59:44.835 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:44 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:44.616+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:44.835 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:44 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:44.616+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:44.835 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:44 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:44.616+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:44.840 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T18:59:44.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:59:44.878 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T18:59:44.878 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T18:59:44.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.6 2024-04-25T18:59:44.934 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:44 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:44.570+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:44.934 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:44 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:44.570+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:44.935 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:44 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:44.570+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:45.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:44 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:44.827+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:45.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:44 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:44.898+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:45.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:44 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:44.898+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:45.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:44 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:44.933+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:45.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:44 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:44.933+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:45.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:44 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:44.933+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:45.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:44 smithi112 systemd[1]: Stopping Ceph osd.6 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:59:45.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:45 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:45.332+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:45.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:45 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:45.332+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:45.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:45 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:45.580+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:45.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:45 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:45.580+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:45.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:45 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:45.580+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:45.724 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:45 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:45.453+0000 7f23bba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:59:45.724 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:45 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:45.453+0000 7f23bba00700 -1 osd.6 111 *** Got signal Terminated *** 2024-04-25T18:59:45.724 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:45 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:45.453+0000 7f23bba00700 -1 osd.6 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T18:59:45.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:45 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:45.583+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:45.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:45 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:45.583+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:45.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:45 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:45.583+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:46.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:45 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:45.919+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:46.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:45 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:45.919+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:46.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:45 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:45.789+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:46.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:45 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:45.789+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:46.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:45 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:45.789+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:46.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:45 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:45.897+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:46.223 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:45 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:45.897+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:46.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:45 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:45.897+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:46.629 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:46 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:46.354+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:46.629 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:46 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:46.354+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:46.629 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:46 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:46.354+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:46.860 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:46.586+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:46.860 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:46.586+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:46.860 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:46.586+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:46.860 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:46.586+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:46.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:46 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:46.628+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:46.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:46 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:46.628+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:46.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:46 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:46.628+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:47.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:46 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:46.904+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:47.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:46 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:46.904+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:47.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:46 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:46.816+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:47.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:46 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:46.816+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:47.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:46 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:46.816+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:47.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:46.858+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:47.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:46.858+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:47.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:46.858+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:47.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:46 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:46.858+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.893120+0000 front 2024-04-25T18:59:24.893200+0000 (oldest deadline 2024-04-25T18:59:46.593438+0000) 2024-04-25T18:59:47.597 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:47 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:47.335+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:47.597 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:47 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:47.335+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:47.597 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:47 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:47.335+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:47.897 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:47 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:47.636+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:47.897 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:47 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:47.636+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:47.897 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:47 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:47.636+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:47.897 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:47 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:47.636+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:47.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:47 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:47.595+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:47.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:47 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:47.595+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:47.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:47 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:47.595+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:47.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:47 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:47.595+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:48.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:47 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:47.859+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:48.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:47 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:47.859+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:48.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:47 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:47.792+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:48.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:47 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:47.792+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:48.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:47 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:47.792+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:48.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:47 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:47.792+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:48.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:47 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:47.895+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:48.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:47 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:47.895+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:48.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:47 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:47.895+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:48.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:47 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:47.895+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.893120+0000 front 2024-04-25T18:59:24.893200+0000 (oldest deadline 2024-04-25T18:59:46.593438+0000) 2024-04-25T18:59:48.569 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:48 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:48.305+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:48.569 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:48 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:48.305+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:48.569 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:48 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:48.305+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:48.871 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:48.608+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:48.872 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:48.608+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:48.872 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:48.608+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:48.872 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:48.608+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:48.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:48 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:48.567+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:48.941 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:48 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:48.567+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:48.942 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:48 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:48.567+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:48.942 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:48 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:48.567+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:49.211 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:48 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:48.897+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:49.211 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:48 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:48.897+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:49.211 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:48 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:48.897+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:23.192942+0000 front 2024-04-25T18:59:23.193237+0000 (oldest deadline 2024-04-25T18:59:48.492882+0000) 2024-04-25T18:59:49.212 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:48 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:48.810+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:49.212 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:48 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:48.810+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:49.212 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:48 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:48.810+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:49.212 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:48 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:48.810+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:49.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:48.869+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:49.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:48.869+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:49.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:48.869+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:49.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:48.869+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.893120+0000 front 2024-04-25T18:59:24.893200+0000 (oldest deadline 2024-04-25T18:59:46.593438+0000) 2024-04-25T18:59:49.550 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:49 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:49.294+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:49.550 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:49 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:49.294+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:49.550 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:49 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:49.294+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:49.879 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:49.560+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:49.880 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:49.560+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:49.880 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:49.560+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:49.880 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:49.560+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:49.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:49 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:49.549+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:49.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:49 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:49.549+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:49.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:49 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:49.549+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:49.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:49 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:49.549+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:50.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:49 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:49.903+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:50.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:49 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:49.903+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:50.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:49 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:49.903+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:23.192942+0000 front 2024-04-25T18:59:23.193237+0000 (oldest deadline 2024-04-25T18:59:48.492882+0000) 2024-04-25T18:59:50.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:49 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:49.827+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:50.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:49 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:49.827+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:50.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:49 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:49.827+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:50.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:49 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:49.827+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:50.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:49.877+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.287246+0000 front 2024-04-25T18:58:57.286900+0000 (oldest deadline 2024-04-25T18:59:22.586545+0000) 2024-04-25T18:59:50.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:49.877+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.289077+0000 front 2024-04-25T18:59:07.289136+0000 (oldest deadline 2024-04-25T18:59:32.589408+0000) 2024-04-25T18:59:50.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:49.877+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:12.590294+0000 front 2024-04-25T18:59:12.590386+0000 (oldest deadline 2024-04-25T18:59:37.890149+0000) 2024-04-25T18:59:50.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:49 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6[53768]: 2024-04-25T18:59:49.877+0000 7f23b7200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.893120+0000 front 2024-04-25T18:59:24.893200+0000 (oldest deadline 2024-04-25T18:59:46.593438+0000) 2024-04-25T18:59:50.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:50 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:50.317+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:50.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:50 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:50.317+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:50.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:50 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:50.317+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:50.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:50 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:50.539+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:50.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:50 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:50.539+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:50.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:50 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:50.539+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:50.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:50 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:50.539+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:50.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:50.534+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:50.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:50.534+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:50.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:50.534+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:50.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:50 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:50.534+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:51.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:50 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:50.861+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:51.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:50 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:50.861+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:51.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:50 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:50.861+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:23.192942+0000 front 2024-04-25T18:59:23.193237+0000 (oldest deadline 2024-04-25T18:59:48.492882+0000) 2024-04-25T18:59:51.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:50 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:50.836+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:51.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:50 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:50.836+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:51.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:50 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:50.836+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:51.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:50 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:50.836+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:51.338 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:51 smithi112 podman[113687]: 2024-04-25 18:59:51.011717225 +0000 UTC m=+5.713991156 container died b2ba727fb9659bb5abba43ced283d633a48cb252b1ca4492f8a021cc19d83a74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:59:51.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:51 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:51.338+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:51.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:51 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:51.338+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:51.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:51 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:51.338+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:51.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:51 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:51.538+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:51.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:51 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:51.538+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:51.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:51 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:51.538+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:51.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:51 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:51.538+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:51.699 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:51 smithi112 podman[113687]: 2024-04-25 18:59:51.455115481 +0000 UTC m=+6.157389448 container cleanup b2ba727fb9659bb5abba43ced283d633a48cb252b1ca4492f8a021cc19d83a74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T18:59:51.699 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:51 smithi112 bash[113687]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6 2024-04-25T18:59:51.699 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:51 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:51.524+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:51.700 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:51 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:51.524+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:51.700 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:51 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:51.524+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:51.700 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:51 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:51.524+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:51.973 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:51 smithi112 podman[113706]: 2024-04-25 18:59:51.697395169 +0000 UTC m=+0.671809307 container remove b2ba727fb9659bb5abba43ced283d633a48cb252b1ca4492f8a021cc19d83a74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:59:52.123 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:51 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:51.863+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:52.123 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:51 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:51.863+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:52.123 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:51 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:51.863+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:23.192942+0000 front 2024-04-25T18:59:23.193237+0000 (oldest deadline 2024-04-25T18:59:48.492882+0000) 2024-04-25T18:59:52.123 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:51 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:51.879+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:52.123 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:51 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:51.879+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:52.124 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:51 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:51.879+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:52.124 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:51 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:51.879+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:52.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:52 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:52.356+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:52.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:52 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:52.356+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:52.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:52 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:52.356+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:52.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:52 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:52.494+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:52.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:52 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:52.494+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:52.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:52 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:52.494+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:52.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:52 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:52.494+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:52.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:52 smithi112 podman[113796]: 2024-04-25 18:59:52.472924233 +0000 UTC m=+0.037653617 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:59:52.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:52 smithi112 podman[113796]: 2024-04-25 18:59:52.615443172 +0000 UTC m=+0.180172579 container create b7f769eef47dcbc3f7efab03b5993c3dadd096d8d590d8f8c697bf35f4f7368b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T18:59:52.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:52 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:52.524+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:52.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:52 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:52.524+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:52.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:52 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:52.524+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:52.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:52 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:52.524+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:53.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:52 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:52.867+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:53.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:52 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:52.867+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:53.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:52 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:52.867+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:23.192942+0000 front 2024-04-25T18:59:23.193237+0000 (oldest deadline 2024-04-25T18:59:48.492882+0000) 2024-04-25T18:59:53.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:52 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:52.897+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:53.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:52 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:52.897+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:53.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:52 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:52.897+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:53.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:52 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:52.897+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:53.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:52 smithi112 podman[113796]: 2024-04-25 18:59:52.902271712 +0000 UTC m=+0.467001126 container init b7f769eef47dcbc3f7efab03b5993c3dadd096d8d590d8f8c697bf35f4f7368b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T18:59:53.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:52 smithi112 podman[113796]: 2024-04-25 18:59:52.914306806 +0000 UTC m=+0.479036186 container start b7f769eef47dcbc3f7efab03b5993c3dadd096d8d590d8f8c697bf35f4f7368b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T18:59:53.224 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:52 smithi112 podman[113796]: 2024-04-25 18:59:52.984311192 +0000 UTC m=+0.549040663 container attach b7f769eef47dcbc3f7efab03b5993c3dadd096d8d590d8f8c697bf35f4f7368b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T18:59:53.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:53 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:53.307+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:53.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:53 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:53.307+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:53.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:53 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:53.307+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:53.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:53 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:53.474+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:53.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:53 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:53.474+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:53.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:53 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:53.474+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:53.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:53 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:53.474+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:53.724 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 18:59:53 smithi112 podman[113796]: 2024-04-25 18:59:53.362039235 +0000 UTC m=+0.926768686 container died b7f769eef47dcbc3f7efab03b5993c3dadd096d8d590d8f8c697bf35f4f7368b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T18:59:53.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:53 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:53.496+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:53.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:53 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:53.496+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:53.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:53 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:53.496+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:53.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:53 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:53.496+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:54.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:53 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:53.891+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:54.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:53 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:53.891+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:54.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:53 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:53.891+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:23.192942+0000 front 2024-04-25T18:59:23.193237+0000 (oldest deadline 2024-04-25T18:59:48.492882+0000) 2024-04-25T18:59:54.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:53 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:53.895+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:54.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:53 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:53.895+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:54.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:53 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:53.895+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:54.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:53 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:53.895+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:54.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T18:59:54.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T18:59:54.335 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T18:59:54.335 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T18:59:54.335 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.7 2024-04-25T18:59:54.584 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:54 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:54.297+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:54.584 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:54 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:54.297+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:54.584 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:54 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:54.297+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:54.584 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:54 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:54.463+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:54.584 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:54 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:54.463+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:54.584 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:54 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:54.463+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:54.584 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:54 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:54.463+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:54.721 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:54 smithi161 systemd[1]: Stopping Ceph osd.7 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T18:59:54.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:54 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:54.457+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:54.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:54 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:54.457+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:54.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:54 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:54.457+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:54.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:54 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:54.457+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:55.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:54 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:54.913+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:55.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:54 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:54.913+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:55.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:54 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:54.913+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:55.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:54 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:54.913+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:55.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:54 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:54.758+0000 7f97ee284700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:59:55.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:54 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:54.759+0000 7f97ee284700 -1 osd.7 111 *** Got signal Terminated *** 2024-04-25T18:59:55.226 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:54 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:54.759+0000 7f97ee284700 -1 osd.7 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T18:59:55.226 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:54 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:54.920+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:55.226 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:54 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:54.920+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:55.226 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:54 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:54.920+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:23.192942+0000 front 2024-04-25T18:59:23.193237+0000 (oldest deadline 2024-04-25T18:59:48.492882+0000) 2024-04-25T18:59:55.603 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:55 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:55.296+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:55.603 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:55 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:55.296+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:55.603 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:55 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:55.296+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:55.603 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:55 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:55.487+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:55.603 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:55 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:55.487+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:55.603 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:55 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:55.487+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:55.603 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:55 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:55.487+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:55.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:55 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:55.452+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:55.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:55 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:55.452+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:55.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:55 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:55.452+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:55.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:55 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:55.452+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:56.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:55 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:55.872+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:56.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:55 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:55.872+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:56.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:55 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:55.872+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:23.192942+0000 front 2024-04-25T18:59:23.193237+0000 (oldest deadline 2024-04-25T18:59:48.492882+0000) 2024-04-25T18:59:56.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:55 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:55.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:56.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:55 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:55.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:56.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:55 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:55.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:56.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:55 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:55.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:56.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:55 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:55.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T18:59:56.504 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:56 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:56.250+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:56.504 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:56 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:56.250+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:56.504 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:56 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:56.250+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:56.504 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:56 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:56.250+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T18:59:56.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:56 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:56.454+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:56.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:56 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:56.454+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:56.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:56 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:56.454+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:56.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:56 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:56.454+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:56.790 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:56 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:56.502+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:56.790 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:56 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:56.502+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:56.790 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:56 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:56.502+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:56.791 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:56 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:56.502+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:57.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:56 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:56.916+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:57.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:56 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:56.916+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:57.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:56 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:56.916+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:23.192942+0000 front 2024-04-25T18:59:23.193237+0000 (oldest deadline 2024-04-25T18:59:48.492882+0000) 2024-04-25T18:59:57.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:56 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:56.931+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:57.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:56 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:56.931+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:57.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:56 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:56.931+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:57.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:56 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:56.931+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:57.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:56 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:56.931+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T18:59:57.466 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:57 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:57.207+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:57.466 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:57 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:57.207+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:57.466 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:57 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:57.207+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:57.466 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:57 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:57.207+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T18:59:57.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:57 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:57.409+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:57.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:57 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:57.409+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:57.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:57 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:57.409+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:57.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:57 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:57.409+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:57.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:57 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:57.465+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:57.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:57 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:57.465+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:57.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:57 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:57.465+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:57.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:57 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:57.465+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:57.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:57 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:57.465+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T18:59:58.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:57 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:57.911+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:58.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:57 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:57.911+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:58.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:57 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:57.911+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:23.192942+0000 front 2024-04-25T18:59:23.193237+0000 (oldest deadline 2024-04-25T18:59:48.492882+0000) 2024-04-25T18:59:58.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:57 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:57.948+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:58.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:57 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:57.948+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:58.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:57 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:57.948+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:58.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:57 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:57.948+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:58.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:57 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:57.948+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T18:59:58.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:58 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:58.245+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:58.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:58 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:58.245+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:58.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:58 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:58.245+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:58.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:58 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:58.245+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T18:59:58.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:58 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:58.479+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:58.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:58 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:58.479+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:58.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:58 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:58.479+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:58.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:58 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:58.479+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:58.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:58 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:58.479+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T18:59:58.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:58 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:58.447+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:58.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:58 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:58.447+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:58.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:58 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:58.447+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:58.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:58 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:58.447+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T18:59:59.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:58 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:58.904+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:09.289575+0000 front 2024-04-25T18:59:09.289665+0000 (oldest deadline 2024-04-25T18:59:30.989771+0000) 2024-04-25T18:59:59.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:58 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:58.904+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.691063+0000 front 2024-04-25T18:59:15.690966+0000 (oldest deadline 2024-04-25T18:59:37.990947+0000) 2024-04-25T18:59:59.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 18:59:58 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7[50732]: 2024-04-25T18:59:58.904+0000 7f97ea878700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:23.192942+0000 front 2024-04-25T18:59:23.193237+0000 (oldest deadline 2024-04-25T18:59:48.492882+0000) 2024-04-25T18:59:59.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:58 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:58.928+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T18:59:59.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:58 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:58.928+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T18:59:59.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:58 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:58.928+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T18:59:59.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:58 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:58.928+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T18:59:59.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:58 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:58.928+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T18:59:59.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:59 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:59.274+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T18:59:59.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:59 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:59.274+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T18:59:59.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:59 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:59.274+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T18:59:59.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 18:59:59 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T18:59:59.274+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T18:59:59.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:59 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:59.462+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T18:59:59.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:59 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:59.462+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T18:59:59.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:59 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:59.462+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T18:59:59.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:59 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:59.462+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T18:59:59.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 18:59:59 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T18:59:59.462+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T18:59:59.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:59 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:59.449+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T18:59:59.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:59 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:59.449+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T18:59:59.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:59 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:59.449+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T18:59:59.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 18:59:59 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T18:59:59.449+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:00.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:59 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:59.927+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:00.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:59 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:59.927+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:00.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:59 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:59.927+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:00.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:59 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:59.927+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:00.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 18:59:59 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T18:59:59.927+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:00.576 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 19:00:00 smithi161 podman[69554]: 2024-04-25 19:00:00.293511378 +0000 UTC m=+5.765718661 container died c1f442d35b5a004a80621d6ab682281aafd3b8312f410f211aca6a6debc336dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T19:00:00.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:00 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:00.298+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T19:00:00.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:00 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:00.298+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T19:00:00.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:00 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:00.298+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T19:00:00.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:00 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:00.298+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T19:00:00.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:00 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:00.499+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:00.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:00 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:00.499+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:00.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:00 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:00.499+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:00.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:00 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:00.499+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:00.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:00 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:00.499+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:00.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:00 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:00.454+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:00.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:00 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:00.454+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:00.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:00 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:00.454+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:00.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:00 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:00.454+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:00.955 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 19:00:00 smithi161 podman[69554]: 2024-04-25 19:00:00.575424872 +0000 UTC m=+6.047632162 container cleanup c1f442d35b5a004a80621d6ab682281aafd3b8312f410f211aca6a6debc336dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:00:00.955 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 19:00:00 smithi161 bash[69554]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7 2024-04-25T19:00:00.955 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 19:00:00 smithi161 podman[69575]: 2024-04-25 19:00:00.809864957 +0000 UTC m=+0.517586131 container remove c1f442d35b5a004a80621d6ab682281aafd3b8312f410f211aca6a6debc336dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:00:01.221 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 19:00:01 smithi161 podman[69667]: 2024-04-25 19:00:01.05053168 +0000 UTC m=+0.018215805 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:00:01.222 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 19:00:01 smithi161 podman[69667]: 2024-04-25 19:00:01.185709797 +0000 UTC m=+0.153393912 container create 647b63c29225165966cb6db84fb6a2362fff54a7a8130c6555968db4dbf68ff0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T19:00:01.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:00 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:00.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:01.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:00 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:00.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:01.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:00 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:00.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:01.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:00 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:00.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:01.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:00 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:00.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:01.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:01 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:01.256+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T19:00:01.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:01 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:01.256+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T19:00:01.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:01 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:01.256+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T19:00:01.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:01 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:01.256+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T19:00:01.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:01 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:01.452+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:01.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:01 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:01.452+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:01.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:01 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:01.452+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:01.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:01 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:01.452+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:01.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:01 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:01.452+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:01.714 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 19:00:01 smithi161 podman[69667]: 2024-04-25 19:00:01.421548612 +0000 UTC m=+0.389232728 container init 647b63c29225165966cb6db84fb6a2362fff54a7a8130c6555968db4dbf68ff0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T19:00:01.714 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 19:00:01 smithi161 podman[69667]: 2024-04-25 19:00:01.424615347 +0000 UTC m=+0.392299463 container start 647b63c29225165966cb6db84fb6a2362fff54a7a8130c6555968db4dbf68ff0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T19:00:01.714 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 19:00:01 smithi161 podman[69667]: 2024-04-25 19:00:01.496297268 +0000 UTC m=+0.463981394 container attach 647b63c29225165966cb6db84fb6a2362fff54a7a8130c6555968db4dbf68ff0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T19:00:01.971 INFO:journalctl@ceph.osd.7.smithi161.stdout:Apr 25 19:00:01 smithi161 podman[69667]: 2024-04-25 19:00:01.713838615 +0000 UTC m=+0.681522729 container died 647b63c29225165966cb6db84fb6a2362fff54a7a8130c6555968db4dbf68ff0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T19:00:01.972 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:01 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:01.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:01.972 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:01 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:01.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:01.972 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:01 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:01.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:01.972 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:01 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:01.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:01.972 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:01 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:01.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:01.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:01.475+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:01.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:01.475+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:01.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:01.475+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:01.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:01 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:01.475+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:02.301 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T19:00:02.339 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:00:02.339 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T19:00:02.339 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T19:00:02.339 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.8 2024-04-25T19:00:02.627 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:02.410+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:02.627 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:02.410+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:02.627 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:02.410+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:02.627 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:02.410+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:02.627 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:02.410+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:02.627 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:02.218+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T19:00:02.627 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:02.218+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T19:00:02.627 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:02.218+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T19:00:02.628 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:02.218+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T19:00:02.628 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:02 smithi171 systemd[1]: Stopping Ceph osd.8 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:00:02.628 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:02.625+0000 7fccd05cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:00:02.628 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:02.625+0000 7fccd05cf700 -1 osd.8 111 *** Got signal Terminated *** 2024-04-25T19:00:02.628 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:02 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:02.625+0000 7fccd05cf700 -1 osd.8 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:00:02.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:02.453+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:02.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:02.453+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:02.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:02.453+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:02.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:02 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:02.453+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:03.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:02 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:02.947+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:03.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:02 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:02.947+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:03.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:02 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:02.947+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:03.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:02 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:02.947+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:03.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:02 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:02.947+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:03.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:03 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:03.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:03.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:03 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:03.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:03.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:03 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:03.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:03.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:03 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:03.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:03.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:03 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:03.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:03.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:03 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:03.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:03.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:03 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:03.259+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T19:00:03.684 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:03 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:03.259+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T19:00:03.684 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:03 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:03.259+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T19:00:03.684 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:03 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:03.259+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T19:00:03.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:03 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:03.495+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:03.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:03 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:03.495+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:03.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:03 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:03.495+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:03.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:03 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:03.495+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:04.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:03 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:03.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:04.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:03 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:03.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:04.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:03 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:03.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:04.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:03 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:03.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:04.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:03 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:03.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:04.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:04 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:04.302+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T19:00:04.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:04 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:04.302+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T19:00:04.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:04 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:04.302+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T19:00:04.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:04 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:04.302+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T19:00:04.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:04 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:04.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:04.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:04 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:04.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:04.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:04 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:04.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:04.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:04 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:04.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:04.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:04 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:04.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:04.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:04 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:04.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:04.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:04 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:04.538+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:04.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:04 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:04.538+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:04.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:04 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:04.538+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:04.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:04 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:04.538+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:05.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:04 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:04.933+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:05.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:04 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:04.933+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:05.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:04 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:04.933+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:05.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:04 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:04.933+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:05.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:04 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:04.933+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:05.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:05 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:05.310+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T19:00:05.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:05 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:05.310+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T19:00:05.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:05 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:05.310+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T19:00:05.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:05 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:05.310+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T19:00:05.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:05 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:05.360+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:05.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:05 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:05.360+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:05.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:05 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:05.360+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:05.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:05 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:05.360+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:05.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:05 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:05.360+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:05.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:05 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:05.360+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:05.881 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:05 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:05.518+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:05.881 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:05 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:05.518+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:05.881 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:05 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:05.518+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:05.881 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:05 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:05.518+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:06.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:05 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:05.968+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:06.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:05 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:05.968+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:06.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:05 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:05.968+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:06.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:05 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:05.968+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:06.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:05 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:05.968+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:06.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:06.265+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T19:00:06.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:06.265+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T19:00:06.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:06.265+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T19:00:06.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:06.265+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T19:00:06.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:06.375+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:06.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:06.375+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:06.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:06.375+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:06.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:06.375+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:06.684 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:06.375+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:06.684 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:06 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:06.375+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:06.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:06 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:06.536+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:06.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:06 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:06.536+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:06.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:06 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:06.536+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:06.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:06 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:06.536+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:06.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:06 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:06.536+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.833990+0000 front 2024-04-25T18:59:40.834041+0000 (oldest deadline 2024-04-25T19:00:06.134188+0000) 2024-04-25T19:00:07.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.008+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:07.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.008+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:07.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.008+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:07.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.008+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:07.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.008+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:07.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.008+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:07.682 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:07 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:07.307+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:03.606613+0000 front 2024-04-25T18:59:03.606762+0000 (oldest deadline 2024-04-25T18:59:35.407144+0000) 2024-04-25T19:00:07.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:07 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:07.307+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:15.407649+0000 front 2024-04-25T18:59:15.407695+0000 (oldest deadline 2024-04-25T18:59:41.808541+0000) 2024-04-25T19:00:07.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:07 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:07.307+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:24.511062+0000 front 2024-04-25T18:59:24.510886+0000 (oldest deadline 2024-04-25T18:59:46.210478+0000) 2024-04-25T19:00:07.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:07 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:07.307+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:33.811937+0000 front 2024-04-25T18:59:33.811918+0000 (oldest deadline 2024-04-25T18:59:55.511892+0000) 2024-04-25T19:00:07.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:07 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8[52082]: 2024-04-25T19:00:07.307+0000 7fccccbc3700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:42.314751+0000 front 2024-04-25T18:59:42.314708+0000 (oldest deadline 2024-04-25T19:00:07.014826+0000) 2024-04-25T19:00:07.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:07 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:07.399+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:07.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:07 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:07.399+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:07.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:07 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:07.399+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:07.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:07 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:07.399+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:07.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:07 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:07.399+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:07.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:07 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:07.399+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:07.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:07 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:07.495+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:07.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:07 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:07.495+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:07.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:07 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:07.495+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:07.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:07 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:07.495+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:07.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:07 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:07.495+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.833990+0000 front 2024-04-25T18:59:40.834041+0000 (oldest deadline 2024-04-25T19:00:06.134188+0000) 2024-04-25T19:00:08.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.966+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:08.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.966+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:08.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.966+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:08.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.966+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:08.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.966+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:08.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:07 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:07.966+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:08.294 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:08 smithi171 podman[70725]: 2024-04-25 19:00:08.033339643 +0000 UTC m=+5.504634200 container died c34404916c5c189a19cefe5341ebe75a1a9213ce0a9d1cf4e926762e92956d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:00:08.660 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:08 smithi171 podman[70725]: 2024-04-25 19:00:08.391997077 +0000 UTC m=+5.863291611 container cleanup c34404916c5c189a19cefe5341ebe75a1a9213ce0a9d1cf4e926762e92956d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T19:00:08.661 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:08 smithi171 bash[70725]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8 2024-04-25T19:00:08.661 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:08 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:08.385+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:08.661 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:08 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:08.385+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:08.661 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:08 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:08.385+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:08.661 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:08 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:08.385+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:08.661 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:08 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:08.385+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:08.661 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:08 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:08.385+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:08.932 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:08 smithi171 podman[70742]: 2024-04-25 19:00:08.659794575 +0000 UTC m=+0.620285281 container remove c34404916c5c189a19cefe5341ebe75a1a9213ce0a9d1cf4e926762e92956d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T19:00:08.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:08 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:08.481+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:08.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:08 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:08.481+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:08.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:08 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:08.481+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:08.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:08 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:08.481+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:08.974 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:08 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:08.481+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.833990+0000 front 2024-04-25T18:59:40.834041+0000 (oldest deadline 2024-04-25T19:00:06.134188+0000) 2024-04-25T19:00:09.263 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:09 smithi171 podman[70835]: 2024-04-25 19:00:08.914604673 +0000 UTC m=+0.017935161 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:00:09.263 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:09 smithi171 podman[70835]: 2024-04-25 19:00:09.035542005 +0000 UTC m=+0.138872494 container create 6fdf8aaa4c1561cbc22e774414ccb7971dcf915f21ea22e9930ba62d61c38988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:00:09.263 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:09 smithi171 podman[70835]: 2024-04-25 19:00:09.262708314 +0000 UTC m=+0.366038805 container init 6fdf8aaa4c1561cbc22e774414ccb7971dcf915f21ea22e9930ba62d61c38988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T19:00:09.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:08 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:08.980+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:09.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:08 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:08.980+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:09.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:08 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:08.980+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:09.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:08 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:08.980+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:09.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:08 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:08.980+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:09.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:08 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:08.980+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:09.566 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:09 smithi171 podman[70835]: 2024-04-25 19:00:09.266064538 +0000 UTC m=+0.369395027 container start 6fdf8aaa4c1561cbc22e774414ccb7971dcf915f21ea22e9930ba62d61c38988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-25T19:00:09.567 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:09 smithi171 podman[70835]: 2024-04-25 19:00:09.363172357 +0000 UTC m=+0.466502853 container attach 6fdf8aaa4c1561cbc22e774414ccb7971dcf915f21ea22e9930ba62d61c38988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T19:00:09.567 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:09 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:09.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:09.567 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:09 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:09.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:09.567 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:09 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:09.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:09.567 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:09 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:09.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:09.567 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:09 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:09.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:09.567 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:09 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:09.382+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:09.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:09 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:09.460+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:09.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:09 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:09.460+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:09.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:09 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:09.460+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:09.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:09 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:09.460+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:09.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:09 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:09.460+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.833990+0000 front 2024-04-25T18:59:40.834041+0000 (oldest deadline 2024-04-25T19:00:06.134188+0000) 2024-04-25T19:00:09.909 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 19:00:09 smithi171 podman[70835]: 2024-04-25 19:00:09.565712089 +0000 UTC m=+0.669042585 container died 6fdf8aaa4c1561cbc22e774414ccb7971dcf915f21ea22e9930ba62d61c38988 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T19:00:10.218 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T19:00:10.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:00:10.256 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T19:00:10.256 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T19:00:10.256 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.9 2024-04-25T19:00:10.467 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:10 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:10.345+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:10.467 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:10 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:10.345+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:10.467 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:10 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:10.345+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:10.467 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:10 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:10.345+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:10.468 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:10 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:10.345+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:10.468 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:10 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:10.345+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:10.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:10 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:10.026+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:10.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:10 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:10.026+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:10.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:10 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:10.026+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:10.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:10 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:10.026+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:10.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:10 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:10.026+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:10.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:10 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:10.026+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:10.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:10 smithi112 systemd[1]: Stopping Ceph osd.9 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:00:10.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:10 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:10.449+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:10.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:10 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:10.449+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:10.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:10 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:10.449+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:10.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:10 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:10.449+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:10.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:10 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:10.449+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.833990+0000 front 2024-04-25T18:59:40.834041+0000 (oldest deadline 2024-04-25T19:00:06.134188+0000) 2024-04-25T19:00:11.224 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:10 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:10.812+0000 7f0a3d600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:00:11.224 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:10 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:10.812+0000 7f0a3d600700 -1 osd.9 111 *** Got signal Terminated *** 2024-04-25T19:00:11.224 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:10 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:10.812+0000 7f0a3d600700 -1 osd.9 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:00:11.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.029+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:11.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.029+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:11.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.029+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:11.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.029+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:11.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.029+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:11.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.029+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:11.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:11 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:11.357+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:11.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:11 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:11.357+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:11.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:11 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:11.357+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:11.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:11 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:11.357+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:11.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:11 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:11.357+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:11.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:11 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:11.357+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:11.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:11 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:11.408+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:11.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:11 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:11.408+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:11.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:11 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:11.408+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:11.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:11 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:11.408+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:11.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:11 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:11.408+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.833990+0000 front 2024-04-25T18:59:40.834041+0000 (oldest deadline 2024-04-25T19:00:06.134188+0000) 2024-04-25T19:00:12.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.998+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:12.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.998+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:12.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.998+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:12.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.998+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:12.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.998+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:12.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:11 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:11.998+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:12.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:12 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:12.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:12.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:12 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:12.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:12.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:12 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:12.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:12.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:12 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:12.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:12.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:12 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:12.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:12.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:12 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:12.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:12.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:12 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:12.361+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:12.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:12 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:12.361+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:12.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:12 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:12.361+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:12.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:12 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:12.361+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:12.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:12 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:12.361+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.833990+0000 front 2024-04-25T18:59:40.834041+0000 (oldest deadline 2024-04-25T19:00:06.134188+0000) 2024-04-25T19:00:13.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:12 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:12.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:13.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:12 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:12.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:13.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:12 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:12.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:13.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:12 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:12.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:13.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:12 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:12.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:13.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:12 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:12.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:13.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:12 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:12.975+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:13.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:13 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:13.411+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:13.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:13 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:13.411+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:13.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:13 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:13.411+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:13.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:13 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:13.412+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:13.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:13 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:13.412+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:13.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:13 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:13.412+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:13.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:13 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:13.354+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:13.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:13 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:13.354+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:13.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:13 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:13.354+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:13.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:13 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:13.354+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:13.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:13 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:13.354+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.833990+0000 front 2024-04-25T18:59:40.834041+0000 (oldest deadline 2024-04-25T19:00:06.134188+0000) 2024-04-25T19:00:13.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:13 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:13.354+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:49.036915+0000 front 2024-04-25T18:59:49.037080+0000 (oldest deadline 2024-04-25T19:00:12.537160+0000) 2024-04-25T19:00:14.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.025+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:14.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.025+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:14.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.025+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:14.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.025+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:14.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.025+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:14.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.025+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:14.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.025+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:14.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:14 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:14.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:14.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:14 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:14.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:14.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:14 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:14.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:14.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:14 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:14.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:14.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:14 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:14.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:14.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:14 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:14.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:14.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:14 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:14.346+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:14.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:14 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:14.346+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:14.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:14 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:14.346+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:14.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:14 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:14.346+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:14.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:14 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:14.346+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.833990+0000 front 2024-04-25T18:59:40.834041+0000 (oldest deadline 2024-04-25T19:00:06.134188+0000) 2024-04-25T19:00:14.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:14 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:14.346+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:49.036915+0000 front 2024-04-25T18:59:49.037080+0000 (oldest deadline 2024-04-25T19:00:12.537160+0000) 2024-04-25T19:00:15.471 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.979+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:15.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.979+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:15.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.979+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:15.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.979+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:15.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.979+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:15.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.979+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:15.472 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:14 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:14.979+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:15.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:15.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:15.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:15.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:15.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:15.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:15.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:15.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:15.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:15.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:15.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:15.404+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:15.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:15.372+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:57.817556+0000 front 2024-04-25T18:58:57.817382+0000 (oldest deadline 2024-04-25T18:59:20.717174+0000) 2024-04-25T19:00:15.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:15.372+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.123841+0000 front 2024-04-25T18:59:08.123935+0000 (oldest deadline 2024-04-25T18:59:34.024097+0000) 2024-04-25T19:00:15.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:15.372+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:14.025372+0000 front 2024-04-25T18:59:14.025289+0000 (oldest deadline 2024-04-25T18:59:39.325552+0000) 2024-04-25T19:00:15.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:15.372+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.628144+0000 front 2024-04-25T18:59:21.628058+0000 (oldest deadline 2024-04-25T18:59:46.328076+0000) 2024-04-25T19:00:15.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:15.372+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.833990+0000 front 2024-04-25T18:59:40.834041+0000 (oldest deadline 2024-04-25T19:00:06.134188+0000) 2024-04-25T19:00:15.724 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:15 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9[60373]: 2024-04-25T19:00:15.372+0000 7f0a38e00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:49.036915+0000 front 2024-04-25T18:59:49.037080+0000 (oldest deadline 2024-04-25T19:00:12.537160+0000) 2024-04-25T19:00:16.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:15.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:16.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:15.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:16.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:15.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:16.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:15.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:16.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:15.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:16.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:15.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:16.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:15 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:15.952+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:16.596 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:16 smithi112 podman[113901]: 2024-04-25 19:00:16.232308448 +0000 UTC m=+5.618012664 container died d659890e9c2aa4c715e52231b946e77ac8445a0bf52c9c45cae9b5522ca57fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T19:00:16.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:16 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:16.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:16.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:16 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:16.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:16.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:16 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:16.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:16.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:16 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:16.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:16.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:16 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:16.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:16.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:16 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:16.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:16.959 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:16 smithi112 podman[113901]: 2024-04-25 19:00:16.764917783 +0000 UTC m=+6.150622019 container cleanup d659890e9c2aa4c715e52231b946e77ac8445a0bf52c9c45cae9b5522ca57fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:00:16.959 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:16 smithi112 bash[113901]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9 2024-04-25T19:00:17.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:16 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:16.924+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:17.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:16 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:16.924+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:17.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:16 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:16.924+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:17.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:16 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:16.924+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:17.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:16 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:16.924+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:17.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:16 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:16.924+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:17.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:16 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:16.924+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:17.223 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:16 smithi112 podman[113920]: 2024-04-25 19:00:16.957643157 +0000 UTC m=+0.708324694 container remove d659890e9c2aa4c715e52231b946e77ac8445a0bf52c9c45cae9b5522ca57fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:00:17.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:17 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:17.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:17.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:17 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:17.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:17.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:17 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:17.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:17.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:17 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:17.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:17.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:17 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:17.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:17.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:17 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:17.400+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:18.072 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:17 smithi112 podman[114005]: 2024-04-25 19:00:17.700433813 +0000 UTC m=+0.033712821 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:00:18.072 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:17 smithi112 podman[114005]: 2024-04-25 19:00:17.817123386 +0000 UTC m=+0.150402421 container create b2dd5916f86f296c4dfa68af95886bb7ac1905ccbafed03c896ebf042926420c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:00:18.072 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:18 smithi112 podman[114005]: 2024-04-25 19:00:18.070453956 +0000 UTC m=+0.403733031 container init b2dd5916f86f296c4dfa68af95886bb7ac1905ccbafed03c896ebf042926420c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T19:00:18.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:17 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:17.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:18.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:17 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:17.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:18.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:17 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:17.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:18.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:17 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:17.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:18.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:17 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:17.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:18.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:17 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:17.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:18.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:17 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:17.953+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:18.475 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:18 smithi112 podman[114005]: 2024-04-25 19:00:18.080970639 +0000 UTC m=+0.414249639 container start b2dd5916f86f296c4dfa68af95886bb7ac1905ccbafed03c896ebf042926420c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:00:18.475 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:18 smithi112 podman[114005]: 2024-04-25 19:00:18.15267738 +0000 UTC m=+0.485956442 container attach b2dd5916f86f296c4dfa68af95886bb7ac1905ccbafed03c896ebf042926420c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T19:00:18.893 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:18 smithi112 podman[114005]: 2024-04-25 19:00:18.531420858 +0000 UTC m=+0.864699897 container died b2dd5916f86f296c4dfa68af95886bb7ac1905ccbafed03c896ebf042926420c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T19:00:18.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:18.435+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:18.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:18.435+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:18.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:18.435+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:18.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:18.435+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:18.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:18.435+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:18.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:18 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:18.435+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:19.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:18.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:19.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:18.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:19.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:18.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:19.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:18.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:19.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:18.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:19.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:18.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:19.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:18 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:18.926+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:19.403 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T19:00:19.470 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:19 smithi112 podman[114030]: 2024-04-25 19:00:19.356095437 +0000 UTC m=+0.809366071 container remove b2dd5916f86f296c4dfa68af95886bb7ac1905ccbafed03c896ebf042926420c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T19:00:19.470 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:19 smithi112 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.9.service: Deactivated successfully. 2024-04-25T19:00:19.471 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:19 smithi112 systemd[1]: Stopped Ceph osd.9 for 2fe18a52-0332-11ef-bc93-c7b262605968. 2024-04-25T19:00:19.471 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 19:00:19 smithi112 systemd[1]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.9.service: Consumed 2min 29.798s CPU time. 2024-04-25T19:00:19.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:00:19.501 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T19:00:19.501 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T19:00:19.501 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.10 2024-04-25T19:00:19.828 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:19 smithi161 systemd[1]: Stopping Ceph osd.10 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:00:19.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:19.437+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:19.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:19.437+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:19.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:19.437+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:19.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:19.437+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:19.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:19.437+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:19.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:19 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:19.437+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:20.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:19.826+0000 7f2d8b0ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:00:20.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:19.826+0000 7f2d8b0ea700 -1 osd.10 111 *** Got signal Terminated *** 2024-04-25T19:00:20.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:19.826+0000 7f2d8b0ea700 -1 osd.10 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:00:20.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:19.886+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:20.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:19.886+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:20.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:19.886+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:20.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:19.886+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:20.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:19.886+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:20.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:19.886+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:20.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:19 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:19.886+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:20.899 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:20.458+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:20.899 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:20.458+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:20.900 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:20.458+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:20.900 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:20.458+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:20.900 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:20.458+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:20.900 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:20.458+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:21.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:20.908+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:21.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:20.908+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:21.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:20.908+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:21.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:20.908+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:21.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:20.908+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:21.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:20.908+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:21.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:20 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:20.908+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:21.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:21.496+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:21.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:21.496+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:21.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:21.496+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:21.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:21.496+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:21.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:21.496+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:21.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:21.496+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:21.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:21 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:21.496+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:22.142 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:21.878+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:22.142 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:21.878+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:22.143 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:21.878+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:22.143 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:21.878+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:22.143 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:21.878+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:22.143 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:21.878+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:22.143 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:21 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:21.878+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:22.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:22.482+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:22.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:22.482+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:22.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:22.482+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:22.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:22.482+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:22.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:22.482+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:22.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:22.482+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:22.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:22 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:22.482+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:23.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:22.831+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:23.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:22.831+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:23.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:22.831+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:23.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:22.831+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:23.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:22.831+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:23.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:22.831+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:23.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:22 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:22.831+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:23.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:23.454+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:23.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:23.454+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:23.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:23.454+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:23.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:23.454+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:23.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:23.454+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:23.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:23.454+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:23.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:23.454+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:24.222 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:23.803+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:59:00.111223+0000 front 2024-04-25T18:59:00.111119+0000 (oldest deadline 2024-04-25T18:59:21.711641+0000) 2024-04-25T19:00:24.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:23.803+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:07.012633+0000 front 2024-04-25T18:59:07.012668+0000 (oldest deadline 2024-04-25T18:59:31.112764+0000) 2024-04-25T19:00:24.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:23.803+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:16.413853+0000 front 2024-04-25T18:59:16.414052+0000 (oldest deadline 2024-04-25T18:59:41.614414+0000) 2024-04-25T19:00:24.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:23.803+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:21.615328+0000 front 2024-04-25T18:59:21.615129+0000 (oldest deadline 2024-04-25T18:59:46.914889+0000) 2024-04-25T19:00:24.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:23.803+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:29.215658+0000 front 2024-04-25T18:59:29.215689+0000 (oldest deadline 2024-04-25T18:59:55.115902+0000) 2024-04-25T19:00:24.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:23.803+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:40.417163+0000 front 2024-04-25T18:59:40.417111+0000 (oldest deadline 2024-04-25T19:00:06.317121+0000) 2024-04-25T19:00:24.223 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:23 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10[57322]: 2024-04-25T19:00:23.803+0000 7f2d836d6700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T18:59:46.317946+0000 front 2024-04-25T18:59:46.318036+0000 (oldest deadline 2024-04-25T19:00:12.217675+0000) 2024-04-25T19:00:24.834 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:24 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:24.497+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:24.834 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:24 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:24.497+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:24.834 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:24 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:24.497+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:24.834 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:24 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:24.497+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:24.834 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:24 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:24.497+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:24.834 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:24 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:24.497+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:24.834 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:24 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:24.497+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:25.594 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:25 smithi161 podman[69783]: 2024-04-25 19:00:25.328923981 +0000 UTC m=+5.635776180 container died 060d4db3ff5ae0bfd5a6f004c423cbd43b6422c4e8683eb501a7546ff68ca97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T19:00:25.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:25 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:25.455+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:25.934 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:25 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:25.455+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:25.934 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:25 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:25.455+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:25.934 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:25 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:25.455+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:25.934 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:25 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:25.455+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:25.934 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:25 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:25.455+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:25.934 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:25 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:25.455+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:25.961 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:25 smithi161 podman[69783]: 2024-04-25 19:00:25.68423204 +0000 UTC m=+5.991084245 container cleanup 060d4db3ff5ae0bfd5a6f004c423cbd43b6422c4e8683eb501a7546ff68ca97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:00:25.962 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:25 smithi161 bash[69783]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10 2024-04-25T19:00:26.221 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:25 smithi161 podman[69812]: 2024-04-25 19:00:25.960580767 +0000 UTC m=+0.627248117 container remove 060d4db3ff5ae0bfd5a6f004c423cbd43b6422c4e8683eb501a7546ff68ca97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:00:26.589 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:26 smithi161 podman[69903]: 2024-04-25 19:00:26.20260643 +0000 UTC m=+0.017931541 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:00:26.589 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:26 smithi161 podman[69903]: 2024-04-25 19:00:26.302961557 +0000 UTC m=+0.118286656 container create 15f6aed4df53c331d3f89a922b5bfdbe5c94e3e50d25fc95fe276361975e486e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T19:00:26.589 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:26 smithi161 podman[69903]: 2024-04-25 19:00:26.513479396 +0000 UTC m=+0.328804495 container init 15f6aed4df53c331d3f89a922b5bfdbe5c94e3e50d25fc95fe276361975e486e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:00:26.589 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:26 smithi161 podman[69903]: 2024-04-25 19:00:26.516449997 +0000 UTC m=+0.331775103 container start 15f6aed4df53c331d3f89a922b5bfdbe5c94e3e50d25fc95fe276361975e486e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:00:26.589 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:26 smithi161 podman[69903]: 2024-04-25 19:00:26.588428582 +0000 UTC m=+0.403753694 container attach 15f6aed4df53c331d3f89a922b5bfdbe5c94e3e50d25fc95fe276361975e486e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:00:26.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:26 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:26.419+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:26.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:26 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:26.419+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:26.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:26 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:26.419+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:26.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:26 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:26.419+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:26.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:26 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:26.419+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:26.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:26 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:26.419+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:26.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:26 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:26.419+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:26.972 INFO:journalctl@ceph.osd.10.smithi161.stdout:Apr 25 19:00:26 smithi161 podman[69903]: 2024-04-25 19:00:26.808128227 +0000 UTC m=+0.623453327 container died 15f6aed4df53c331d3f89a922b5bfdbe5c94e3e50d25fc95fe276361975e486e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T19:00:27.398 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T19:00:27.437 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:00:27.438 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T19:00:27.438 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T19:00:27.438 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.11 2024-04-25T19:00:27.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:27.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:27.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:27.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:27.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:27.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:27.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:27.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:27.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:27.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:27.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:27.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:27.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:27.389+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:27.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:27 smithi171 systemd[1]: Stopping Ceph osd.11 for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:00:28.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:27.734+0000 7fd9f10e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:00:28.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:27.734+0000 7fd9f10e5700 -1 osd.11 111 *** Got signal Terminated *** 2024-04-25T19:00:28.182 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:27 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:27.734+0000 7fd9f10e5700 -1 osd.11 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:00:28.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:28 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:28.393+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:28.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:28 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:28.393+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:28.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:28 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:28.393+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:28.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:28 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:28.393+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:28.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:28 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:28.393+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:28.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:28 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:28.393+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:28.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:28 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:28.393+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:28.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:28 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:28.393+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-25T19:00:06.014495+0000 front 2024-04-25T19:00:06.014517+0000 (oldest deadline 2024-04-25T19:00:28.314509+0000) 2024-04-25T19:00:29.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:29 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:29.388+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:29.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:29 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:29.388+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:29.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:29 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:29.388+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:29.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:29 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:29.388+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:29.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:29 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:29.388+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:29.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:29 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:29.388+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:29.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:29 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:29.388+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:29.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:29 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:29.388+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-25T19:00:06.014495+0000 front 2024-04-25T19:00:06.014517+0000 (oldest deadline 2024-04-25T19:00:28.314509+0000) 2024-04-25T19:00:30.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:30.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:30.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:30.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:30.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:30.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:30.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:30.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:30.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:30.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:30.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:30.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:30.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:30.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:30.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:30 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:30.426+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-25T19:00:06.014495+0000 front 2024-04-25T19:00:06.014517+0000 (oldest deadline 2024-04-25T19:00:28.314509+0000) 2024-04-25T19:00:31.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:31 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:31.451+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:31.932 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:31 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:31.451+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:31.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:31 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:31.451+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:31.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:31 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:31.451+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:31.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:31 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:31.451+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:31.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:31 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:31.451+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:31.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:31 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:31.451+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:31.933 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:31 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:31.451+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-25T19:00:06.014495+0000 front 2024-04-25T19:00:06.014517+0000 (oldest deadline 2024-04-25T19:00:28.314509+0000) 2024-04-25T19:00:32.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:32 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:32.414+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T18:58:54.802217+0000 front 2024-04-25T18:58:54.802192+0000 (oldest deadline 2024-04-25T18:59:20.702118+0000) 2024-04-25T19:00:32.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:32 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:32.414+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-04-25T18:59:08.105071+0000 front 2024-04-25T18:59:08.105021+0000 (oldest deadline 2024-04-25T18:59:33.405074+0000) 2024-04-25T19:00:32.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:32 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:32.414+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T18:59:13.405417+0000 front 2024-04-25T18:59:13.405722+0000 (oldest deadline 2024-04-25T18:59:38.006059+0000) 2024-04-25T19:00:32.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:32 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:32.414+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T18:59:22.007860+0000 front 2024-04-25T18:59:22.007964+0000 (oldest deadline 2024-04-25T18:59:47.307684+0000) 2024-04-25T19:00:32.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:32 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:32.414+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-04-25T18:59:30.709255+0000 front 2024-04-25T18:59:30.709224+0000 (oldest deadline 2024-04-25T18:59:56.609076+0000) 2024-04-25T19:00:32.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:32 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:32.414+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T18:59:41.810201+0000 front 2024-04-25T18:59:41.810241+0000 (oldest deadline 2024-04-25T19:00:02.910228+0000) 2024-04-25T19:00:32.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:32 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:32.414+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-04-25T18:59:57.313408+0000 front 2024-04-25T18:59:57.313394+0000 (oldest deadline 2024-04-25T19:00:20.813308+0000) 2024-04-25T19:00:32.683 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:32 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11[58685]: 2024-04-25T19:00:32.414+0000 7fd9e96d1700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-25T19:00:06.014495+0000 front 2024-04-25T19:00:06.014517+0000 (oldest deadline 2024-04-25T19:00:28.314509+0000) 2024-04-25T19:00:33.404 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:33 smithi171 podman[70964]: 2024-04-25 19:00:33.112708182 +0000 UTC m=+5.480137794 container died 68e61bfcdb0593706b90d3825bfa7af094ac73c3674eaca414c1905c40a017ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T19:00:33.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:33 smithi171 podman[70964]: 2024-04-25 19:00:33.502091813 +0000 UTC m=+5.869521423 container cleanup 68e61bfcdb0593706b90d3825bfa7af094ac73c3674eaca414c1905c40a017ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T19:00:33.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:33 smithi171 bash[70964]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11 2024-04-25T19:00:34.119 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:33 smithi171 podman[70981]: 2024-04-25 19:00:33.769642776 +0000 UTC m=+0.651853395 container remove 68e61bfcdb0593706b90d3825bfa7af094ac73c3674eaca414c1905c40a017ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T19:00:34.424 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:34 smithi171 podman[71073]: 2024-04-25 19:00:34.020835842 +0000 UTC m=+0.017979170 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:00:34.424 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:34 smithi171 podman[71073]: 2024-04-25 19:00:34.120375715 +0000 UTC m=+0.117519073 container create 5432c54a940d84464f222819cdab8425bcac8beaa82901fbe86aa942be049875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T19:00:34.424 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:34 smithi171 podman[71073]: 2024-04-25 19:00:34.339964814 +0000 UTC m=+0.337108130 container init 5432c54a940d84464f222819cdab8425bcac8beaa82901fbe86aa942be049875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:00:34.424 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:34 smithi171 podman[71073]: 2024-04-25 19:00:34.342995463 +0000 UTC m=+0.340138784 container start 5432c54a940d84464f222819cdab8425bcac8beaa82901fbe86aa942be049875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:00:34.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:34 smithi171 podman[71073]: 2024-04-25 19:00:34.423167309 +0000 UTC m=+0.420310631 container attach 5432c54a940d84464f222819cdab8425bcac8beaa82901fbe86aa942be049875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T19:00:34.682 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:34 smithi171 podman[71073]: 2024-04-25 19:00:34.637663383 +0000 UTC m=+0.634806702 container died 5432c54a940d84464f222819cdab8425bcac8beaa82901fbe86aa942be049875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:00:35.194 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 19:00:35 smithi171 podman[71099]: 2024-04-25 19:00:35.176663421 +0000 UTC m=+0.529409521 container remove 5432c54a940d84464f222819cdab8425bcac8beaa82901fbe86aa942be049875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T19:00:35.198 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T19:00:35.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:00:35.237 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T19:00:35.237 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T19:00:35.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.a 2024-04-25T19:00:35.723 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 19:00:35 smithi112 systemd[1]: Stopping Ceph mds.a for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:00:36.224 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 19:00:35 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a[67160]: 2024-04-25T19:00:35.839+0000 7f9761a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:00:36.224 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 19:00:35 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a[67160]: 2024-04-25T19:00:35.839+0000 7f9761a00700 -1 mds.a *** got signal Terminated *** 2024-04-25T19:00:38.708 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 19:00:38 smithi112 podman[114133]: 2024-04-25 19:00:38.378396583 +0000 UTC m=+2.674714766 container died d1f85f3692b806b87e0a5bc12b10997449c9bb65c785436058eefb2169fb6bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:00:38.974 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 19:00:38 smithi112 podman[114133]: 2024-04-25 19:00:38.866304061 +0000 UTC m=+3.162622243 container cleanup d1f85f3692b806b87e0a5bc12b10997449c9bb65c785436058eefb2169fb6bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T19:00:38.974 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 19:00:38 smithi112 bash[114133]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a 2024-04-25T19:00:39.474 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 19:00:39 smithi112 podman[114147]: 2024-04-25 19:00:39.084101822 +0000 UTC m=+0.694411428 container remove d1f85f3692b806b87e0a5bc12b10997449c9bb65c785436058eefb2169fb6bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:00:39.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T19:00:39.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:00:39.668 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T19:00:39.668 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T19:00:39.668 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.d 2024-04-25T19:00:40.224 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 19:00:39 smithi112 systemd[1]: Stopping Ceph mds.d for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:00:40.724 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 19:00:40 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-d[65980]: 2024-04-25T19:00:40.268+0000 7fbf6c200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:00:40.724 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 19:00:40 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-d[65980]: 2024-04-25T19:00:40.268+0000 7fbf6c200700 -1 mds.d *** got signal Terminated *** 2024-04-25T19:00:41.901 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 19:00:41 smithi112 podman[114254]: 2024-04-25 19:00:41.558261815 +0000 UTC m=+1.422992116 container died aa1c2907f74f8b0893f0e19d519d753a82d55b3f7d0c680bc38c9884493297ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T19:00:42.224 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 19:00:42 smithi112 podman[114254]: 2024-04-25 19:00:42.087176529 +0000 UTC m=+1.951906854 container cleanup aa1c2907f74f8b0893f0e19d519d753a82d55b3f7d0c680bc38c9884493297ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416) 2024-04-25T19:00:42.224 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 19:00:42 smithi112 bash[114254]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-d 2024-04-25T19:00:42.723 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 19:00:42 smithi112 podman[114268]: 2024-04-25 19:00:42.388073102 +0000 UTC m=+0.809773430 container remove aa1c2907f74f8b0893f0e19d519d753a82d55b3f7d0c680bc38c9884493297ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T19:00:42.873 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T19:00:42.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:00:42.971 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T19:00:42.971 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T19:00:42.972 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.g 2024-04-25T19:00:43.473 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 19:00:43 smithi112 systemd[1]: Stopping Ceph mds.g for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:00:43.974 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 19:00:43 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: 2024-04-25T19:00:43.614+0000 7f8146800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:00:43.974 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 19:00:43 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g[86787]: 2024-04-25T19:00:43.614+0000 7f8146800700 -1 mds.g *** got signal Terminated *** 2024-04-25T19:00:46.723 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 19:00:46 smithi112 podman[114374]: 2024-04-25 19:00:46.37707834 +0000 UTC m=+2.938385038 container died bb6a0b6f2936a37191de5db5c455224f873181f6a900f5cb5d2eac2dbda5f124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T19:00:46.974 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 19:00:46 smithi112 podman[114374]: 2024-04-25 19:00:46.848704452 +0000 UTC m=+3.410011189 container cleanup bb6a0b6f2936a37191de5db5c455224f873181f6a900f5cb5d2eac2dbda5f124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T19:00:46.974 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 19:00:46 smithi112 bash[114374]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g 2024-04-25T19:00:47.473 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 19:00:47 smithi112 podman[114389]: 2024-04-25 19:00:47.133325766 +0000 UTC m=+0.738527884 container remove bb6a0b6f2936a37191de5db5c455224f873181f6a900f5cb5d2eac2dbda5f124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T19:00:47.685 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T19:00:47.782 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:00:47.782 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T19:00:47.782 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T19:00:47.782 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.j 2024-04-25T19:00:47.973 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 19:00:47 smithi112 systemd[1]: Stopping Ceph mds.j for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:00:48.724 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 19:00:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-j[65559]: 2024-04-25T19:00:48.384+0000 7f5d6ae00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:00:48.724 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 19:00:48 smithi112 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-j[65559]: 2024-04-25T19:00:48.384+0000 7f5d6ae00700 -1 mds.j *** got signal Terminated *** 2024-04-25T19:00:52.653 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 19:00:52 smithi112 podman[114495]: 2024-04-25 19:00:52.314828517 +0000 UTC m=+4.096392863 container died 8bec2ea380494db5be8516275e6112b5d9788239006a0809fdb8552a3b640bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T19:00:52.973 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 19:00:52 smithi112 podman[114495]: 2024-04-25 19:00:52.810925388 +0000 UTC m=+4.592489684 container cleanup 8bec2ea380494db5be8516275e6112b5d9788239006a0809fdb8552a3b640bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T19:00:52.973 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 19:00:52 smithi112 bash[114495]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-j 2024-04-25T19:00:53.409 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 19:00:53 smithi112 podman[114521]: 2024-04-25 19:00:53.145334627 +0000 UTC m=+0.810107957 container remove 8bec2ea380494db5be8516275e6112b5d9788239006a0809fdb8552a3b640bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T19:00:53.448 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T19:00:53.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:00:53.549 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T19:00:53.549 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T19:00:53.549 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.b 2024-04-25T19:00:53.843 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 19:00:53 smithi161 systemd[1]: Stopping Ceph mds.b for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:00:54.098 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 19:00:53 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b[62830]: 2024-04-25T19:00:53.841+0000 7ff1ccc20700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:00:54.099 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 19:00:53 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b[62830]: 2024-04-25T19:00:53.841+0000 7ff1ccc20700 -1 mds.b *** got signal Terminated *** 2024-04-25T19:00:58.168 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 19:00:57 smithi161 podman[70008]: 2024-04-25 19:00:57.852634825 +0000 UTC m=+4.119317330 container died 742d4f112ada94b3d143b172b774ff3ef71567edc561a188d9e6a8ae4f49a99c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:00:58.471 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 19:00:58 smithi161 podman[70008]: 2024-04-25 19:00:58.257663146 +0000 UTC m=+4.524345649 container cleanup 742d4f112ada94b3d143b172b774ff3ef71567edc561a188d9e6a8ae4f49a99c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T19:00:58.471 INFO:journalctl@ceph.mds.b.smithi161.stdout:Apr 25 19:00:58 smithi161 bash[70008]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-b 2024-04-25T19:00:58.641 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T19:00:58.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:00:58.676 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T19:00:58.676 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T19:00:58.676 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.e 2024-04-25T19:00:58.972 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 19:00:58 smithi161 systemd[1]: Stopping Ceph mds.e for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:00:59.471 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 19:00:58 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-e[62197]: 2024-04-25T19:00:58.982+0000 7f964b414700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:00:59.471 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 19:00:58 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-e[62197]: 2024-04-25T19:00:58.982+0000 7f964b414700 -1 mds.e *** got signal Terminated *** 2024-04-25T19:01:00.870 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 19:01:00 smithi161 podman[70142]: 2024-04-25 19:01:00.548367248 +0000 UTC m=+1.652166032 container died 4b70ac893a33dd0fee57803a8e6d0e303025d075144f14651314b3b9060340c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T19:01:00.870 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 19:01:00 smithi161 podman[70142]: 2024-04-25 19:01:00.869182622 +0000 UTC m=+1.972981415 container cleanup 4b70ac893a33dd0fee57803a8e6d0e303025d075144f14651314b3b9060340c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-25T19:01:01.221 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 19:01:00 smithi161 bash[70142]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-e 2024-04-25T19:01:01.221 INFO:journalctl@ceph.mds.e.smithi161.stdout:Apr 25 19:01:01 smithi161 podman[70156]: 2024-04-25 19:01:01.070241984 +0000 UTC m=+0.523590222 container remove 4b70ac893a33dd0fee57803a8e6d0e303025d075144f14651314b3b9060340c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:01:01.290 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T19:01:01.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:01:01.322 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T19:01:01.322 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T19:01:01.322 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.h 2024-04-25T19:01:01.632 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 19:01:01 smithi161 systemd[1]: Stopping Ceph mds.h for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:01:01.971 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 19:01:01 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h[62511]: 2024-04-25T19:01:01.630+0000 7f4b861d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:01:01.971 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 19:01:01 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h[62511]: 2024-04-25T19:01:01.630+0000 7f4b861d0700 -1 mds.h *** got signal Terminated *** 2024-04-25T19:01:05.924 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 19:01:05 smithi161 podman[70279]: 2024-04-25 19:01:05.64326568 +0000 UTC m=+4.100816469 container died 929ece4abbccd3e617fb90e43c560e82362aeea00b07d5ce213698b6dcf9c297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T19:01:06.221 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 19:01:06 smithi161 podman[70279]: 2024-04-25 19:01:06.00594324 +0000 UTC m=+4.463494026 container cleanup 929ece4abbccd3e617fb90e43c560e82362aeea00b07d5ce213698b6dcf9c297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T19:01:06.221 INFO:journalctl@ceph.mds.h.smithi161.stdout:Apr 25 19:01:06 smithi161 bash[70279]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-h 2024-04-25T19:01:06.380 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T19:01:06.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:01:06.413 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T19:01:06.413 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T19:01:06.414 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.k 2024-04-25T19:01:06.971 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 19:01:06 smithi161 systemd[1]: Stopping Ceph mds.k for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:01:06.972 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 19:01:06 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-k[61878]: 2024-04-25T19:01:06.725+0000 7f19bdde7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:01:06.972 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 19:01:06 smithi161 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-k[61878]: 2024-04-25T19:01:06.725+0000 7f19bdde7700 -1 mds.k *** got signal Terminated *** 2024-04-25T19:01:10.826 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 19:01:10 smithi161 podman[70400]: 2024-04-25 19:01:10.521889983 +0000 UTC m=+3.891015730 container died fcfccf6165d139fea0bebd47fee996c3163605c531653deda8e76d4cf8a8218a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T19:01:10.826 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 19:01:10 smithi161 podman[70400]: 2024-04-25 19:01:10.825200899 +0000 UTC m=+4.194326658 container cleanup fcfccf6165d139fea0bebd47fee996c3163605c531653deda8e76d4cf8a8218a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T19:01:11.221 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 19:01:10 smithi161 bash[70400]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-k 2024-04-25T19:01:11.221 INFO:journalctl@ceph.mds.k.smithi161.stdout:Apr 25 19:01:11 smithi161 podman[70414]: 2024-04-25 19:01:11.034350443 +0000 UTC m=+0.508811949 container remove fcfccf6165d139fea0bebd47fee996c3163605c531653deda8e76d4cf8a8218a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T19:01:11.248 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T19:01:11.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:01:11.279 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T19:01:11.279 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T19:01:11.279 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.c 2024-04-25T19:01:11.643 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 19:01:11 smithi171 systemd[1]: Stopping Ceph mds.c for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:01:11.932 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 19:01:11 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c[63670]: 2024-04-25T19:01:11.641+0000 7f5fd9798700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:01:11.932 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 19:01:11 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c[63670]: 2024-04-25T19:01:11.641+0000 7f5fd9798700 -1 mds.c *** got signal Terminated *** 2024-04-25T19:01:14.682 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 19:01:14 smithi171 podman[71206]: 2024-04-25 19:01:14.313595568 +0000 UTC m=+2.847073966 container died 5e7c956c31a12f78536dd8c34fe13e62097d75e5683045f49778579c342db605 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:01:15.061 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 19:01:14 smithi171 podman[71206]: 2024-04-25 19:01:14.858375577 +0000 UTC m=+3.391853974 container cleanup 5e7c956c31a12f78536dd8c34fe13e62097d75e5683045f49778579c342db605 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T19:01:15.062 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 19:01:14 smithi171 bash[71206]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-c 2024-04-25T19:01:15.271 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T19:01:15.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:01:15.307 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T19:01:15.308 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T19:01:15.308 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.f 2024-04-25T19:01:15.639 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 19:01:15 smithi171 systemd[1]: Stopping Ceph mds.f for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:01:15.932 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 19:01:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-f[63358]: 2024-04-25T19:01:15.637+0000 7f7bdd1c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:01:15.932 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 19:01:15 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-f[63358]: 2024-04-25T19:01:15.637+0000 7f7bdd1c7700 -1 mds.f *** got signal Terminated *** 2024-04-25T19:01:19.682 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 19:01:19 smithi171 podman[71329]: 2024-04-25 19:01:19.347218306 +0000 UTC m=+3.827921205 container died 36603a081b1af301bc4947de505436c212dced7a0744487be60ce1ce2782cac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T19:01:19.981 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 19:01:19 smithi171 podman[71329]: 2024-04-25 19:01:19.78806359 +0000 UTC m=+4.268766558 container cleanup 36603a081b1af301bc4947de505436c212dced7a0744487be60ce1ce2782cac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T19:01:19.982 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 19:01:19 smithi171 bash[71329]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-f 2024-04-25T19:01:20.207 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T19:01:20.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:01:20.239 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T19:01:20.239 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T19:01:20.239 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.i 2024-04-25T19:01:20.567 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 19:01:20 smithi171 systemd[1]: Stopping Ceph mds.i for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:01:20.932 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 19:01:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i[63990]: 2024-04-25T19:01:20.566+0000 7f8fa9d4b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:01:20.932 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 19:01:20 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i[63990]: 2024-04-25T19:01:20.566+0000 7f8fa9d4b700 -1 mds.i *** got signal Terminated *** 2024-04-25T19:01:22.682 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 19:01:22 smithi171 podman[71449]: 2024-04-25 19:01:22.316329544 +0000 UTC m=+1.855394663 container died 87941493cbf977f657c19588b4198a67c31e6063058d79ec257e08585603aeff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T19:01:23.068 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 19:01:22 smithi171 podman[71449]: 2024-04-25 19:01:22.816337576 +0000 UTC m=+2.355402697 container cleanup 87941493cbf977f657c19588b4198a67c31e6063058d79ec257e08585603aeff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:01:23.068 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 19:01:22 smithi171 bash[71449]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-i 2024-04-25T19:01:23.242 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T19:01:23.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:01:23.275 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T19:01:23.275 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T19:01:23.275 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.l 2024-04-25T19:01:23.588 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 19:01:23 smithi171 systemd[1]: Stopping Ceph mds.l for 2fe18a52-0332-11ef-bc93-c7b262605968... 2024-04-25T19:01:23.932 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 19:01:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-l[63049]: 2024-04-25T19:01:23.587+0000 7fbfc61b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:01:23.932 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 19:01:23 smithi171 ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-l[63049]: 2024-04-25T19:01:23.587+0000 7fbfc61b2700 -1 mds.l *** got signal Terminated *** 2024-04-25T19:01:27.855 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 19:01:27 smithi171 podman[71570]: 2024-04-25 19:01:27.5937866 +0000 UTC m=+4.107502330 container died a9264b1d0db3be078a41eb9e8a99656bafc069cd7eacb5f8ae19f5ae3876086a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:01:28.182 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 19:01:27 smithi171 podman[71570]: 2024-04-25 19:01:27.945467983 +0000 UTC m=+4.459183701 container cleanup a9264b1d0db3be078a41eb9e8a99656bafc069cd7eacb5f8ae19f5ae3876086a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:01:28.182 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 19:01:27 smithi171 bash[71570]: ceph-2fe18a52-0332-11ef-bc93-c7b262605968-mds-l 2024-04-25T19:01:28.371 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2fe18a52-0332-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T19:01:28.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:01:28.402 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T19:01:28.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T19:01:28.725 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T19:01:33.730 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T19:01:33.921 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T19:01:36.336 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T19:01:36.527 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T19:01:38.983 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:01:39.077 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:01:39.108 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:01:39.134 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T19:01:39.136 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614/remote/smithi112/crash 2024-04-25T19:01:39.137 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/crash -- . 2024-04-25T19:01:39.268 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T19:01:39.268 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-04-25T19:01:39.273 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614/remote/smithi161/crash 2024-04-25T19:01:39.274 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/crash -- . 2024-04-25T19:01:39.310 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T19:01:39.311 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-04-25T19:01:39.313 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614/remote/smithi171/crash 2024-04-25T19:01:39.313 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/crash -- . 2024-04-25T19:01:39.353 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /var/lib/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T19:01:39.354 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now 2024-04-25T19:01:39.355 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T19:01:39.355 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T19:01:39.461 INFO:tasks.cephadm:Compressing logs... 2024-04-25T19:01:39.461 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:01:39.465 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:01:39.468 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:01:39.493 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.log 2024-04-25T19:01:39.493 INFO:teuthology.orchestra.run.smithi171.stderr:: No such file or directory 2024-04-25T19:01:39.493 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T19:01:39.493 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T19:01:39.494 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T19:01:39.494 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T19:01:39.494 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T19:01:39.494 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T19:01:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.log 2024-04-25T19:01:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:: No such file or directory 2024-04-25T19:01:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T19:01:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.7.log/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.log: 2024-04-25T19:01:39.499 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T19:01:39.500 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T19:01:39.500 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.b.log: --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T19:01:39.500 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T19:01:39.500 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T19:01:39.500 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T19:01:39.500 INFO:teuthology.orchestra.run.smithi161.stderr: 87.3%/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.h.log: -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T19:01:39.500 INFO:teuthology.orchestra.run.smithi161.stderr: 90.5% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T19:01:39.502 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T19:01:39.502 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T19:01:39.503 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.4% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T19:01:39.503 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T19:01:39.503 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T19:01:39.503 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T19:01:39.503 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T19:01:39.504 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T19:01:39.504 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.11.log: 94.4% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T19:01:39.504 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T19:01:39.506 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.8.log: 92.1% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T19:01:39.506 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T19:01:39.508 INFO:teuthology.orchestra.run.smithi161.stderr: 92.1% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T19:01:39.508 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T19:01:39.508 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T19:01:39.508 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T19:01:39.508 INFO:teuthology.orchestra.run.smithi161.stderr: 83.9% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T19:01:39.513 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T19:01:39.515 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T19:01:39.515 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T19:01:39.520 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T19:01:39.520 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T19:01:39.521 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T19:01:39.521 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.log 2024-04-25T19:01:39.522 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T19:01:39.523 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T19:01:39.524 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T19:01:39.525 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T19:01:39.525 INFO:teuthology.orchestra.run.smithi171.stderr: 94.6% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T19:01:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.j.log: 87.3% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T19:01:39.526 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T19:01:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.3.log: 90.3% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T19:01:39.527 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T19:01:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T19:01:39.528 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.k.log: 91.0% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T19:01:39.529 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T19:01:39.530 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T19:01:39.530 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T19:01:39.531 INFO:teuthology.orchestra.run.smithi112.stderr: 94.4% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T19:01:39.532 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T19:01:39.533 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T19:01:39.533 INFO:teuthology.orchestra.run.smithi161.stderr: 94.5% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T19:01:39.534 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.2% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T19:01:39.536 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T19:01:39.536 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T19:01:39.540 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T19:01:39.542 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T19:01:39.547 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T19:01:39.555 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 90.9% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T19:01:39.556 INFO:teuthology.orchestra.run.smithi112.stderr: 94.3% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T19:01:39.565 INFO:teuthology.orchestra.run.smithi112.stderr: 93.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T19:01:39.576 INFO:teuthology.orchestra.run.smithi161.stderr: 94.0% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T19:01:39.739 INFO:teuthology.orchestra.run.smithi171.stderr: 94.7% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T19:01:39.811 INFO:teuthology.orchestra.run.smithi171.stderr: 93.8% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T19:01:39.947 INFO:teuthology.orchestra.run.smithi112.stderr: 90.1% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T19:01:40.049 INFO:teuthology.orchestra.run.smithi171.stderr: 91.8% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T19:01:40.145 INFO:teuthology.orchestra.run.smithi161.stderr: 91.6% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T19:01:40.944 INFO:teuthology.orchestra.run.smithi112.stderr: 90.4% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T19:01:41.248 INFO:teuthology.orchestra.run.smithi112.stderr: 93.9% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T19:01:42.292 INFO:teuthology.orchestra.run.smithi171.stderr: 93.6% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T19:01:42.810 INFO:teuthology.orchestra.run.smithi171.stderr: 94.0% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T19:01:43.122 INFO:teuthology.orchestra.run.smithi112.stderr: 95.7% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T19:01:43.437 INFO:teuthology.orchestra.run.smithi161.stderr: 94.2% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T19:01:44.146 INFO:teuthology.orchestra.run.smithi112.stderr: 93.6% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T19:01:44.178 INFO:teuthology.orchestra.run.smithi112.stderr: 94.3% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T19:01:44.455 INFO:teuthology.orchestra.run.smithi161.stderr: 94.2% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T19:01:44.572 INFO:teuthology.orchestra.run.smithi171.stderr: 94.0% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T19:01:44.837 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T19:01:44.885 INFO:teuthology.orchestra.run.smithi161.stderr: 93.9% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T19:01:44.971 INFO:teuthology.orchestra.run.smithi112.stderr: 93.5% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T19:01:45.578 INFO:teuthology.orchestra.run.smithi171.stderr: 94.5% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T19:01:45.583 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-04-25T19:01:45.583 INFO:teuthology.orchestra.run.smithi171.stderr:real 0m6.097s 2024-04-25T19:01:45.583 INFO:teuthology.orchestra.run.smithi171.stderr:user 0m17.766s 2024-04-25T19:01:45.583 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m0.655s 2024-04-25T19:01:46.452 INFO:teuthology.orchestra.run.smithi112.stderr: 93.8% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T19:01:46.458 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T19:01:46.458 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m6.960s 2024-04-25T19:01:46.458 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m26.613s 2024-04-25T19:01:46.458 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m2.221s 2024-04-25T19:02:07.919 INFO:teuthology.orchestra.run.smithi161.stderr: 93.3% -- replaced with /var/log/ceph/2fe18a52-0332-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T19:02:07.920 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-04-25T19:02:07.920 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m28.438s 2024-04-25T19:02:07.921 INFO:teuthology.orchestra.run.smithi161.stderr:user 0m47.124s 2024-04-25T19:02:07.921 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m1.616s 2024-04-25T19:02:07.921 INFO:tasks.cephadm:Archiving logs... 2024-04-25T19:02:07.921 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614/remote/smithi112/log 2024-04-25T19:02:08.031 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:02:11.363 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614/remote/smithi161/log 2024-04-25T19:02:11.365 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:02:18.325 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614/remote/smithi171/log 2024-04-25T19:02:18.326 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:02:20.864 INFO:tasks.cephadm:Removing cluster... 2024-04-25T19:02:20.864 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 --force 2024-04-25T19:02:21.186 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T19:02:21.853 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 --force 2024-04-25T19:02:22.043 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T19:02:22.421 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2fe18a52-0332-11ef-bc93-c7b262605968 --force 2024-04-25T19:02:22.613 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 2fe18a52-0332-11ef-bc93-c7b262605968 2024-04-25T19:02:22.930 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T19:02:22.930 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T19:02:22.989 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T19:02:23.006 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T19:02:23.021 INFO:tasks.cephadm:Teardown complete 2024-04-25T19:02:23.022 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T19:02:23.045 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T19:02:23.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:02:23.053 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:02:23.056 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:02:23.209 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T19:02:23.221 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T19:02:23.222 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:02:23.230 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:02:23.232 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:02:23.244 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:02:23.246 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:02:23.250 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:02:23.250 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-04-25T19:02:23.250 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +863us[ +885us] +/- 42ms 2024-04-25T19:02:23.250 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -1472us[-1472us] +/- 65ms 2024-04-25T19:02:23.251 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +217us[ +217us] +/- 75ms 2024-04-25T19:02:23.251 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:02:23.251 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:02:23.252 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-25T19:02:23.252 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +857us[ +858us] +/- 42ms 2024-04-25T19:02:23.252 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -1469us[-1469us] +/- 65ms 2024-04-25T19:02:23.252 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +218us[ +219us] +/- 75ms 2024-04-25T19:02:23.252 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:02:23.276 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:02:23.286 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:02:23.287 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T19:02:23.287 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 24 +889us[ +893us] +/- 42ms 2024-04-25T19:02:23.287 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 24 -1402us[-1397us] +/- 65ms 2024-04-25T19:02:23.287 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 87 +268us[ +273us] +/- 74ms 2024-04-25T19:02:23.287 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:02:23.288 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T19:02:23.300 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T19:02:23.301 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T19:02:23.311 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:02:23.527 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:02:23.597 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:02:23.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:02:23.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:02:23.758 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:02:23.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:02:23.788 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:02:23.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:02:23.815 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T19:02:23.828 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T19:02:23.840 INFO:teuthology.task.internal:Duration was 2207.612571 seconds 2024-04-25T19:02:23.840 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T19:02:23.852 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T19:02:23.852 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:02:23.860 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:02:23.862 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:02:23.899 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:02:23.900 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:02:24.000 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:02:24.262 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T19:02:24.262 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T19:02:24.262 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:02:24.328 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-04-25T19:02:24.328 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:02:24.358 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T19:02:24.358 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:02:24.384 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T19:02:24.384 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:02:24.389 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:02:24.401 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:02:24.449 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T19:02:24.450 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:02:24.454 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:02:24.466 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:02:25.154 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T19:02:25.167 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T19:02:25.167 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:02:25.242 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:02:25.267 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:02:25.292 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T19:02:25.303 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:02:25.311 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:02:25.313 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:02:25.334 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-04-25T19:02:25.355 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-04-25T19:02:25.401 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-04-25T19:02:25.443 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:02:25.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:02:25.490 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:02:25.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:02:25.504 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:02:25.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:02:25.519 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T19:02:25.534 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T19:02:25.535 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614/remote/smithi112 2024-04-25T19:02:25.535 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:02:25.640 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614/remote/smithi161 2024-04-25T19:02:25.640 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:02:25.675 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614/remote/smithi171 2024-04-25T19:02:25.676 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:02:25.712 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T19:02:25.713 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:02:25.720 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:02:25.722 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:02:25.767 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T19:02:25.782 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T19:02:25.782 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T19:02:25.793 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T19:02:25.793 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:02:25.799 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:02:25.801 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:02:25.813 INFO:teuthology.orchestra.run.smithi161.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:02 /home/ubuntu/cephtest 2024-04-25T19:02:25.818 INFO:teuthology.orchestra.run.smithi171.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:02 /home/ubuntu/cephtest 2024-04-25T19:02:25.844 INFO:teuthology.orchestra.run.smithi112.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:02 /home/ubuntu/cephtest 2024-04-25T19:02:25.847 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T19:02:25.859 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T19:02:25.884 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T19:02:25.910 DEBUG:teuthology.nuke:shortname: smithi112 2024-04-25T19:02:25.910 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T19:02:25.930 DEBUG:teuthology.nuke:shortname: smithi161 2024-04-25T19:02:25.930 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T19:02:25.934 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:17:01.681102', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBtLquDmnzWLI7/a0Pegb0caYyJ6TT7NFuELdJvksl2kYlm40L0T+HQg4QZ+1DuhB6/9gYPNADj6KhjNDoQFVhM='} 2024-04-25T19:02:25.952 DEBUG:teuthology.nuke:shortname: smithi171 2024-04-25T19:02:25.953 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T19:02:25.957 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:17:01.682846', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPKokEoAPGlXidTm96brqEZqrR7QVzJqwiJ4hr6x7pZ47ri28vg5eNmjpGnZjoc2B93uxQYFlv/SekHfdTXcBLQ='} 2024-04-25T19:02:25.979 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673614', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:17:01.684937', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLEndUvYqYuYwgvvFy78a1Q6MNAKu2YM9O9qUmz9eFpDVEgxn5VzbENb1+R0iVsfPhQf5IAHvFcJe1lkn9AXEfs='} 2024-04-25T19:02:25.985 INFO:teuthology.orchestra.console.smithi112:Power off 2024-04-25T19:02:25.985 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T19:02:26.009 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-04-25T19:02:26.009 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:02:26.025 INFO:teuthology.orchestra.console.smithi161:Power off 2024-04-25T19:02:26.025 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T19:02:26.035 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-25T19:02:26.048 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off 2024-04-25T19:02:26.048 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:02:26.062 INFO:teuthology.orchestra.console.smithi171:Power off 2024-04-25T19:02:26.062 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T19:02:26.073 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-04-25T19:02:26.085 DEBUG:teuthology.orchestra.console.smithi171:power off output: Chassis Power Control: Down/Off 2024-04-25T19:02:26.085 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:02:26.109 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is on 2024-04-25T19:02:30.036 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:02:30.074 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:02:30.110 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:02:30.147 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-25T19:02:30.185 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-04-25T19:02:30.220 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is on 2024-04-25T19:02:34.152 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:02:34.186 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:02:34.222 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:02:34.264 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-25T19:02:34.296 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2024-04-25T19:02:34.332 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is on 2024-04-25T19:02:34.398 INFO:teuthology.orchestra.console.smithi161:Power off completed 2024-04-25T19:02:38.270 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:02:38.333 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:02:38.380 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-04-25T19:02:38.443 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is off 2024-04-25T19:02:38.481 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-04-25T19:02:38.544 INFO:teuthology.orchestra.console.smithi171:Power off completed 2024-04-25T19:02:38.650 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 2207.61257147789 failure_reason: 'error during quiesce thrashing: Error quiescing set ''5aed9937'': 110 (ETIMEDOUT)' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-25T19:02:38.651 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T19:02:38.741 INFO:teuthology.run:FAIL